test(privacy): the census row for the workspace_set_tools pre-flight's capability sample - #258
Conversation
…I last ran it `the_sites_that_decide_how_far_a_caller_reaches_are_exactly_these` is red on main (run 34649064272): two sites were added while rust.yml still ran only `--lib --bins`, so both were green when they merged and neither is described in the census. - `agents/workspace_inspector.rs` 1 -> 2 (added by e60213c): the `workspace_set_tools` pre-flight in `WorkspaceMutationInspector`, which asks `set_tools_preflight_refusal` whether the change CAN be made before the user is asked to approve it, and so needs the caller's pair to ask the tier gate with. It samples lazily — only once a `set_tools` call is in the batch — and memoises it, and like its neighbour it prefers a capability threaded in and samples only when handed `None`. The row now states that invariant, which is the property this census exists to protect. - `providers/coding_agent/bridge.rs` 1 -> 2 (added by 2f61d27): a second TEST fixture, the module-level `inert_grant_for_test()`. It cannot call `mod tests`' private `test_capability()` and cannot itself live in `mod tests`, being `pub(crate)` for other modules' tests — so the file's "one named helper" intent now costs two occurrences. The row says so; collapsing them is the bridge authors' call. Neither site is wrong, so neither is removed. No production code changes.
… alone My first commit also took `providers/coding_agent/bridge.rs` to count 2. That was wrong to ship: #228 is removing the extra site instead (its `inert_grant_for_test()` inlined the constructor; the helper now takes its pair from the file's documented `tests::test_capability()`), which returns that file to one occurrence and leaves the existing row, its count and its prose true. A row of 2 here would turn main red again the moment their fix lands. What remains is one row for one file: `agents/workspace_inspector.rs`, 1 -> 2. Extending the existing row rather than adding a second is forced by the test — `found` is keyed by (needle, file) and summed, `want` is one tuple per row with no aggregation, so two rows for one file fail even at count 1 each. The census mechanism is untouched. The row now states what I read in the code rather than what the pre-flight is called: both inspectors funnel into `inspect_with_pinned_capability`, whose `let mut sampled = capability` uses a pinned pair as is and never reads the provider mutex (only the coding-agent bridge pins one; the agent loop and the approval relay pass None); the sample runs lazily inside the `is_set_tools_call` arm and memoises, so a batch with no such call samples nothing; and the pair is handed to `preflight_set_tools`, which asks the §7 write row, Gate F reach for added extensions, the removals' manageability refusal and `bind_allowed` for a provider switch. It is not the gate — `handle_set_tools` re-runs the same pre-flight against the admitted capability, so a stale sample fails safe. On this branch alone the census still fails, with exactly the bridge.rs delta that #228's PR removes. Green needs both halves.
Broccolito
left a comment
There was a problem hiding this comment.
Review from the #228/#259 side, since the bridge half of this census is mine. I checked every claim in the workspace_inspector.rs row against the code at d6b39693 rather than reading it for plausibility. The row is accurate and the count is right — one wording correction below, and it is worth making because the census's whole value is that its prose can be trusted without re-deriving it.
Verified
| Claim | Evidence at d6b39693 |
|---|---|
| exactly two sites, both non-comment | workspace_inspector.rs:392 and :744 — I ran the census's own rule (skip lines whose trimmed start is //, count occurrences): 2 |
| (1) crossing = first-crossing disclosure, sampled once per batch | :741-744, one match before the per-candidate loop |
| (1) samples only after a cheap name check | :735-737 if candidates.is_empty() { return Ok(Vec::new()); } precedes it |
(2) mutation = set_tools pre-flight (F4), lazy, memoised, one model for two calls in a batch |
:389-395 inside the is_set_tools_call arm, sampled = Some(cap) |
| neither inherits on the ordinary agent loop | both ToolInspector::inspect impls pass None (:469, :828) |
| only the coding-agent bridge pins a pair | inspect_with_capability forwards it verbatim (:480-490); the only non-None caller of inspect_tools_with_capability is bridge.rs:511, while tool_inspection.rs:103 passes None |
| a pinned pair is used as is, mutex never read | true of both — (1) match capability, (2) the memo seeded from it |
Both sites are legitimate deciders on the ordinary path, and (2)'s "it is NOT the gate — handle_set_tools re-runs the same pre-flight on the admitted capability, so staleness is fail-safe" is the right thing to have written down.
One correction
⚠ Both funnel into
inspect_with_pinned_capability, whose first act islet mut sampled = capability
That is true of (2) only. let mut sampled = capability is :367, in WorkspaceMutationInspector. WorkspaceCrossingInspector has no sampled binding at all — the single occurrence of that word in its body is the comment at :739 — and it consumes the pinned pair at :741-744:
let cap = match capability {
Some(capability) => capability,
None => crate::privacy::CallCapability::sample(&self.provider).await,
};deliberately after the candidates.is_empty() early return, which is the same cheap name check the row credits it with two sentences earlier. So the sentence as written contradicts (1)'s own description, and a later reader grepping sampled in the crossing inspector will not find it.
The property you are asserting holds for both; only the mechanism differs. Something like:
⚠ Both take the pinned pair AS IS and never read the provider mutex when given one — (1) by
match capabilityat the point of use, (2) by seeding itssampledmemo from it (let mut sampled = capability). The only caller that pins one is …
On the dropped bridge.rs hunk
Right call, and thank you for turning it around so fast. For the record, your reasoning there was sound on what you could see: inert_grant_for_test() is pub(crate) and could not reach mod tests' private test_capability(). #259 makes that helper pub(super), so the fixture now calls it and the file collapses back to one occurrence — which is why that row stays true exactly as it was, untouched.
With the hunk gone the two PRs are independent in both directions: this one touches only the census file, #259 only bridge.rs. Either merge order is safe.
…third outcome Review of #258 caught two prose inaccuracies, both checked against the code: the ⚠ sentence attributed `let mut sampled = capability` to BOTH inspectors, but the crossing inspector has no such binding — it matches the `Option` once at its sample point, after its early return; and the pre-flight has a third outcome besides Deny and the card, namely no card-worthy reason and on to dispatch. The property the census protects is unchanged; only the mechanism each row names is now greppable.
|
Review from the author of the change this row describes (#244, the Narrowing to The tagged question: is sampling here intended, or should the pair have been threaded from further up?Intended — and threading it from further up would be a regression rather than a cleanup. The agent loop's authoritative tool-call sample lives inside Keeping the inspector's sample separate and non-authoritative is what makes its staleness fail safe: Two precision fixes — optional, neither blocks a mergeThis file's value is that a later reader can grep what a row claims, which is the only reason these are worth raising:
Re-checked and holdingThe funnel through Independent measurement of
|
|
Measured: #258 + #259 together make the census green. Neither is green alone, so this is the only state that can be verified before merging. Scratch branch = Row state in the combined tree — one row per file, which the census requires since
⚠ Expect each PR's own |
What is broken
privacy_capability::the_sites_that_decide_how_far_a_caller_reaches_are_exactly_theseis red onmain(run 34649064272,d6b39693), so every PR that mergesmaininherits it. Two sites that decide how far a caller reaches were added whilerust.ymlstill ran only--lib --bins; #239 (merged 20:49) made CI runtests/*.rs, and the census has been red since.This PR is one half of the fix —
agents/workspace_inspector.rs. The other half is #259 (fix/census-bridge-test-capability), which removesbridge.rs's extra site rather than describing it.maingoes green only when both land, and the two are disjoint — #259 never touches the census file, so either merge order is safe.Measured on the combination (this branch with #259 merged locally):
On this branch alone the census still fails, with exactly one delta and nothing else —
bridge.rsCallCapability::public_enforced(found 2 against a row of 1 — which is #259's half. That failure is expected here.I deliberately do not add a
bridge.rsrow: #259 takes that file back to one occurrence, so a row of 2 here would re-breakmainthe moment it lands. ⚠ Note for anyone checking by hand: the census skips comment lines, so a plaingit grep -cover-counts that file by one —bridge.rs:2490is a doc comment naming the constructor, not a site.The change
agents/workspace_inspector.rs,CallCapability::sample(:count: 1→2, with the row's prose rewritten to describe both sites. No production code. The census mechanism is untouched.Extending the existing row rather than adding a second is forced by the test:
foundis aBTreeMapkeyed by(needle, file)with+= hits,wantis one tuple perEXPECTEDrow with no aggregation, and the comparison is a flatassert_eq!— so two rows for one file fail even atcount: 1each. The file's convention agrees (agent.rsis one row at 5).Should the second site exist? Yes
An inspector runs before the dispatch that would admit a capability, so there is none in scope to inherit on the ordinary agent loop — the same reason already written for its sibling in this file. The alternatives are worse: read the master toggle directly (the bug
CallCapabilityexists to prevent), or hoist the privacy gates out of the pre-flight so the always-confirm card is raised without them. And it is not the gate:handle_set_toolsre-runs the same pre-flight against the capability the call is finally admitted on, so a model swapped between inspection and dispatch can only make this sample stale in the fail-safe direction. Nothing is removed.What I verified by reading the code (please still check me)
let mut sampled = capabilityand memoises into it; the crossing inspector has nosampledbinding at all and matches theOptiononce at its sample point, after itscandidates.is_empty()early return.sample(runs only inside theis_set_tools_callarm, only when nothing was pinned: at most one sample per batch, zero for a batch with noworkspace_set_toolscall.bridge.rs:815);tool_inspection.rs's non-capability entry point passesNone, as do the agent loop and the approval relay.set_tools_preflight_refusal→preflight_set_tools, which asksrefuse_unless_writable(§7's write row),resolve_added_extensions(Gate F reach per added extension),preflight_extension_removals(manageability per removed one) andprivacy::bind_allowed(a provider switch).handle_set_toolscallspreflight_set_tools(caller_session_id, cap, &args)at dispatch — the re-run above.bridge.rs's occurrences are#[cfg(test)]helpers, never production deciders. With Census: the bridge's inert test grant takes the file's one capability pair (#228 follow-up) #259 in, the census counts one there (:2513), matching its unchanged row; a plain grep also reports the doc comment at:2490, which the census skips.Previously tagged as inferred, now answered by #244's author and checkable in the tree: sampling here is intended, and threading the pair from further up would be a regression. The agent loop's authoritative sample lives inside
dispatch_tool_call(agent.rs, "THE sample for the agent loop's tool calls"), which runs after inspection and after any approval; the loop's inspection entries pass no capability. Hoisting that read to before inspection would fix the pair before an always-confirm card that can park for minutes, so a model swapped while the card sat parked would be ignored — the fail-open direction. Keeping this inspector's sample separate and non-authoritative is what makes its staleness fail safe, which is why the row says it is not the gate. The bridge is the one case where fixing the pair earlier is right, for the opposite reason: its child's calls arrive from another process with nothing to inherit.Verification
⚠ This may not turn the ubuntu check green even once both halves land: the job's earlier
--lib --binsstep has been failing intermittently onagent_drafter::bundle::tests::a_timed_out_esbuild_reaps_its_whole_process_group(a process-reaping race inbiorouter-mcp; it passes onmain), and when it does, the integration step never runs.🤖 Generated with Claude Code