CI: run every tests/*.rs integration binary on ubuntu, offline

[Broccolito]

Summary

No workflow ran the integration binaries under crates/*/tests/*.rs. rust.yml's test job selects --lib --bins, and clippy --all-targets only compiles them. So the guards CLAUDE.md treats as load-bearing (the privacy master-switch binaries, approval_no_user_key and knowledge_tier_no_user_key, privacy_ar15_is_retired, every_test_binary_is_sandboxed) ran only when someone remembered to run them. From eb594de until #229, the_documented_closure_is_the_one_the_code_performs scanned the wrong function and would have stayed green with the AR-15 gate deleted.

This PR adds a step to the test job, ubuntu only, that runs every integration target in the workspace except an exclusion table. Each table entry gives its reason inline:

excluded	why
sandbox	The step before it already runs it, on all three OSes.
ui_example_apps	Bundles each example with npx --yes esbuild when no local esbuild exists, and this job installs none. Offline it passes vacuously: npx waits out the bundler's 60 s timeout, then every example falls back to the type-stripper and esbuild never runs. apps-smoke.yml now runs the whole binary after npm ci.

How the step works:

• Exclusion list, not allowlist. A new binary runs the day it lands. new_chat_no_user_key from fix(serve): a new chat starts on the host's configured private model (QA F1) #229 is picked up automatically when fix(serve): a new chat starts on the host's configured private model (QA F1) #229 merges, with no edit here. (Naming it explicitly today would fail: no test target named new_chat_no_user_key.) A table entry for a target that no longer exists fails the step, and so does a target name shared by two packages. I planted both failures against the extracted step to check they fire.
• Never --ignored. The live tests (real providers, vendor CLIs, llama.cpp downloads, GitHub fetches) are #[ignore]d inside binaries whose other tests do run, so those binaries stay in and their live tests stay out.
• No network, enforced. The tests run under sudo unshare --net, which leaves only loopback, then setpriv back to the runner user (as root, every "permission denied" assertion would pass for the wrong reason). A test that dials out fails immediately with a connection error. Before trusting it, I ran a positive control on the hosted runner: uid is non-zero inside, https://example.com is refused, and a 127.0.0.1 server answers.
• --workspace --test <name>, not -p <pkg>. A -p selection changes cargo's feature unification and rebuilt dependencies when I measured it. --workspace reuses what the lib+bins step built (0.6 s, nothing rebuilt).
• Keyring off (BIOROUTER_DISABLE_KEYRING=true), like the existing step, and CARGO_NET_OFFLINE=true.

A required side change: ubuntu test debuginfo

With ubuntu's default debug=2, building the integration binaries ran the runner out of disk: 70 GB free fell to 1.4 GB, then rustc-LLVM ERROR: IO failure on output stream: No space left on device (run 34628149105). On Linux, every test executable embeds the DWARF of everything it links. I measured that cargo test builds dependencies with the test profile (flipping only CARGO_PROFILE_TEST_DEBUG set cargo recompiling the whole dependency graph: 515 crates in the first 25 s, when I stopped it). So the variable has to be job-wide; set on the new step alone, it would rebuild every dependency. It now applies to ubuntu as well as windows/macOS. With it, the integration executables total 14.5 GiB and the runner keeps 65 GB free.

⚠ Cost: ubuntu's rust-cache key changes, so ubuntu builds cold until main saves a new cache. That includes this PR's own runs. Measured: the lib+bins build goes from 3.7 to 7.5 min. The workflow comment that deliberately kept ubuntu out ("to fix a problem ubuntu does not have") is rewritten, because ubuntu now has the problem.

Measurements

Each binary run on its own, with a per-binary timeout, keyring off. Scratch runs: 34628149105 (all OSes, then-current debuginfo settings) and 34629544896 (ubuntu, debuginfo 0, offline).

	lib+bins build	integration build	executables	binaries green	tests passed / failed / ignored	time in tests
ubuntu, offline (debug 0, cold)	451 s	135 s	14.5 GiB	119 / 119	687 / 0 / 49	197 s (137 s for the 117 selected)
ubuntu, debug 2 (warm)	222 s	❌ disk full	—	—	—	—
macOS (cache nearly empty)	933 s	430 s	12.9 GiB	118 / 119	686 / 1 / 49	178 s
windows (warm)	463 s	383 s	10.3 GiB	110 / 119	645 / 31 / 49	241 s
local macOS, offline (Seatbelt)	—	—	—	119 / 119	687 / 0 / 49	126 s

Flakiness: I ran the step's exact command locally 6 times (3 at default threads, 3 at RUST_TEST_THREADS=32, the stressor that exposed this repo's shared-fake races before). All 117 binaries passed every time, 126–141 s per pass.

On time: the step should add about 5 min to test (ubuntu-latest) warm (135 s build plus 137 s of tests plus process startup), against today's 12–19 min and a 40-min cap. macOS (~20 min) sets the workflow's wall clock today.

Why ubuntu only

• windows-latest: 31 tests fail in 9 binaries, none of which had ever run on Windows. They look like real Windows defects, not runner noise:
  • session_store_dispatch_boundary (2): "a script read every conversation on this machine, with no inspector anywhere". The Feature request: privacy tiers — keep private conversations away from externally hosted models #56 session-store guard for execute_code does not appear to hold on Windows. Worth a look on its own.
  • code_execution_integration (6): a Windows path is spliced into JS unescaped (\b became a backspace).
  • hooks_agent_loop_tests (9), hooks_integration_tests (7), global_memory_consent_agent_loop (2), loop_safety_observability_tests (1): Tool 'developer__shell' not found, the same shape as the already-known open Windows bridge failures.
  • parallel_tool_batch_concurrency_cap, _serialization_lever, _stress (4): the tool-concurrency cap did not bound the batch (18 concurrent against a ceiling of 8, and 7 with BIOROUTER_TOOL_MAX_CONCURRENT=1).
• macOS passed except output_recovery_agent_loop::private_provider_error_logging_retry_keeps_details_out_of_diagnostics, which failed once on the CI runner. It passed 7 times locally, 60 more times when hammered (40 at 8/32 threads, 20 throttled with taskpolicy -b), and on ubuntu and Windows. My unconfirmed guess is a log-capture race, since it scopes its tracing subscriber to one future with .with_subscriber(). It could still flake on ubuntu at a low rate.
• unshare is Linux-only. Widening to another OS needs its failures fixed, and its own way to stay offline (Seatbelt works on macOS; I used it for the local run).

Also in this PR

• apps-smoke.yml runs the whole ui_example_apps binary (--include-ignored) instead of filtering by name to the Chromium test. The binary's other five tests ran in no workflow before.
• Two stale rust.yml comments are fixed. One claimed the sandbox target was the only thing outside --lib/--bins. The other claimed tests/ needed cassettes and credentials; in fact mcp_integration_test replays its cassettes offline, and credentialed tests are #[ignore]d.
• CLAUDE.md's Testing section now says what CI runs.

Not in this PR

• Fixing the Windows failures above.
• A pre-existing hazard: the --lib step's build_app_bundles_with_available_toolchain goes through the same npx --yes esbuild fallback on runners, so it probably already downloads esbuild from npm on every run. I haven't verified this in a CI log.
• A measurement artifact worth knowing about: locally, find_esbuild() walks six parent directories and finds the main checkout's ui/desktop/node_modules/.bin/esbuild from inside a .claude/worktrees/* worktree, so ui_example_apps passes locally for a reason CI never has.

Test plan

• Scratch CI measurement on all three OSes, plus ubuntu offline with debuginfo 0 (links above)
• Local: every binary offline, then the step's exact command 6× (default and 32 threads)
• Step logic, extracted from the YAML with sudo stubbed: selects 117 binaries (including all 10 privacy_* binaries, both *_no_user_key and every_test_binary_is_sandboxed); a planted stale entry exits 1; a planted duplicate name exits 1
• This PR's test (ubuntu-latest) runs the new step green (run 34631673813). From the job log: both exclusions printed, running 117 integration binaries, build 2m08s, 117 test result: ok, 0 failed, step total 4m33s. The whole ubuntu job took 22 min on this cold-cache first run (budget 40; lib+bins went from 9.1 to 11.7 min cold).
• reference-apps (apps-smoke) runs the whole ui_example_apps binary green
• test (windows-latest): the first attempt failed in the lib step on routes::agent::knowledge_selection_tests::applying_a_workflow_hides_a_base_that_lands_mid_call (git NotFound under the Windows temp dir). That's a known Windows flake: it hit renderer-only Chat history on the chat measure (760px) #175 the same way, and nothing in this diff runs on Windows beyond the unchanged debuginfo setting. The rerun passed.

🤖 Generated with Claude Code