Skip to content

Mask provider secrets; a coding agent without its CLI is not Configured (QA F5, F6) - #225

Open
Broccolito wants to merge 4 commits into
mainfrom
claude/serene-satoshi-844a47
Open

Mask provider secrets; a coding agent without its CLI is not Configured (QA F5, F6)#225
Broccolito wants to merge 4 commits into
mainfrom
claude/serene-satoshi-844a47

Conversation

@Broccolito

@Broccolito Broccolito commented Sep 11, 2026

Copy link
Copy Markdown
Collaborator

Fixes two MEDIUM findings from the 2026-09-10 provider QA run on merged main (7c96d796), report ~/biorouter-runs/test-drive/qa-e/report.md §F5 and §F6.

Update: merged origin/main (6455bc21)

main moved 60 commits. The only conflict was SwitchModelModal.tsx, against #222 ("the model switcher refuses a barred selection before the click", F3). The file now keeps both pre-flights:

The other overlapping files merged automatically. config_management.rs only picked up the master-switch changes from #227 and the config-write fix from #223; ProviderDetails gained nothing else, so the test literals still compile. CLAUDE.md, openapi.json and types.gen.ts merged cleanly, and regenerating the schema and client from the merged tree reproduces those files exactly (below).

One new assertion pins the merged contract: when the dialog opens on an unavailable provider, the confirm's aria-describedby names the provider-field reason, as #222 requires for its own refusal.

Re-verification on the merge commit

$ BIOROUTER_DISABLE_KEYRING=true cargo test -p biorouter-server --lib routes::utils routes::coding_agents routes::config_management
error: unexpected argument 'routes::coding_agents' found

Usage: cargo test [OPTIONS] [TESTNAME] [-- [ARGS]...]

For more information, try '--help'.
EXIT=1

The brief's command still stops at cargo's argument parser (one positional TESTNAME); the same filters after --:

$ BIOROUTER_DISABLE_KEYRING=true cargo test -p biorouter-server --lib -- routes::utils routes::coding_agents routes::config_management
    Finished `test` profile [unoptimized + debuginfo] target(s) in 10m 14s
     Running unittests src/lib.rs (target/debug/deps/biorouter_server-a0b457219cca8ef3)

running 27 tests
test routes::config_management::affiliation_wire_tests::an_unresolved_tier_is_a_rendered_null ... ok
test routes::config_management::readiness_wire_tests::a_row_with_nothing_to_explain_serialises_an_explicit_null ... ok
test routes::config_management::tests::a_recovery_that_could_not_write_says_so_in_the_sentence_and_in_the_flag ... ok
test routes::config_management::affiliation_wire_tests::a_provider_with_no_affiliation_serialises_an_explicit_null ... ok
test routes::config_management::affiliation_wire_tests::the_affiliation_travels_beside_the_metadata_not_inside_it ... ok
test routes::config_management::affiliation_wire_tests::the_resolved_tier_can_disagree_with_the_type_level_one ... ok
test routes::config_management::tests::a_config_that_loads_but_cannot_be_written_is_not_called_in_memory_only ... ok
test routes::config_management::tests::a_recovery_that_persisted_carries_no_warning ... ok
test routes::config_management::affiliation_wire_tests::the_resolved_tier_travels_beside_the_metadata_under_its_own_key ... ok
test routes::config_management::tests::a_recovery_with_nothing_to_recover_still_reports_that_it_could_not_write ... ok
test routes::config_management::tests::persisted_is_exactly_the_absence_of_a_write_error ... ok
test routes::config_management::affiliation_wire_tests::a_row_without_the_field_still_reads ... ok
test routes::utils::tests::the_cli_requirement_applies_to_the_coding_agents_only ... ok
test routes::config_management::tests::test_read_model_limits ... ok
test routes::config_management::readiness_wire_tests::a_codex_row_whose_cli_is_missing_is_unconfigured_and_says_why ... ok
test routes::utils::tests::a_coding_agent_whose_cli_is_missing_is_not_configured ... ok
test routes::config_management::privacy_disclosure_tests::the_acknowledgement_is_recorded_once_and_is_not_agent_writable ... ok
test routes::config_management::tests::a_recovery_after_the_config_was_healed_reports_persisted_with_no_note ... ok
test routes::config_management::tests::detectable_providers_route_lists_known_providers ... ok
test routes::config_management::tests::pricing_endpoint_uses_shared_resolver_and_exposes_cache_rates ... ok
test routes::config_management::readiness_wire_tests::a_codex_row_whose_cli_resolves_is_configured_with_no_reason ... ok
test routes::utils::tests::a_coding_agent_whose_cli_resolves_is_configured ... ok
test routes::utils::tests::not_configured_is_exactly_what_the_status_route_calls_not_installed ... ok
test routes::config_management::privacy_disclosure_tests::the_route_serves_the_one_copy_rather_than_a_second_one ... ok
test routes::config_management::tests::each_recovery_describes_the_config_as_it_is_now ... ok
test routes::coding_agents::tests::status_reports_every_agent_even_when_none_is_installed ... ok
test routes::coding_agents::tests::the_response_never_carries_a_token ... ok

test result: ok. 27 passed; 0 failed; 0 ignored; 0 measured; 580 filtered out; finished in 0.44s

EXIT=0

Schema and client regenerated from the merged tree. git status stays clean, so the auto-merged openapi.json and types.gen.ts are exactly what the generator produces:

$ cargo run -p biorouter-server --bin generate_schema && (cd ui/desktop && npm run generate-api) && git status --short -- ui/desktop/openapi.json ui/desktop/src/api
(no output)
$ cd ui/desktop && npm run test:run
(tail)
 FAIL  src/utils/artifactCdnAssets.browser.test.ts > a CDN-mode Mermaid figure in a real browser
 Test Files  1 failed | 446 passed (447)
      Tests  5020 passed | 1 skipped (5021)
   Start at  14:16:51
   Duration  90.84s (transform 25.89s, setup 178.22s, import 204.20s, tests 279.26s, environment 559.69s)
EXIT=1

The one failed suite is not a failed test, and it is not this branch. src/utils/artifactCdnAssets.browser.test.ts passes both of its tests, then its afterAll (await browser?.close()) hits the 30 s hook timeout. It does the same run on its own (2 passed, Hook timed out in 30000ms). The machine's load average was 355–436 at the time, from other sessions. This PR does not touch that file or anything it imports (git diff --name-only origin/main...HEAD | grep -c artifactCdnAssets0). Of my three full runs earlier today, it passed in two; the third hit the same teardown timeout while a Rust build ran beside it. CI's Unit tests (vitest) passed on this PR's previous head.

$ npm run lint:check
(excerpt: every step header and verdict)
> biorouter-app@1.90.3 lint:check
> npm run typecheck && eslint "src/**/*.{ts,tsx}" --max-warnings 0 --no-warn-ignored && npm run check:themes && npm run check:contrast && npm run check:tokens
> biorouter-app@1.90.3 typecheck
> tsc --noEmit
> biorouter-app@1.90.3 check:themes
> node scripts/generate-themes.mjs --check
OK — generated artifacts are current (3 themes)
> biorouter-app@1.90.3 check:contrast
> node scripts/check-contrast.mjs
OK — all 332 contrast assertions pass
> biorouter-app@1.90.3 check:tokens
> node scripts/check-token-mirrors.mjs
OK — every semantic colour token used by a utility has a @theme inline mirror
     (341 declared, 91 mirrored, 57 reached from a utility)
EXIT=0
$ npx prettier --check src/api/types.gen.ts src/components/onboarding/codingAgentControls.tsx src/components/settings/models/subcomponents/SwitchModelModal.tsx src/components/settings/models/subcomponents/SwitchModelModal.unavailable.test.tsx src/components/settings/providers/ProviderCatalog.test.tsx src/components/settings/providers/ProviderCatalog.tsx src/components/settings/providers/modal/ProviderConfiguationModal.tsx src/components/settings/providers/modal/subcomponents/forms/CustomProviderForm.tsx src/components/settings/providers/modal/subcomponents/forms/DefaultProviderSetupForm.test.tsx src/components/settings/providers/modal/subcomponents/forms/DefaultProviderSetupForm.tsx src/components/settings/providers/modal/subcomponents/handlers/DefaultSubmitHandler.tsx src/components/ui/secret-input.tsx 
Checking formatting...
All matched files use Prettier code style!
EXIT=0

F5: provider secrets were typed and shown in cleartext

Cause. DefaultProviderSetupForm.tsx (the form every built-in provider shares) hard-coded type="text" for every parameter, secrets included, with spellcheck on. It already read parameter.secret, but only to avoid echoing a stored value. CustomProviderForm.tsx masked its key. The two had drifted apart; nobody decided on it.

Fix.

  • New primitive ui/secret-input.tsx. It stays type="password" until an explicit reveal toggle is pressed. It forces autoComplete="off" and spellCheck={false} after the caller's props, so a caller can't switch either back on. It starts masked on every mount.
  • DefaultProviderSetupForm renders secret parameters with it. Non-secret ones (endpoint, region) stay readable type="text" fields. Labels are now associated with their fields.
  • CustomProviderForm's key uses the same primitive, so both forms mask and reveal a key the same way.

F6: a provider with a missing CLI was badged "Configured" and stayed selectable

Cause. routes/utils.rs treated a coding agent's one required, defaulted key as "configured" whenever it was saved. It never checked whether the command resolved. So CODEX_COMMAND=/nonexistent produced "Codex · Not installed ✓ Configured", and the picker offered a provider whose bind would then fail in from_env.

Fix (daemon).

  • check_provider_configured now asks discovery::resolve_configured for the coding agents. That is a few stat calls, never a spawn, and it is the same lookup /coding_agents/status uses for "Not installed". The badge and the pill now agree by construction.
  • Rows in GET /config/providers carry a new unavailable_reason. It is set only when a provider the user set up cannot run; today that means a coding agent whose saved command doesn't resolve. The text is CodingAgentKind::not_installed_summary, which is also the first sentence of the error a turn would have raised.
  • Sign-in is deliberately not part of "configured" ("installed" is the minimum the brief allowed). Checking it means spawning the CLI, and this route runs for every provider on every settings open, which discovery.rs's module header forbids. The pill shows sign-in state, and a turn that reaches a signed-out CLI fails with the vendor's own login command.

Fix (renderer).

  • SwitchModelModal keeps such a provider in its list, disabled with the reason on the row. It uses react-select's own aria-disabled option with the reason as the detail line, the same shape as the private-chat refusal, applied to a provider instead of a model. If the dialog opens on such a provider (the bound one), it shows the reason and won't submit.
  • The catalog re-reads the provider list after an explicit "Check again". It also re-runs the CLI probe after any change to an agent's setup. The runtime check showed this second part is needed: correcting CODEX_COMMAND in the configure form brought back the ✓ check next to a pill that still said "Not installed", which is F6's contradiction arriving from the other direction.
  • The configure modal still offers "Remove" for a saved key whose CLI is missing, so that key isn't stranded.
  • The inline card's guidance is unchanged.

openapi.json and the TS client are regenerated; the only change is the new field.

Tests: each new case fails on main, and each has a control

Test Against main With fix
routes::utils::tests::a_coding_agent_whose_cli_is_missing_is_not_configured (route logic) panics: Codex pointed at …/no-such-dir/codex must not report is_configured (EXIT=101) ok
routes::config_management::readiness_wire_tests::* (the row the route serves) n/a (field is new) ok, plus a control that a resolving CLI is configured with no reason
routes::utils::tests::not_configured_is_exactly_what_the_status_route_calls_not_installed n/a ok: probe() says NotInstalled exactly when the check says not configured
DefaultProviderSetupForm.test.tsx: secret renders masked Expected the element to have attribute: type="password" / Received: type="text" ok
same file: non-secret stays readable (control) passes passes
SwitchModelModal.unavailable.test.tsx: Codex row disabled with reason Unable to find role="option" and name /Codex/ ok (aria-disabled="true" + reason); Versa stays aria-disabled="false"; never-configured OpenAI still omitted
same file: opens on an unavailable provider Unable to find … switch-model-provider-error ok, button disabled, changeModel never called
ProviderCatalog.test.tsx: "Check again" finds the CLI gone, check disappears the check stays ok
same file: correcting the key in the configure form re-probes Unable to find … Ready · signed in on your subscription ok

The F5 test finds fields by placeholder, so it runs unchanged against main's form, and the assertion that fails there is the masking assertion itself, not the lookup.

Verification (on the original head, 0873d12d)

⚠ The Rust command in the brief doesn't run. Cargo accepts only one positional TESTNAME:

$ BIOROUTER_DISABLE_KEYRING=true cargo test -p biorouter-server --lib routes::utils routes::coding_agents routes::config_management
error: unexpected argument 'routes::coding_agents' found

Usage: cargo test [OPTIONS] [TESTNAME] [-- [ARGS]...]

For more information, try '--help'.
EXIT=1

The equivalent with the filters passed to libtest:

$ BIOROUTER_DISABLE_KEYRING=true cargo test -p biorouter-server --lib -- routes::utils routes::coding_agents routes::config_management
    Finished `test` profile [unoptimized + debuginfo] target(s) in 1m 01s
     Running unittests src/lib.rs (target/debug/deps/biorouter_server-a0b457219cca8ef3)

running 24 tests
test routes::config_management::tests::a_recovery_that_persisted_carries_no_warning ... ok
test routes::config_management::tests::a_recovery_with_nothing_to_recover_still_reports_that_it_could_not_write ... ok
test routes::config_management::tests::a_recovery_that_could_not_write_says_so_in_the_sentence_and_in_the_flag ... ok
test routes::config_management::affiliation_wire_tests::an_unresolved_tier_is_a_rendered_null ... ok
test routes::config_management::readiness_wire_tests::a_row_with_nothing_to_explain_serialises_an_explicit_null ... ok
test routes::config_management::tests::persisted_is_exactly_the_absence_of_a_write_error ... ok
test routes::config_management::affiliation_wire_tests::a_provider_with_no_affiliation_serialises_an_explicit_null ... ok
test routes::config_management::affiliation_wire_tests::the_resolved_tier_travels_beside_the_metadata_under_its_own_key ... ok
test routes::config_management::affiliation_wire_tests::the_affiliation_travels_beside_the_metadata_not_inside_it ... ok
test routes::config_management::affiliation_wire_tests::the_resolved_tier_can_disagree_with_the_type_level_one ... ok
test routes::config_management::affiliation_wire_tests::a_row_without_the_field_still_reads ... ok
test routes::config_management::readiness_wire_tests::a_codex_row_whose_cli_is_missing_is_unconfigured_and_says_why ... ok
test routes::config_management::tests::test_read_model_limits ... ok
test routes::utils::tests::a_coding_agent_whose_cli_is_missing_is_not_configured ... ok
test routes::utils::tests::a_coding_agent_whose_cli_resolves_is_configured ... ok
test routes::utils::tests::not_configured_is_exactly_what_the_status_route_calls_not_installed ... ok
test routes::config_management::privacy_disclosure_tests::the_acknowledgement_is_recorded_once_and_is_not_agent_writable ... ok
test routes::config_management::tests::pricing_endpoint_uses_shared_resolver_and_exposes_cache_rates ... ok
test routes::config_management::tests::detectable_providers_route_lists_known_providers ... ok
test routes::config_management::readiness_wire_tests::a_codex_row_whose_cli_resolves_is_configured_with_no_reason ... ok
test routes::utils::tests::the_cli_requirement_applies_to_the_coding_agents_only ... ok
test routes::config_management::privacy_disclosure_tests::the_route_serves_the_one_copy_rather_than_a_second_one ... ok
test routes::coding_agents::tests::the_response_never_carries_a_token ... ok
test routes::coding_agents::tests::status_reports_every_agent_even_when_none_is_installed ... ok

test result: ok. 24 passed; 0 failed; 0 ignored; 0 measured; 569 filtered out; finished in 0.29s

EXIT=0

The core crate's coding-agent tests (the discovery and error-text changes live there):

$ BIOROUTER_DISABLE_KEYRING=true cargo test -p biorouter --lib -- providers::coding_agent
    Finished `test` profile [unoptimized + debuginfo] target(s) in 3m 15s
     Running unittests src/lib.rs (target/debug/deps/biorouter-3f7447b259284cd8)
running 134 tests
… (127 further `providers::coding_agent::*` lines, all `ok`) …
test providers::coding_agent::discovery::tests::auth_state_only_reports_subscription_for_the_subscription_variant ... ok
test providers::coding_agent::discovery::tests::blank_configuration_falls_back_to_the_default_command ... ok
test providers::coding_agent::discovery::tests::from_provider_id_inverts_provider_id_and_nothing_else ... ok
test providers::coding_agent::discovery::tests::provider_ids_match_the_pricing_block_list ... ok
test providers::coding_agent::discovery::tests::the_claude_label_is_the_one_that_was_chosen ... ok
test providers::coding_agent::discovery::tests::the_not_installed_summary_is_the_errors_first_sentence ... ok
test providers::coding_agent::discovery::tests::an_explicit_path_bypasses_the_search_path ... ok
test result: ok. 134 passed; 0 failed; 0 ignored; 0 measured; 3664 filtered out; finished in 7.56s
EXIT=0
$ cd ui/desktop && npm run test:run
(tail of the run; the full per-file list is 444 lines)
 Test Files  444 passed (444)
      Tests  4982 passed | 1 skipped (4983)
   Start at  01:59:54
   Duration  55.58s (transform 23.49s, setup 110.21s, import 132.53s, tests 151.21s, environment 348.64s)
EXIT=0
$ npm run lint:check
(excerpt: every step header and every verdict line; the 332-row contrast table is omitted)
> biorouter-app@1.90.3 lint:check
> npm run typecheck && eslint "src/**/*.{ts,tsx}" --max-warnings 0 --no-warn-ignored && npm run check:themes && npm run check:contrast && npm run check:tokens
> biorouter-app@1.90.3 typecheck
> tsc --noEmit
> biorouter-app@1.90.3 check:themes
> node scripts/generate-themes.mjs --check
OK — generated artifacts are current (3 themes)
> biorouter-app@1.90.3 check:contrast
> node scripts/check-contrast.mjs
OK — all 332 contrast assertions pass
> biorouter-app@1.90.3 check:tokens
> node scripts/check-token-mirrors.mjs
OK — every semantic colour token used by a utility has a @theme inline mirror
     (341 declared, 91 mirrored, 57 reached from a utility)
EXIT=0
$ npx prettier --check src/api/types.gen.ts src/components/onboarding/codingAgentControls.tsx src/components/settings/models/subcomponents/SwitchModelModal.tsx src/components/settings/models/subcomponents/SwitchModelModal.unavailable.test.tsx src/components/settings/providers/ProviderCatalog.test.tsx src/components/settings/providers/ProviderCatalog.tsx src/components/settings/providers/modal/ProviderConfiguationModal.tsx src/components/settings/providers/modal/subcomponents/forms/CustomProviderForm.tsx src/components/settings/providers/modal/subcomponents/forms/DefaultProviderSetupForm.test.tsx src/components/settings/providers/modal/subcomponents/forms/DefaultProviderSetupForm.tsx src/components/settings/providers/modal/subcomponents/handlers/DefaultSubmitHandler.tsx src/components/ui/secret-input.tsx 
Checking formatting...
All matched files use Prettier code style!
EXIT=0
$ cargo clippy -p biorouter -p biorouter-server --all-targets -- -D warnings
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 2m 39s
EXIT=0

Runtime check (own sandboxed instance, CDP 9389, never set_viewport)

The instance ran on this branch's biorouterd (with the test-auth seam) and renderer, in the sandbox ~/biorouter-runs/f5f6-codex-secrets cloned from the seed. Screenshots are in ~/biorouter-runs/f5f6-codex-secrets/shots/:

  1. 01-f5-bedrock-secret-masked.png: Versa Bedrock configure card with a dummy value typed into Secret Access Key, shown as dots. Live DOM: both Bedrock secrets type=password, autocomplete=off, spellcheck=false, labelled. The reveal toggle flips password → text → password. Endpoint and region stay readable. Cancelled, never saved (the sandbox secrets.yaml has no Bedrock keys).
  2. 02-f6-codex-row-no-configured.png: with CODEX_COMMAND: /nonexistent in the sandbox config, the row reads "Codex · Not installed" with no ✓ Configured. Claude Code keeps its check. The inline guidance is unchanged.
  3. 03-f6-picker-codex-disabled.png: Settings → Switch models. Codex is aria-disabled="true" with "Unavailable: Codex is not installed, or is not on a path Biorouter searches". Every other row is enabled. Clicking the disabled row selects nothing.
  4. 04-f6-codex-after-correcting-path.png: CODEX_COMMAND corrected to codex through the Configure form. Pill and check both return together ("Ready · signed in on your subscription ✓ Configured").

Afterwards CODEX_COMMAND was back to codex in the sandbox, BIOROUTER_PROVIDER was still versa_azure, and the instance was stopped. The QA instances (CDP 9371–9376, ~/biorouter-runs/qa-merged-*) were never touched, and no process was killed by pattern.

Not in this PR

  • Sign-in state stays out of is_configured, for the reason above.
  • LeadWorkerSettings and IngestModelPicker list only configured providers, so a not-installed agent drops out of them rather than showing disabled. The brief named the Switch-models list.
  • A few other key fields (ElevenLabsKeyInput, ExternalBackendSection) are already type="password" but have no reveal toggle. They could adopt SecretInput later.

🤖 Generated with Claude Code

F5 of the 2026-09-10 provider QA run. The form every built-in provider
shares rendered each parameter as <input type="text"> with spellcheck on,
secrets included, so the Versa Bedrock Secret Access Key (and every
provider's API key) was on screen and readable from the DOM as it was
typed. The component already read `parameter.secret`, but only to avoid
echoing a stored value; the custom-provider form masked its key. A
divergence, not a decision.

- New `ui/secret-input.tsx` primitive: `type="password"` until an explicit
  reveal toggle, with `autoComplete="off"` and `spellCheck={false}` applied
  after the caller's props so neither can be switched back on. Starts
  masked on every mount.
- `DefaultProviderSetupForm` renders secret parameters with it; non-secret
  parameters (endpoint, region) stay readable text fields. Labels are now
  associated with their fields.
- `CustomProviderForm`'s key uses the same primitive, so the two forms mask
  and reveal a key identically.

The vitest locates fields by placeholder so it runs unchanged against the
old form: there the masked case fails on the `type` assertion itself and
the non-secret control passes.
F6 of the 2026-09-10 provider QA run. With CODEX_COMMAND pointed at a path
that did not exist, the Codex row read "Not installed" and "Configured" on
one line, and Codex stayed selectable in the Switch-models picker, where
the bind then failed in `from_env`. `check_provider_configured` treated
the agent's one defaulted key as configured whenever it was saved, with no
reference to whether the command resolved.

Daemon:
- `check_provider_configured` now asks `discovery::resolve_configured` (a
  few `stat` calls, never a spawn) for the coding agents, which is the
  same lookup `/coding_agents/status` uses for "Not installed". The two
  answers agree by construction.
- `GET /config/providers` rows carry a new `unavailable_reason`: set only
  when a provider the user set up cannot run, today a coding agent whose
  saved command does not resolve. The sentence is
  `CodingAgentKind::not_installed_summary`, the first sentence of the
  error a turn would have raised.
- Sign-in is deliberately not folded in: learning it spawns the CLI, and
  this route runs for every provider on every settings open.

Renderer:
- `SwitchModelModal` keeps such a provider in its list, disabled with the
  reason on the row. That is react-select's own `aria-disabled` option,
  the private-chat pre-flight's shape one level up. A dialog that opens on
  one says why and will not submit.
- The catalog re-reads the provider list after an explicit "Check
  again", and re-probes the CLIs after any change to an agent's setup.
  Driving the running app showed the second half is needed: correcting
  CODEX_COMMAND in the configure form brought the check back beside a pill
  still reading "Not installed".
- The configure modal still offers "Remove" for a saved key whose CLI is
  missing.

OpenAPI schema and TS client regenerated (the one new field).
…-844a47

# Conflicts:
#	ui/desktop/src/components/settings/models/subcomponents/SwitchModelModal.tsx
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant