Skip to content

feat(extension): crew_profile asks the user for a model itself - #144

Merged
nikolasd merged 5 commits into
mainfrom
model-ask-dialog
Sep 9, 2026
Merged

nikolasd merged 5 commits into
mainfrom
model-ask-dialog

Conversation

@nikolasd

@nikolasd nikolasd commented Sep 9, 2026

Copy link
Copy Markdown
Owner

crew_profile's ask was gated on whether the leader supplied a model (input.model === undefined), so a leader-supplied model for a catalogued adapter (claude/codex/copilot) skipped the ask entirely and was accepted as though it were the user's own choice -- the orchestration skill's own step-1 example passed one, making that the normal path.

Inverts the trigger: the ask now fires whenever no model is configured for the adapter yet, regardless of what (or whether) the leader suggested. A leader-supplied model only preselects a row in the dialog if it resolves to one of the models offered; the human's own pick is what gets used and persisted. Once a model is configured, behavior is unchanged -- an explicit value is only ever compared to it.

The dialog

With no configured model and an interactive UI attached, crew opens the model-selection dialog itself (extCtx.ui.select) rather than instructing the leader to ask and call back. The dialog's title names which source answered:

  • omp's model catalogue, when readCatalogue has an entry for the adapter's provider.
  • the vendor's own model family table, when it doesn't (currentModels() in models.ts) -- e.g. a machine with no anthropic credentials in omp has no catalogue entry for claude, even though the claude binary itself works fine.

Fail-closed paths

Three distinct situations all return the same model-not-configured error code (a caller's retry logic only needs to know registration didn't happen), distinguished by a reason in details and adapter-specific message text:

  • no-ui -- no interactive UI is attached to ask through.
  • no-current-models -- neither the catalogue nor a vendor family table has anything to offer.
  • dialog-timeout -- the dialog resolved undefined (same treatment as plan.ts's existing dialogs).

ompRpc and any caller-defined adapter are unaffected by all of this: they have no catalogue and no alias source, so there is no bounded list to build a dialog from. This is checked before the hasUI check and is independent of it, so an explicit model for these adapters is used exactly as given whether or not a UI is attached -- matching prior behavior and its existing regression test. This is a deliberate departure from the design draft's pseudocode ordering (which checked hasUI unconditionally first): that ordering, read literally, would have made non-catalogued adapters fail closed whenever no UI was attached even with an explicit model given, which is a real behavior change with no test coverage calling for it and an existing test asserting the opposite.

Drift guard

VENDOR_ALIASES.claude's values are now also what the dialog offers when omp's catalogue has nothing for claude, so the table silently drifting from the vendor is no longer cosmetic -- it would mean the dialog offering models that no longer exist or omitting the vendor's current one. Added a test that reads the INSTALLED claude binary's own baked latest_per_family object directly off its raw bytes (greppable, well under a second even at ~200MB) and compares it to the table. Skips visibly (a printed reason) rather than failing when the binary is absent or unreadable -- CI has no vendor CLI. A drift failure names both sides and says what to do; a machine whose installed claude is older or newer than the table is the guard working as intended, not a defect in the table.

Also fixed

  • A stale comment in models.ts claiming omp's catalogue held "725 models across 7 providers on the machine this was written on" -- the catalogue is scoped to providers the local omp installation holds credentials for, which is a different question from whether a vendor CLI itself accepts a given model name (the reason currentModels()'s fallback exists at all).
  • docs/manual-testing.md §3e and the crew-orchestration skill's step-1 example, both rewritten to describe the dialog-driven flow in place of the old "ask the user, then call back" instruction to the leader.

Verification

  • bun run typecheck -- clean.
  • bun test packages/extension -- 357/357 pass.
  • bun run format:check -- clean.
  • bun run check -- passed (schema-drift, format, typecheck, build, bun test, full Rust workspace test suite).
  • Official marker guard (check-markers.ts) -- zero findings repository-wide.

nikolasd and others added 5 commits September 9, 2026 12:36
The ask was gated on whether the leader supplied a model
(input.model === undefined), so a leader-supplied model for a catalogued
adapter (claude/codex/copilot) bypassed it entirely and was accepted as
though it were the user's own choice -- the orchestration skill's own
step-1 example passed one, making that the normal path.

Inverts the trigger: the ask now fires whenever no model is configured
for the adapter yet, regardless of what (or whether) the leader
suggested. With no configured model and an interactive UI attached,
crew opens the model-selection dialog itself, offering the current
models for that adapter (omp's own catalogue, or the vendor's own
model family table when omp has no catalogue entry for that provider --
currentModels() in models.ts, naming which source answered). A
leader-supplied model only preselects a row if it resolves to one of
the options shown; the human's own pick is what gets used and
persisted. Without an interactive UI, on a dialog timeout, or with
nothing to offer, registration is refused with a typed
model-not-configured error. Once a model is configured, behavior is
unchanged: an explicit value is only ever compared to it.

ompRpc and other adapters omp does not catalogue are unaffected --
they have no catalogue and no dialog either way, so an explicit model
is still used exactly as given, matching prior behavior.

Adds a drift guard: VENDOR_ALIASES.claude's values are now also what
the dialog offers when omp's catalogue has nothing for claude, so the
table silently drifting from the vendor is no longer cosmetic. A test
reads the installed claude binary's own baked latest_per_family object
directly off its bytes and compares it to the table, skipping visibly
when the binary is absent or unreadable.

Also fixes a stale comment in models.ts claiming omp's catalogue held
725 models across 7 providers on the machine it was written on --
the catalogue is scoped to providers omp holds credentials for, which
can differ from what a vendor CLI itself accepts (the reason
currentModels() has a fallback at all). Docs (manual-testing.md,
crew-orchestration skill) rewritten to describe the dialog-driven flow
in place of the old ask-then-call-back instruction to the leader.
@nikolasd
nikolasd merged commit a7ac141 into main Sep 9, 2026
17 checks passed
@nikolasd
nikolasd deleted the model-ask-dialog branch September 9, 2026 10:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant