diff --git a/docs/webui.md b/docs/webui.md index 14c64c0d..fb4d1111 100644 --- a/docs/webui.md +++ b/docs/webui.md @@ -254,7 +254,7 @@ Two consequences of that table are deliberate rather than incidental: - **The capability 501 carries no `fallback`.** The hint is the degraded action for a feature that exists and whose call failed. Where the engine has no mode write at all there is nothing to degrade to, and advertising `send_plan_as_prompt` from a "this is not available" response would offer a workaround for a missing feature. The engine's own `unsupported` refusal keeps its hint. - **On the default `acp` transport nothing changes at all.** No provider is registered for `acp` until migration step M4, so the gate reports `unregistered-transport` and every response is the pre-M3 one. The refusals above are reachable on the `runtime` transport, where `local-runtime-v2` is the registered provider. -**The bridge.** A provider can refuse the *generic* config-option write and still have the two dedicated writers webui's own controls depend on. #68's gate therefore asks for a sub-item derived from the request: `model` asks for `selectModel` and `permissionMode` asks for `setPermissionMode`, both of which pass a provider that denies `setConfigOption`; every other config id asks for `setConfigOption` and gets the 501. The exemption is exactly two named ids — never a prefix, never a default — and it does not survive a `none`: a provider with no `authCredentials` at all has no dedicated writer either. +**The bridge.** A provider can refuse the *generic* config-option write and still have the dedicated writers webui's own controls depend on. #68's gate therefore asks for a sub-item derived from the request: `model` asks for `selectModel`, `permissionMode` asks for `setPermissionMode` and `thinkingEffort` asks for `setThinkingEffort`, all of which pass a provider that denies `setConfigOption`; every other config id asks for `setConfigOption` and gets the 501. The exemption is exactly three named ids — never a prefix, never a default — and it does not survive a `none`: a provider with no `authCredentials` at all has no dedicated writer either. (The third id arrived in M3-B14; see below.) **What the user sees.** The permission-mode selector and the model selector are hidden, not disabled and not accompanied by an error message (`webapp/lib/engine-capabilities.ts`, wired in `webapp/components/composer.tsx`). A toast would report a failure for something the user was never able to do, offer nothing to act on, and reappear on every click. The rule is fail-open: the controls are shown until the declaration positively says the engine cannot do it, so a failed or slow `/api/engine-capabilities` request never removes a working control. @@ -281,9 +281,81 @@ Two forms the picker deals with are deliberately different and stay that way. Wh **`contextWindow` is still recorded and never pushed.** The engine's ACP surface has no channel for it, so the pick is a webui-side preference the picker reflects immediately. -**These two endpoints are not gated, and that is an open decision rather than an oversight.** #59 writes `permissionMode` only, so gating it on `authCredentials.setPermissionMode` would be behaviourally inert today and safe against the shipped UI (the permission selector is already hidden under exactly that declaration) — it is one `assertEngineCapability` call. #58 also writes `thinkingEffort`, which is a *generic* config id: gating it the same way would make the thinking-effort control answer 501 for the same reason #68 does for an unrecognised id. Both branches are costed in the KNOWN DEBT section of `model-writes.js` — bridge `thinkingEffort` as a third bridged id, or accept the 501 and extend the frontend's degradation to a third control. Until that is decided, #58 keeps its pre-B10 behaviour. +**These two endpoints were not gated in this batch, and that was an open decision rather than an oversight.** #59 writes `permissionMode` only, so gating it on `authCredentials.setPermissionMode` would be behaviourally inert today and safe against the shipped UI (the permission selector is already hidden under exactly that declaration) — it is one `assertEngineCapability` call. #58 also writes `thinkingEffort`, which was a *generic* config id: gating it the same way would make the thinking-effort control answer 501 for the same reason #68 does for an unrecognised id. Both branches were costed in the KNOWN DEBT section of `model-writes.js` — bridge `thinkingEffort` as a third bridged id, or accept the 501 and extend the frontend's degradation to a third control. **M3-B14 took the first branch**, and the gate landed with it; #58 keeps its pre-B10 behaviour only on the paths that never reach the engine. -**The bridge is no longer an unverified exemption.** `selectModel` and `setPermissionMode` — the two sub-items `MODE_WRITE_BRIDGED_CONFIG_IDS` names — are now in the snapshot audit's `REQUIRED_METHODS`, so a real booted host is checked for both of them on the adapter *and* the CliService surface, and a declaration that stops listing one goes red. Neither surface carries a `setThinkingEffort` / `selectThinkingEffort`, which is the fact the gating decision above turns on. +**The bridge is no longer an unverified exemption.** `selectModel` and `setPermissionMode` — the first two sub-items `MODE_WRITE_BRIDGED_CONFIG_IDS` named — are in the snapshot audit's `REQUIRED_METHODS`, so a real booted host is checked for both of them on the adapter *and* the CliService surface, and a declaration that stops listing one goes red. Neither surface carries a `setThinkingEffort` / `selectThinkingEffort`, which is the fact the gating decision above turns on. The third id M3-B14 added points at that same absent method, so the audit tracks it as a **proven absence** rather than as a presence — see the M3-B14 section for what that means when the engine ships the writer. + +### M3-B11: the provider family moves behind the facade, and the two provider files become one (storage change) + +`GET /api/providers` (#62), `PUT /api/providers` (#63), `POST /api/providers/test` (#64), `GET /api/providers/presets` (#65) and `POST /api/providers/preset/:id/enable` (#66) are the last catalogue family in the migration, and the only one that changes where a user's data lives. + +**What changed.** webui kept two files describing the same providers: `~/.mcode-webui/providers.json` (the v2 catalogue, ordered, lossless) and the engine's `/config.yaml` `custom_provider` tree (a projection of the first, written by a double-write that had no transaction across it). The projection was lossy and the loss was invisible precisely because nothing read it back: a disabled provider, a `coding-plan` provider, a `preset` name and the gemini-vs-openai protocol distinction all vanished on the way to the engine, and the catalogue's ordering came from the file that was about to stop being authoritative. There is now one file. Each webui-managed entry carries its webui record beside its engine fields: + +```yaml +custom_provider: + acme-gateway: + name: Acme Gateway + kind: custom + api: openai-completions + options: { apiKey: …, baseURL: …, authMode: api-key } + models: { glm-5.3: { limit: { context: 128000 } } } + _webui_owned: true # ownership: webui wrote this entry + _webui_provider: { … } # the authoritative v2 record, verbatim +``` + +Both marker fields are ignored by the engine, which parses `config.yaml` through js-yaml with no schema rejection and reads named fields. A provider the engine cannot express still gets its key, its marker and its record — it simply has no engine fields, which is the whole difference from the double write. + +**The migration, and the fallback.** While the store carries no `_webui_provider_migration` marker, the deprecated `providers.json` is still the authority; webui folds it into the store on the next read and stamps the marker on success, after which the file is never read again. A migration that fails — an unparseable `config.yaml`, a write that could not complete — leaves the store byte-identical and the old format readable, and the next read retries. The marker is a field rather than an inference ("the tree has webui entries") for one concrete reason: an operator who deletes every provider leaves a tree with no webui entries, and an inferred marker would hand authority back to the stale file and resurrect what they had just removed. + +Field-by-field equivalence and both fallback paths are pinned in `packages/webui/test/lib/engine/provider-migration.test.js`, on a fixture built to break every assumption the migration could be quietly making: several providers, every schema field, and the boundary values (empty label, absent `preset`, disabled, `coding-plan`, the gemini protocol, a zero context limit, empty thinking levels, a model id the engine key grammar rejects, unicode, a 4096-character key). + +**PUT atomicity is now structural.** There is one file and one `rename`, so the two-file disagreement the old arrangement allowed — the catalogue committed, the engine projection failed, a 200 with a warning nobody had to read — cannot be constructed. A refused write (an unparseable `config.yaml` is refused, never overwritten, because rewriting it would destroy every engine setting the store does not own) or a failed write leaves the previous document intact, and a concurrent reader always sees a whole catalogue. + +**The gates.** The two write endpoints declare `authCredentials` and gate **hard** on `updateUserModelProvider` / `createUserModelProvider`: the catalogue the operator is about to see is read by the engine, so a provider that cannot write providers cannot truthfully answer 200. The three read endpoints declare the same capability and gate **soft** — a provider with no provider surface still serves a well-defined catalogue, so hard-gating them would delete a working UI over an enrichment. As in B9, an unregistered transport (`acp`, until M4) is not a 501. + +**What a client observes.** The endpoint shapes, statuses, masking rule, keep-key convention, probe semantics and the `providers.updated` SSE frame are unchanged. Two response *values* moved with the storage: `PUT`'s `path` is now the engine's `config.yaml`, and it also reports `engineSync: {ok, written, keys}` for the store write itself. `GET`'s `sources` and `userPath` are unchanged in both field and value — they still name the deprecated file, because "which files did the server resolve" is a question an operator asks when a provider is missing, and the answer is now carried by the bilingual docs rather than by a renamed field. + +**Three decisions are recorded rather than taken.** `POST /api/providers/test` names `testUserModelProvider` in its gate, and that method cannot answer it: the engine's tester is keyed on a *persisted* provider, while the endpoint tests an unsaved candidate from a form. The probe stays webui-local, which is also the only option that keeps its two load-bearing properties (the local key-format check runs before any network call, and the apiKey goes to the configured baseURL and nowhere else). The preset gallery is still webui's own template list, and the engine has a different one; the two are not the same taxonomy, so the plan's "align the two template sets" is made visible rather than closed. And a webui provider whose engine key collides with an operator's hand-written entry still overwrites it, because the key *is* the runtime id and a silent rename would turn a recorded model pick into an unresolvable one. All three are costed in the KNOWN DEBT sections of `provider-reads.js` and `provider-writes.js`. + +### M3-B14: `thinkingEffort` becomes a bridged config id, and #58/#59 get capability gates + +M3-B10 moved these two endpoints behind the facade and left one decision open. This batch closes it, and the part worth reading is why the obvious gate on #58 would have been wrong. + +**The decision that was open.** #59 writes `permissionMode` and nothing else, so gating it on `authCredentials.setPermissionMode` is one call and no behaviour change. #58 also writes `thinkingEffort`, and that config id was *generic* — the one the plan (§3a, row 68) says has nowhere to be delivered under a provider with no generic write. Gating #58 the same way would have made the thinking-effort control answer 501 for exactly the reason #68 does for an unrecognised id. Two branches were costed: bridge `thinkingEffort` as a third bridged id, or accept the 501 and hide the control. **The bridge was chosen**, and the exemption list is now three names: + +| config id | #68 asks for | #58 asks for | what the engine receives | +| --- | --- | --- | --- | +| `model` | `selectModel` | — the model push rides the model capability | `m:::u`, or `:v:` for a switchable builtin | +| `permissionMode` | `setPermissionMode` | — #59 is its own endpoint | an engine vocabulary word | +| `thinkingEffort` | `setThinkingEffort` | `setThinkingEffort`, **on the effort channel only** | a bare level | +| anything else | `setConfigOption` → 501 | — | — | + +The exemption is still three named ids — never a prefix, never a default — and it still does not survive a `none`. + +**Why #58's gate is on the effort channel and not on the endpoint.** #58 has two channels, and they use different capabilities. A switchable builtin (ticket 36) has no engine effort vocabulary at all, so **one** `model` push carries the model *and* the on/off level. The level there rides the **model** capability, and gating it on an effort sub-item would 501 a model switch for a capability the switch never uses. A model-only pick on the effort channel has no effort write to gate either. The predicate is therefore read off the plan rather than off the request's fields — `Boolean(plan.thinkingPush)` — and the variant channel is ungated by construction rather than by a second condition somebody has to keep in sync: + +```mermaid +flowchart TD + A["POST /api/set-model"] --> B{"a live session?"} + B -- no --> B1["200 + local-only warning
nothing reaches the engine, so there is
nothing for a gate to be honest about"] + B -- yes --> C{"variant channel?
(switchable builtin)"} + C -- yes --> D["one model push carries
model + on/off level"] + C -- no --> E{"plan.thinkingPush
non-null?"} + E -- no --> F["model-only or cleared effort
NOT gated"] + E -- yes --> G{"authCredentials .
setThinkingEffort"} + G -- allowed --> H["model push, then
thinkingEffort push"] + G -- denied --> I["501 engine_capability_
not_supported"] +``` + +A pure model switch answering 200 while an effort write on the same session, the same provider and the same request frame answers 501 is not an inconsistency — it is the point, and both directions are pinned in `packages/webui/test/lib/engine/model-writes.test.js`. + +**What a user sees.** One change, and it is a UI change rather than a status change: the thinking-effort selector is now the **third** control the engine-capability rule governs (`webapp/lib/engine-capabilities.ts`, wired in `webapp/components/composer.tsx`). Under a provider that declares the dedicated effort writer absent it is hidden, not disabled and not accompanied by a message, for the same reason the other two are. No registered provider declares it today, so **no control disappears on the current builds**; the rule stays fail-open, and a failed or slow `/api/engine-capabilities` request still shows everything. + +**What changed for #68.** `POST /api/protocol/set-config-option` with `key: "thinkingEffort"` no longer answers 501 under such a provider. Nothing in the shipped webapp calls #68, so there is no client to break, and the change makes the two endpoints agree: a config id must not be deliverable through #58 and refused through #68 for the same provider. `contextWindow` is the honest generic example now, and both the suite and this table say so. + +**The name is a forward contract, and the audit says so rather than implying otherwise.** `selectModel` and `setPermissionMode` are methods the audited surfaces really carry, which is why B10 could add them to the snapshot's `REQUIRED_METHODS` and have the audit check them on the adapter *and* the CliService. **`setThinkingEffort` is not.** The snapshot test probes the real booted host by reflection and asserts its absence on both surfaces, in a new `unimplemented` list that means precisely one thing — *this surface must not carry this method* — and that turns the audit **red** the moment either surface grows one. That is the whole closure mechanism, and it is deliberately one-directional: the engine shipping a dedicated effort writer is an event nobody here can schedule, and the audit is what makes it impossible to miss. When it happens, the name moves from `unimplemented` to `methods`, the declaration is re-audited, and the control comes back on its own. + +What is deliberately **not** done: no provider's `authCredentials` declaration was edited to list `setThinkingEffort` in `missing`. Listing it would make every provider refuse the effort write and remove the control for every user today — the other branch's cost, not this one's. The gate reads the declaration, the declaration describes the surface, the surface really has no such method, and the gate is therefore inert. That is the truthful state of the world rather than a faked one. ### Migration state and constraints @@ -324,6 +396,12 @@ An error rather than a synthetic "cancelled" result says plainly that this clien The seam for a real surface is the `clientRequest` constructor option: `(method, params) => result | Promise`. Its resolved value becomes the JSON-RPC `result`; a throw or rejection becomes an error response carrying the thrown `message` and, when it has one, its `code` (otherwise `-32603`). Nothing in the webui installs a handler yet — routing a decision through to the browser is separate work, and the honest current state is that the webui has no interactive surface to offer. +### Engine stderr in the crash alert + +The engine announces its own failures on stderr and then dies; the crash alert is raised by the webui, not by the engine. `McodeAcpClient` therefore keeps a bounded tail of that stream — the last 2KB and the last 20 lines, cleared at every `start()` so one process's crash text can never be blamed on the next — and the `[mcode-acp.start]` and `[mcode-acp.stream]` error alerts carry it as `data.stderrTail`, prefixed with `[acp stderr truncated, showing the tail]` when anything was dropped. An exit code is not a diagnosis: `mcode acp exited (code=1)` cannot separate a lock the engine could not take from a configuration it refused to parse, while the engine's own line (`agent_name_conflict_migration_failed:lock`) says which. + +`stderrTail` is additive and optional. A silent engine leaves `data` byte-identical to what it was before the field existed, so no consumer of the alert contract has to learn a new required key. The `debug` constructor option keeps its old job — mirroring the stream live to the server's own stderr as it arrives — but all three construction sites in the shipped server pass `debug: false`, so in a running webui the alert's tail is the only channel that stderr has. + ### What this does and does not buy `plan: {}` turns on a **notification**, not a question. A plan review carries a single `approve` option and the Runtime pins `allowOther: true` on every step, so the engine settles it fail-closed through the questionnaire path rather than turning it into a permission request — which is why advertising `plan` is safe for a client that cannot answer anything. The permission-request path is a separate switch the webui never turns on. @@ -453,10 +531,65 @@ because they are load-bearing elsewhere: falls back to the engine session id, which does not change. This is why a duplicate send into a first-turn conversation is answered `session-busy` rather than `cid-busy` once the backfill has landed — both refuse. +- **The claim moves with the id, and remembers where it was.** The promotion + is the one instant the conversation's identity changes, so `mcode-acp.js` + re-keys the claim there (`moveRunSession`) on both transports. The registry + keeps the retired key as an alias on the entry, which is what lets the + route's `finally { endRun(cid, runSessionId) }` — still holding the key it + claimed under — find and release the re-keyed claim. + + Without the re-key the guard has a hole, and the hole is about + acknowledgement rather than about locking. `beginRun` cannot see a turn + whose key the view no longer presents, and its remaining guard + (`runsBySid`) is populated by a separate mid-turn backfill. In a window + where neither matches, the server answers `200` and hands a **second + concurrent turn** to an engine session that is already executing — while + the new turn's `›` echo lands in a live `cs.chat` that the run-mirror's + finalize then writes over from a snapshot taken before it. The result is + the one failure this whole area exists to prevent: the engine ran the + message and the webui holds no record of it, so the user gets neither the + bubble nor the history entry and the text is gone. (Observed in the 16:00 + UAT round, 2026-10-03, exception #1.) A guard that cannot see a turn must + not ack it. - **`MAX_CONCURRENT` counts turns, not busy clients.** One tab running two conversations spends two of the slots, because that is two engine subprocesses; that is the resource the ceiling exists to bound. +### Sending while a turn is running + +A message sent into a conversation that is already running a turn is +**refused, not queued**. `POST /api/send` answers `409` with +`reason: "cid-busy"` or `"session-busy"`, the turn is never handed to the +engine, the `›` line is never written, and nothing reaches the persisted +record. The refused text comes back to the composer. + +The 409's `error` field is written for the person reading it — it names the +decision and the next action — because the composer renders it verbatim. +`reason` is the stable machine-readable key, and it is what the client +branches on rather than on the wording. + +There is no queue, and the three send outcomes in the composer are kept +distinct because they ask for opposite behaviour: + +| State | What the server did | What the banner says | What the user should do | +| --- | --- | --- | --- | +| accepted | `200`; the turn runs | — | nothing | +| refused, conversation busy | `409 cid-busy` / `session-busy`; the engine has nothing | not delivered, text is back, wait for the turn | send again when the turn ends | +| unconfirmed | no answer, and the probe against the server could not establish whether the turn started | status unknown, or "the engine is running it, do not resend" | read the history first | + +The third state is the one that must never lie about a side effect. It used +to treat "a turn is running" as proof that *this* send was accepted — the +reasoning being that a busy conversation answers `409` immediately, so a turn +seen after a deadline expiry is this one. That is false for the case that +actually produced the field report: the send was made **into** a running +conversation, so the running turn the probe sees is the previous one. The +banner then told the user "the engine is running your message, do not send it +again" about a message the engine never received. `stateAcceptsSend` now +requires the prompt's own echo line in the transcript, and consults the +running flag only when the snapshot carries no transcript at all — the one +place it cannot be contradicted, and where ignoring it is what made +`sleep 35` execute twice under webui-parity 81 D-2. + What stays tab-scoped, and why it is safe under two live turns: | Concern | Key | Why it is still correct | @@ -2468,8 +2601,8 @@ marker), not by tool name. | `POST` | `/api/auth/decision` | `lib/authorize.js#handleAuthDecision` | `{requestId, approve}`; `200` resolved; `404` no such pending request; `400` bad body; idempotency guard via resolved-set delete | | `POST` | `/api/upload` | `routes/upload.js` | multipart required; `400` if not; `413 {code:"UPLOAD_REQ_TOO_LARGE"\|"UPLOAD_FILE_TOO_LARGE"\|"UPLOAD_QUOTA_EXCEEDED"}`; `400 {code:"UPLOAD_MALFORMED"\|"UPLOAD_ABORTED"}`; write-ahead audit `upload.create.intent` before disk, `upload.create` after; `200 {ok, path, name, size}` | | `GET` | `/api/models` | `routes/model.js#handleGetModels` | engine model + webui label/limit projection; `thinkingLevels` from both engine thinking schemas (effort list verbatim, switchable builtins as `["off","on"]`). Response `{ok, models, groups, current, currentThinking, source, reason?}` — `models` the flat list; `groups` provider-grouped for the picker (`{id, label, auth:{hasKey,type}, protocol?, models}`, `auth`/`protocol` only on config groups — `__engine`/`minimax_api` carry `id/label/models`); `current` the active id or `null` (never a fabricated default); `currentThinking` the active level (`thinkingEffort.currentValue` → `cs.model.thinking` → `null`); `source` = `acp-session-config`\|`config+mcode-cli-bundle`\|`mcode-cli-bundle` (which layer answered); `reason:"no_catalogue"` only when `models` is empty | -| `POST` | `/api/set-model` | `routes/model.js#handleSetModel` | `{model, thinking?}`; `400` only when `model` is empty **and** `thinking` is absent (missing-parameter, not unknown-model — an unknown model name is recorded and pushed, never validated here); effort models push model+`thinkingEffort`, variant models fold the on/off level into one model selection | -| `POST` | `/api/permissions` | `routes/model.js#handleSetPermissions` | `{mode}`; mapped to engine mode via `WEBUI_TO_MCODE_PERMISSION` | +| `POST` | `/api/set-model` | `routes/model.js#handleSetModel` | `{model, thinking?}`; `400` only when `model` is empty **and** `thinking` is absent (missing-parameter, not unknown-model — an unknown model name is recorded and pushed, never validated here); effort models push model+`thinkingEffort`, variant models fold the on/off level into one model selection. Gated on `authCredentials.setThinkingEffort` for a standalone effort write only (M3-B14): a pure model switch and a variant-channel pick are not gated | +| `POST` | `/api/permissions` | `routes/model.js#handleSetPermissions` | `{mode}`; mapped to engine mode via `WEBUI_TO_MCODE_PERMISSION`; gated on `authCredentials.setPermissionMode`, which is behaviourally inert today (M3-B14) | | `GET` | `/api/permissions-modes` | `routes/model.js#handleListPermissionModes` | engine's current `availableModes` | | `POST` | `/api/answer` | `routes/model.js#handleAnswer` | **Removed capability — tombstone only.** Always `410 {ok:false, removed:true, error}`. It used to answer `200 {ok:true, deprecated:true}` without reaching the engine, and four buttons called it, so a click looked successful while the prompt stayed pending. `webapp/lib/api.ts` deliberately exports no client for it; do not add one without a channel that reaches the engine. See "Blocking prompts: what each one can actually answer" | | `GET` | `/api/providers` | `routes/providers.js#handleGetProviders` | masked catalogue | diff --git a/docs/webui.zh-CN.md b/docs/webui.zh-CN.md index 1f6af167..3781fad2 100644 --- a/docs/webui.zh-CN.md +++ b/docs/webui.zh-CN.md @@ -254,7 +254,7 @@ GET /api/engine-capabilities[?provider=] - **能力 501 不带 `fallback`。** 这个提示是「功能存在、但这次调用失败」的降级动作。引擎压根没有模式写入面时，没有任何东西可以降级过去；从一个「此功能不可用」的应答里推销 `send_plan_as_prompt`，等于给一个缺失的功能兜售替代方案。引擎自身的 `unsupported` 拒绝保留它的提示。 - **默认 `acp` 传输下什么都不变。** M4 把 ACP 包成 provider 之前，没有 provider 认领 `acp`，门报 `unregistered-transport`，每个应答都是 M3 之前的那个。上面的拒绝只在 `runtime` 传输上可达——那里注册的 provider 是 `local-runtime-v2`。 -**桥接。** provider 可以拒绝**通用**配置项写入，同时仍保有 webui 自己的两个控件依赖的专用写入面。因此 #68 的门按请求推导子项：`model` 问 `selectModel`、`permissionMode` 问 `setPermissionMode`，两者都能通过一个拒绝 `setConfigOption` 的 provider；其余任何 config id 问 `setConfigOption`，拿到 501。豁免严格只有两个具名 id——绝不是前缀，绝不是默认分支——而且它撑不过 `none`：完全没有 `authCredentials` 的 provider 同样没有专用写入面。 +**桥接。** provider 可以拒绝**通用**配置项写入，同时仍保有 webui 自己的控件依赖的专用写入面。因此 #68 的门按请求推导子项：`model` 问 `selectModel`、`permissionMode` 问 `setPermissionMode`、`thinkingEffort` 问 `setThinkingEffort`，三者都能通过一个拒绝 `setConfigOption` 的 provider；其余任何 config id 问 `setConfigOption`，拿到 501。豁免严格只有三个具名 id——绝不是前缀，绝不是默认分支——而且它撑不过 `none`：完全没有 `authCredentials` 的 provider 同样没有专用写入面。（第三个 id 由 M3-B14 补上，见下文。） **用户看到什么。** 权限模式选择器与模型选择器被**隐藏**，不是禁用，也不配任何错误提示（`webapp/lib/engine-capabilities.ts`，接线在 `webapp/components/composer.tsx`）。toast 会为一件用户从来就做不到的事报一次失败、无从处理、而且每点一次就再报一次。这条规则是 fail-open 的：控件会一直显示，直到声明明确说引擎做不到——因此一次失败或超时的 `/api/engine-capabilities` 请求绝不会拿掉一个本来能用的控件。 @@ -281,9 +281,81 @@ GET /api/engine-capabilities[?provider=] **`contextWindow` 依旧只记录、不推送。** 引擎 ACP 面没有它的通道，因此这项选择是 webui 侧的偏好，选择器立刻就能反映。 -**这两个端点没有挂门，而这是一个待人拍板的开口，不是疏漏。** #59 只写 `permissionMode`，所以把它挂到 `authCredentials.setPermissionMode` 上，今天在行为上是空转的，而且对已发布 UI 安全（权限选择器本来就按同一条声明被隐藏）——那只是一次 `assertEngineCapability` 调用。#58 还会写 `thinkingEffort`，而它是**通用** config id：照样挂门会让思考强度控件因为与 #68 遇到无法识别的 id 时完全相同的原因开始答 501。两个分支的成本都写在 `model-writes.js` 的 KNOWN DEBT 段——把 `thinkingEffort` 桥接成第三个 id，还是接受 501 并把前端降级扩到第三个控件。在拍板之前，#58 保持 B10 之前的行为。 +**这两个端点在本批没有挂门，而那是一个待人拍板的开口，不是疏漏。** #59 只写 `permissionMode`，所以把它挂到 `authCredentials.setPermissionMode` 上，今天在行为上是空转的，而且对已发布 UI 安全（权限选择器本来就按同一条声明被隐藏）——那只是一次 `assertEngineCapability` 调用。#58 还会写 `thinkingEffort`，而它当时是**通用** config id：照样挂门会让思考强度控件因为与 #68 遇到无法识别的 id 时完全相同的原因开始答 501。两个分支的成本都写在 `model-writes.js` 的 KNOWN DEBT 段——把 `thinkingEffort` 桥接成第三个 id，还是接受 501 并把前端降级扩到第三个控件。**M3-B14 选了前一个分支**，门随之落地；#58 只在那些根本不会到达引擎的路径上保持 B10 之前的行为。 -**桥接不再是未经核实的豁免。** `selectModel` 与 `setPermissionMode`——`MODE_WRITE_BRIDGED_CONFIG_IDS` 点名的两个子项——现已进入快照审计的 `REQUIRED_METHODS`，因此真实启动的 host 会在 adapter **与** CliService 两个面上被检查这两个方法，而停止列出其中之一的声明会变红。两个面都没有 `setThinkingEffort` / `selectThinkingEffort`，这正是上面那个挂门决策所依据的事实。 +**桥接不再是未经核实的豁免。** `selectModel` 与 `setPermissionMode`——`MODE_WRITE_BRIDGED_CONFIG_IDS` 点名的前两个子项——现已进入快照审计的 `REQUIRED_METHODS`，因此真实启动的 host 会在 adapter **与** CliService 两个面上被检查这两个方法，而停止列出其中之一的声明会变红。两个面都没有 `setThinkingEffort` / `selectThinkingEffort`，这正是上面那个挂门决策所依据的事实。M3-B14 补上的第三个 id 指向的正是同一个不存在的方法，因此审计把它记作**已证实的不存在**，而不是记作存在——当引擎真的交付这个写入方法时会发生什么，见 M3-B14 一节。 + +### M3-B11：provider 端点族搬进引擎门面，两个 provider 文件合为一个（存储变更） + +`GET /api/providers`（#62）、`PUT /api/providers`（#63）、`POST /api/providers/test`（#64）、`GET /api/providers/presets`（#65）与 `POST /api/providers/preset/:id/enable`（#66）是迁移里最后一个目录族，也是唯一一个会改变用户数据落盘位置的一批。 + +**变了什么。** webui 曾经用两个文件描述同一批 provider：`~/.mcode-webui/providers.json`（v2 目录，有序、无损）与引擎的 `<引擎数据目录>/config.yaml` 里 `custom_provider` 节点（第一个文件的投影，由一次跨不过事务的双写产生）。那个投影有损，而这份损失不可见，恰恰因为没有任何代码把它读回来：被禁用的 provider、`coding-plan` 类型的 provider、`preset` 名、以及 gemini 与 openai 的协议区分，都在通往引擎的路上蒸发了；而目录的顺序来自那个即将不再权威的文件。现在只有一个文件。每条 webui 管理的条目在自己的引擎字段旁边带着它的 webui 记录： + +```yaml +custom_provider: + acme-gateway: + name: Acme Gateway + kind: custom + api: openai-completions + options: { apiKey: …, baseURL: …, authMode: api-key } + models: { glm-5.3: { limit: { context: 128000 } } } + _webui_owned: true # 归属标记：这条是 webui 写的 + _webui_provider: { … } # 权威的 v2 记录，逐字保留 +``` + +两个标记字段都会被引擎忽略——引擎用 js-yaml 解析 `config.yaml`，不做 schema 拒绝，读取的是具名字段。引擎表达不了的 provider 照样拿到自己的 key、标记与记录，只是没有引擎字段——这正是它与旧双写的全部差别。 + +**迁移与回退。** 存储上没有 `_webui_provider_migration` 标记时，已废弃的 `providers.json` 仍是权威来源；webui 在下一次读取时把它折叠进存储，成功后打上标记，此后该文件不再被读取。迁移失败——无法解析的 `config.yaml`、没能完成的写入——会让存储逐字节保持原样，旧格式继续可读，下一次读取会重试。标记是一个字段而不是推断（「树里有 webui 条目」），理由很具体：运维删光所有 provider 之后，树里一条 webui 条目都没有，若按推断判定，就会把权威交还给那份陈旧文件，把刚删掉的东西复活。 + +逐字段等价与两条回退路径由 `packages/webui/test/lib/engine/provider-migration.test.js` 钉死，fixture 专门用来打破迁移可能暗中依赖的每一个假设：多个 provider、schema 的每个字段，以及边界值（空 label、缺失的 `preset`、被禁用、`coding-plan`、gemini 协议、零 contextLimit、空的 thinkingLevels、引擎 key 语法拒绝的模型 id、unicode、4096 字符长的密钥）。 + +**PUT 原子性现在是结构性的。** 一个文件、一次 `rename`，因此旧安排允许的那种两文件分歧——目录已落盘、引擎投影失败、回一个带 warning 的 200 而没人必须读它——无法被构造。被拒绝的写入（无法解析的 `config.yaml` 会被拒绝、绝不覆盖，因为覆盖会毁掉存储并不拥有的全部引擎配置）与失败的写入都让前一份文档保持完整，并发读到的永远是一份完整的目录。 + +**门控。** 两个写端点声明 `authCredentials`，对自己的 `updateUserModelProvider` / `createUserModelProvider` 做**硬**门控：运维马上要看到的目录是引擎读的那份，因此管不了 provider 的 provider 无法如实回 200。三个读端点声明同一能力，做**软**门控——没有 provider 面的 provider 依然能给出定义良好的目录，硬门控等于为一个 enrichment 删掉一个能用的 UI。与 B9 相同，未注册的传输（M4 之前的 `acp`）不算 501。 + +**客户端能观察到什么。** 端点形态、状态码、掩码规则、keep-key 约定、探测语义与 `providers.updated` SSE 帧都不变。有两个**取值**随存储一起搬了家：`PUT` 的 `path` 现在是引擎的 `config.yaml`，并额外回报这次存储写入本身 `engineSync: {ok, written, keys}`。`GET` 的 `sources` 与 `userPath` 字段与取值都不变——它们仍然指向那份已废弃的文件，因为「服务端解析了哪些文件」正是 provider 缺失时运维要问的问题，而新答案由双语文档承载，而不是靠改字段名。 + +**三处只记录、未拍板的决策。** `POST /api/providers/test` 在门里写了 `testUserModelProvider`，而那个方法答不了它：引擎的探测器以**已持久化**的 provider 为键，而这个端点探测的是一份还没保存的候选配置。因此探测留在 webui 本地——这也是唯一能保住它两条承重性质的选项（本地 key 格式校验发生在任何网络调用之前；apiKey 只发往配置的 baseURL）。preset 画廊仍然是 webui 自己的模板列表，而引擎有另一套；两者不是同一套分类法，所以计划里的「两套模板对齐」在本批只是变得可见，并没有关闭。还有，引擎 key 与运维手写条目冲突的 webui provider 依然会覆盖对方，因为这个 key **就是**运行时 id，静默改名会把用户已选的模型变成无法解析的。三处都在 `provider-reads.js` 与 `provider-writes.js` 的 KNOWN DEBT 里逐条算了账。 + +### M3-B14：`thinkingEffort` 成为第三个被桥接的 config id，#58/#59 挂上能力门 + +M3-B10 把这两个端点搬到了门面之后，留下了**一个**待人拍板的开口。本批把它关掉；值得一读的部分，是为什么给 #58 挂一个"显而易见的"门反而是错的。 + +**当时待拍板的是什么。** #59 只写 `permissionMode`，把它挂到 `authCredentials.setPermissionMode` 上是一次调用、零行为变化。#58 还会写 `thinkingEffort`，而这个 config id 当时是**通用**的——正是方案（§3a，第 68 行）所说"在无通用写入面的 provider 下无处投递"的那一个。照同样方式给 #58 挂门，会让思考强度控件因为与 #68 遇到无法识别的 id 时**完全相同**的原因开始答 501。两个分支都做过成本核算：把 `thinkingEffort` 桥接成第三个 id，或者接受 501 并隐藏该控件。**结果是桥接**，豁免名单现在是三个具名 id： + +| config id | #68 问 | #58 问 | 引擎收到什么 | +| --- | --- | --- | --- | +| `model` | `selectModel` | —— 模型推送骑的是模型能力 | `m:::u`；可切换内置模型则是 `:v:` | +| `permissionMode` | `setPermissionMode` | —— #59 是它自己的端点 | 一个引擎词汇 | +| `thinkingEffort` | `setThinkingEffort` | `setThinkingEffort`，**且只在强度通道上** | 一个裸档位 | +| 其余任何 id | `setConfigOption` → 501 | —— | —— | + +豁免依然严格是三个具名 id——绝不是前缀，绝不是默认分支——而且依然撑不过 `none`。 + +**为什么 #58 的门挂在强度通道上，而不是挂在端点上。** #58 有两条通道，而它们用的是**不同的能力**。可切换内置模型（ticket 36）根本没有引擎强度词汇，所以**一次** `model` 推送同时携带模型与 on/off 档位。那里的档位骑的是**模型**能力，用强度子项去卡它，等于为一个该切换根本没用到的能力把模型切换 501 掉。强度通道上的纯模型切换同样没有强度写入可卡。因此这个判据是**从 plan 上读出来的，而不是从请求字段读出来的**——`Boolean(plan.thinkingPush)`——variant 通道因此是**结构上**免门的，而不是靠一个日后要有人负责同步的第二条件： + +```mermaid +flowchart TD + A["POST /api/set-model"] --> B{"有活跃会话？"} + B -- 否 --> B1["200 + 仅本地警告
没有任何东西到达引擎，
因此门无可诚实之处"] + B -- 是 --> C{"variant 通道？
（可切换内置模型）"} + C -- 是 --> D["一次 model 推送
携带模型 + on/off 档位"] + C -- 否 --> E{"plan.thinkingPush
非空？"} + E -- 否 --> F["纯模型切换或清空档位
不挂门"] + E -- 是 --> G{"authCredentials .
setThinkingEffort"} + G -- 允许 --> H["先 model 推送，
后 thinkingEffort 推送"] + G -- 拒绝 --> I["501 engine_capability_
not_supported"] +``` + +纯模型切换答 200、而同一会话、同一 provider、同一段请求框内的强度写入答 501，这不是不一致——这正是要点。两个方向都钉在 `packages/webui/test/lib/engine/model-writes.test.js` 里。 + +**用户看到什么。** 一处变化，而且它是 UI 变化而不是状态码变化：思考强度选择器现在是引擎能力规则治理的**第三个**控件（`webapp/lib/engine-capabilities.ts`，接线在 `webapp/components/composer.tsx`）。在声明缺少专用强度写入面的 provider 下，它被隐藏——不是禁用，也不伴随任何提示——理由与另外两个完全相同。当前**没有任何**已注册 provider 这么声明，所以**当前构建下不会有控件消失**；规则依然是 fail-open，`/api/engine-capabilities` 请求失败或缓慢时依然照常显示全部控件。 + +**#68 变了什么。** 在这类 provider 下，`POST /api/protocol/set-config-option` 带 `key: "thinkingEffort"` 不再答 501。已发布的 webapp 里没有任何代码调用 #68，所以没有客户端会被打破；这个变化让两个端点**保持一致**：同一个 config id 不该在同一个 provider 下能经 #58 投递、却被 #68 拒绝。现在诚实的"通用 id"例子是 `contextWindow`，测试与上表都是这么写的。 + +**这个名字是一份前瞻契约，审计是这么说的，而不是暗示它已存在。** `selectModel` 与 `setPermissionMode` 是被审计的真实面上确实存在的方法，所以 B10 能把它们加进快照的 `REQUIRED_METHODS`，让审计在 adapter **与** CliService 两个面上检查它们。**`setThinkingEffort` 不是。** 快照测试用反射探测真实启动的 host，在一个新的 `unimplemented` 列表里断言它在两个面上都不存在——这个列表只表达一件事，*这个面不得携带该方法*——并且在任一面长出该方法的瞬间让审计**变红**。这就是整套收口机制，而且它是刻意单向的：引擎交付专用强度写入面是一件这里无法排期的事件，而审计是让它无法被漏掉的机制。当它真的发生，名字从 `unimplemented` 移进 `methods`，声明被重新审计，控件自行恢复。 + +刻意**不**做的事：没有把 `setThinkingEffort` 写进任何 provider 声明的 `authCredentials.missing`。写进去会让每个 provider 都拒绝强度写入、把控件对所有用户都拿掉——那是另一个分支的代价，不是本分支的。门读声明，声明描述实现面，实现面确实没有这个方法，因此门是惰性的。这是世界的真实状态，而不是伪造出来的状态。 ### 迁移状态与边界 @@ -324,6 +396,12 @@ ACP 握手是双向的，两个方向都由同一份 `initialize` 载荷决定 留给真实交互界面的接缝是构造函数选项 `clientRequest`：`(method, params) => result | Promise`。它的 resolved 值成为 JSON-RPC 的 `result`；抛错或 reject 变成错误响应，携带抛出的 `message` 与（若有）`code`，否则为 `-32603`。webui 目前没有安装任何处理器——把一次决定真正送到浏览器是另一件事，诚实的现状是 webui 没有可提供的交互界面。 +### 崩溃告警里的引擎 stderr + +引擎把自己的失败写在 stderr 上然后死掉，而崩溃告警是 webui 发的，不是引擎发的。所以 `McodeAcpClient` 会留住这段输出的一个**有界尾部**——最后 2KB 且最后 20 行，并且在每次 `start()` 时清空，免得一个进程的崩溃信息被算到下一个进程头上——`[mcode-acp.start]` 与 `[mcode-acp.stream]` 这两条错误告警把它作为 `data.stderrTail` 带出去；若确实丢掉了内容，前面会加上 `[acp stderr truncated, showing the tail]`。退出码不是诊断：`mcode acp exited (code=1)` 分不清是引擎拿不到锁，还是配置被它拒绝解析；引擎自己那一行（`agent_name_conflict_migration_failed:lock`）才能说明是哪一种。 + +`stderrTail` 是新增的可选字段。引擎若什么都没写，`data` 与这个字段出现之前逐字节相同，所以告警契约的任何消费方都不必学到一个新的必填键。`debug` 构造选项保留它原来的职责——把这段流实时镜像到服务端自己的 stderr——但要清楚：已发布服务端的三个构造点全部传 `debug: false`，因此在一个真正跑起来的 webui 里，告警里的尾部是 stderr 唯一的出口。 + ### 这次拿到了什么、没拿到什么 `plan: {}` 打开的是**通知**，不是提问。计划评审只有一个 `approve` 选项，而运行时把 `allowOther: true` 固定在每一步上，所以引擎会走问卷通道 fail-closed 地了结它，而不会把它变成一次权限请求——这正是「声明 `plan`」对一个什么都答不了的客户端仍然安全的原因。权限请求通道是另一个开关，webui 从不打开它。 @@ -423,9 +501,51 @@ ACP 握手是双向的，两个方向都由同一份 `initialize` 载荷决定 与当前视图匹配。因此每个回答「这个会话是不是正在流式输出的那个」的查找， 都会回退到引擎会话 id——它是不变的。这也是为什么在回填落地之后，往一个 首回合会话的重复发送得到的是 `session-busy` 而不是 `cid-busy`：两者都拒绝。 +- **占用跟着 id 走，并且记得自己来自哪里。** 提升是会话身份改变的唯一时刻， + 所以 `mcode-acp.js` 在那里（两条传输路径都做）用 `moveRunSession` 给占用 + 换键。注册表把退役的键作为别名留在该条目上——这正是路由里的 + `finally { endRun(cid, runSessionId) }`（它手上仍是最初占用时的那个键）还能 + 找到并释放换过键的占用的原因。 + + 没有这次换键，守卫就有一个洞，而这个洞关乎**确认**而不是加锁本身。 + `beginRun` 看不见那些键已不被视图呈现的回合，而它剩下的那道守卫 + （`runsBySid`）由另一处回合中途的回填填充。在两者都不命中的窗口里， + 服务器会回 `200`，并把**第二个并发回合**交给一个正在执行的引擎会话—— + 与此同时该回合的 `›` 回显写进了活的 `cs.chat`，而 run-mirror 的 finalize + 随后用一个更早的快照把它覆盖掉。结果正是这整块机制要防的那一种失败： + 引擎跑了这条消息，而 webui 没有任何记录，用户既看不到气泡也看不到历史条目， + 文字凭空消失。（2026-10-03 16 点轮 UAT 异常 #1 实测。）看不见回合的守卫， + 不能给它回确认。 - **`MAX_CONCURRENT` 计的是回合数，不是忙碌客户端数。** 一个标签页跑两个会话 会占用两个名额，因为这本来就是两个引擎子进程——这正是该上限要约束的资源。 +### 回合进行中发消息 + +往**已经在跑回合**的会话里发消息，是**明确拒绝，不是排队**。`POST /api/send` +回 `409`，`reason` 为 `"cid-busy"` 或 `"session-busy"`；这个回合不会交给引擎， +`›` 行不会写入，落库记录里也不会有任何东西。被拒的原文回到输入框。 + +409 的 `error` 字段是写给读它的人的——它给出这个决定和下一步动作——因为 +composer 会逐字渲染它。`reason` 是稳定的机读键，客户端按它分支，而不是按 +文案。 + +这里没有队列，composer 的三种发送态被刻意区分，因为它们要求的是相反的动作： + +| 态 | 服务器做了什么 | 提示说什么 | 用户该做什么 | +| --- | --- | --- | --- | +| 已接受 | `200`，回合开始执行 | —— | 无 | +| 被拒（会话忙） | `409 cid-busy` / `session-busy`，引擎侧什么都没有 | 未送达，原文已放回，等回合结束 | 回合结束后再发一次 | +| 未能确认 | 没有回答，且向服务器回查也无法确定回合是否已开始 | 状态未知，或「引擎正在执行，请勿重复发送」 | 先看会话历史再决定 | + +第三种态最关键：它绝不能对副作用撒谎。它过去把「有回合在跑」当成本次发送 +已被接受的证据——理由是忙碌的会话会立刻回 `409`，所以截止时间之后看到的回合 +就是这一个。这个推理对真正产生那份现场报告的情形是错的：消息是**往一个正在 +运行的会话里发的**，于是回查看到的那个回合是**上一个**回合。提示于是对一条 +引擎从未收到的消息说「引擎正在执行它，请勿重复发送」。`stateAcceptsSend` 现在 +要求转录里有这条消息自己的回显行，只有当快照完全没有转录时才看运行标志—— +那是唯一无从反驳的地方，而在那里忽略它正是 webui-parity 81 D-2 下 +`sleep 35` 跑了两遍的原因。 + 保持标签页级的部分，以及在两个活动回合下为何依然正确： | 关注点 | 键 | 仍然正确的原因 | @@ -1847,8 +1967,8 @@ createdAtMs, updatedAtMs}`）下发，按 `toolCallId` 幂等、上限 32 条、 | `POST` | `/api/auth/decision` | `lib/authorize.js#handleAuthDecision` | `{requestId, approve}`；`200` 已决；`404` 无该挂起请求；`400` 非法 body；请求处理后通过删除已决条目实现幂等 | | `POST` | `/api/upload` | `routes/upload.js` | 必须是 `multipart/form-data`；否则 `400`；`413 {code:"UPLOAD_REQ_TOO_LARGE"\|"UPLOAD_FILE_TOO_LARGE"\|"UPLOAD_QUOTA_EXCEEDED"}`；`400 {code:"UPLOAD_MALFORMED"\|"UPLOAD_ABORTED"}`；先写 `upload.create.intent` 后写 `upload.create`，全部 fail-closed；`200 {ok, path, name, size}` | | `GET` | `/api/models` | `routes/model.js#handleGetModels` | 引擎模型 + webui 标签/限额投影；`thinkingLevels` 取自引擎两种思考 schema（档位原样、可开关内建为 `["off","on"]`）。响应含 `groups`（按供应商分组，供选择器分节）、`current`（当前模型 id，无则 `null`）、`currentThinking`（当前思考等级）、`models`（扁平列表）与 `source`（目录来源）——字段全表见 [`webui.md`](webui.md) | -| `POST` | `/api/set-model` | `routes/model.js#handleSetModel` | `{model, thinking?}`；仅当 `model` 为空**且**未传 `thinking` 时 → `400`（缺参数，不是"未知模型"——不存在的模型名照样记录下发，接口不校验名字）；effort 模型下发 model+`thinkingEffort`，变体模型把开/关档折进一次模型选择 | -| `POST` | `/api/permissions` | `routes/model.js#handleSetPermissions` | `{mode}`；映射到引擎 `WEBUI_TO_MCODE_PERMISSION` | +| `POST` | `/api/set-model` | `routes/model.js#handleSetModel` | `{model, thinking?}`；仅当 `model` 为空**且**未传 `thinking` 时 → `400`（缺参数，不是"未知模型"——不存在的模型名照样记录下发，接口不校验名字）；effort 模型下发 model+`thinkingEffort`，变体模型把开/关档折进一次模型选择。挂门于 `authCredentials.setThinkingEffort`，且**仅**对独立的强度写入挂门（M3-B14）：纯模型切换与 variant 通道选择不挂门 | +| `POST` | `/api/permissions` | `routes/model.js#handleSetPermissions` | `{mode}`；映射到引擎 `WEBUI_TO_MCODE_PERMISSION`；挂门于 `authCredentials.setPermissionMode`，今天在行为上是空转的（M3-B14） | | `GET` | `/api/permissions-modes` | `routes/model.js#handleListPermissionModes` | 引擎当前的 `availableModes` | | `POST` | `/api/answer` | `routes/model.js#handleAnswer` | **已移除的能力，仅留墓碑路由。** 恒为 `410 {ok:false, removed:true, error}`。它过去返回 `200 {ok:true, deprecated:true}` 却从未触达引擎，而四个按钮都在调它——点击看着成功，提问其实一直挂着。`webapp/lib/api.ts` 刻意不为它导出任何客户端；在拿到真正能触达引擎的通道前不要补回来。详见「阻断式弹窗：各自到底能应答什么」 | | `GET` | `/api/providers` | `routes/providers.js#handleGetProviders` | 掩码后的目录 | diff --git a/packages/webui/acp.mjs b/packages/webui/acp.mjs index d20fe273..94197581 100644 --- a/packages/webui/acp.mjs +++ b/packages/webui/acp.mjs @@ -34,6 +34,20 @@ const DEFAULT_CWD = process.cwd() const JSON_RPC_METHOD_NOT_FOUND = -32601 const JSON_RPC_INTERNAL_ERROR = -32603 +// Bounded tail of the engine subprocess's stderr. +// +// The engine reports its OWN failures on stderr — a failed migration, a +// lock it could not take, a config it refused to parse — and the crash +// alert is raised by the webui, not by the engine. Without a tail the +// whole diagnostic dies with the pipe: the operator sees only +// `mcode acp exited (code=1)` and cannot tell a lock contention from a +// missing binary. Both bounds are needed: bytes alone let one long +// stack trace push the real message out of the window, and lines alone +// let one pathological line carry megabytes. +const STDERR_TAIL_MAX_BYTES = 2048 +const STDERR_TAIL_MAX_LINES = 20 +const STDERR_TRUNCATION_MARKER = '[acp stderr truncated, showing the tail]' + /** * The capabilities this client advertises in `initialize`. * @@ -96,12 +110,31 @@ export class McodeAcpClient extends EventEmitter { // singleton (the previous PR's bug: `_mcodeAcpSingleton.alive` always // undefined) is now actually detected and replaced on the next call. this._alive = false + // Bounded stderr tail (see STDERR_TAIL_MAX_BYTES). Reset per + // `start()` because each start is a different subprocess. + this._stderrTail = '' + this._stderrTruncated = false } get alive() { return this._alive && this.child !== null && this.started === true } + /** + * The engine subprocess's stderr, bounded to the last ~2KB / ~20 lines, + * prefixed with a truncation marker when anything was dropped. + * + * `''` when the engine wrote nothing to stderr — a caller reporting a + * crash omits the field rather than attaching an empty string, so the + * alert it builds keeps the shape it had before this existed. + */ + get stderrTail() { + if (!this._stderrTail) return '' + const lines = this._stderrTail.split('\n') + const kept = lines.slice(-STDERR_TAIL_MAX_LINES).join('\n') + return this._stderrTruncated ? STDERR_TRUNCATION_MARKER + '\n' + kept : kept + } + async start() { if (this.started) return this.capabilities // Windows .cmd shim handling: Node 22+ rejects `spawn('mcode.cmd', { shell:false })` @@ -111,6 +144,10 @@ export class McodeAcpClient extends EventEmitter { // On Linux/macOS, plain `spawn('mcode')` walks PATH. .js/.mjs entries run under // process.execPath on every platform. const resolved = resolveMcodeCmd() + // A new subprocess gets a new tail: a stale line from a previous + // process would misattribute its failure to this one. + this._stderrTail = '' + this._stderrTruncated = false let cmd, args if (/\.(js|mjs)$/i.test(resolved)) { cmd = process.execPath @@ -156,9 +193,7 @@ export class McodeAcpClient extends EventEmitter { this.child.stdout.setEncoding('utf8') this.child.stdout.on('data', (chunk) => this._onData(chunk)) this.child.stderr.setEncoding('utf8') - this.child.stderr.on('data', (c) => { - if (this.debug) process.stderr.write('[acp stderr] ' + c) - }) + this.child.stderr.on('data', (c) => this._onStderr(c)) this.capabilities = await this.request('initialize', { protocolVersion: 1, clientInfo: { name: 'mcode-webui', version: '0.1.0' }, @@ -183,6 +218,22 @@ export class McodeAcpClient extends EventEmitter { this.pending.clear() } + // Record the engine's stderr for the crash alert, and mirror it live + // in debug mode (the dev-loop behavior this handler had before the + // tail existed — unchanged). The tail is kept regardless of `debug`: + // in production nobody is reading the server's own stderr, which is + // precisely why the engine's message has to travel inside the alert. + _onStderr(chunk) { + if (this.debug) process.stderr.write('[acp stderr] ' + chunk) + const next = this._stderrTail + chunk + if (next.length > STDERR_TAIL_MAX_BYTES) { + this._stderrTail = next.slice(-STDERR_TAIL_MAX_BYTES) + this._stderrTruncated = true + } else { + this._stderrTail = next + } + } + _onData(chunk) { this.buf += chunk let nl diff --git a/packages/webui/docs/API.md b/packages/webui/docs/API.md index b28d3fa6..4634685e 100644 --- a/packages/webui/docs/API.md +++ b/packages/webui/docs/API.md @@ -191,6 +191,19 @@ not a total-turn ceiling. `"at-capacity"` (the server is at `MAX_CONCURRENT`, which `/api/health` reports as `maxConcurrent`) +**A 409 is terminal for that message, and it is not a failure.** The turn is +never handed to the engine, the `›` line is never written, and nothing reaches +the persisted record — a refused send cannot be half-applied, and cannot be +one the engine ran while the transcript lost. There is no send queue: "refused, +try again when the turn ends" is the whole contract. + +`error` is the user-facing sentence (the composer renders it verbatim) and +`reason` is the stable machine-readable key; branch on `reason`. For +`cid-busy` and `session-busy` that sentence states the conversation is already +running a turn and the message was not delivered, rather than repeating the +internal detail — which reads "another window" and is wrong for the common +case of the same tab sending again a moment later. + ### `POST /api/stop` Cancel the current run. Tries `session/cancel` via acp (the cancel @@ -1973,9 +1986,14 @@ actually read for each layer, so an operator can confirm which file the live config came from. Layered resolution: `MCODE_WEBUI_MODELS_CONFIG` env → cwd `models.json` -→ user-level `~/.mcode-webui/providers.json` (the PUT write target). -Same-id provider deep merge; models dedupe by id with the higher layer -winning. +→ the engine's `/config.yaml` under `custom_provider` +(the PUT write target). Same-id provider deep merge; models dedupe by id +with the higher layer winning. + +The env and cwd layers are deployment-owned and are never written by +any handler. The third layer used to be a webui file of its own +(`~/.mcode-webui/providers.json`); it is now the engine's own provider +store, and that file is **deprecated** — see "Provider storage" below. **Response 200** ```json @@ -2014,26 +2032,70 @@ winning. } ``` +- `sources.user` and `userPath` still name the **deprecated** + `~/.mcode-webui/providers.json`. The fields did not change and the + values did not either: both are documented as "the files this server + resolved", and an operator diagnosing a missing provider still needs + to be told what to look at. What changed is the answer — the file is + read only until the migration completes, and is never written again. + The live catalogue is the engine store; `GET /api/models` reads it + there too. - `auth.apiKeyMasked` is the only apiKey shape returned by any route in this surface. A test (and `scripts/check-docs-alignment.mjs`) pins the rule: the plaintext key MUST NEVER appear in any `/api/providers*` response, regardless of which layer held it. +**Path forms are reported as the server resolved them, and nothing is +re-resolved.** `sources.cwd` is `/models.json`, and +`process.cwd()` is the kernel-reported working directory — on macOS +that is the fully-resolved form, so a server started under `/var` +reports `/private/var/...`. That is the correct answer to "which file +did you read", and the write side uses the same resolver, so the file +the response names is the file the `PUT` will land in. `sources.user` +and `userPath` come straight from `MCODE_WEBUI_DATA_DIR` and are +reported exactly as configured. + - `sources.env` is `null` when `MCODE_WEBUI_MODELS_CONFIG` is unset; `sources.cwd` is omitted from the layer set in that case (the env override is the cwd file). ### `PUT /api/providers` -Validate-and-persist a v2 provider config to the user-level file -(`~/.mcode-webui/providers.json`, the file written by this handler). -The env / cwd layers are deployment-owned and never written here. - -The handler atomically writes via rename (no half-written file on -disk), reloads the layer set on the next call, and broadcasts an -SSE `providers.updated` named event with the masked payload so -every connected client refreshes its catalogue without polling. -`/api/models` picks up the change on the next request — no restart -required. +Validate-and-persist a v2 provider config to the **engine's provider +store** — `/config.yaml` under `custom_provider`, +written with mode `0600`. The env / cwd layers are deployment-owned and +never written here, and neither is the deprecated +`~/.mcode-webui/providers.json`. + +The handler performs **one** write: a temporary file plus a single +`rename` of the whole document. There is no second file to fall out of +step, so a request either lands completely or changes nothing — a +concurrent reader always sees a whole catalogue, never a mixture, and +never a partially written YAML document. The layer set is re-read on +the next call, and the handler broadcasts an SSE `providers.updated` +named event with the masked payload so every connected client refreshes +its catalogue without polling. `/api/models` picks up the change on +the next request — no restart required. + +**Capability gate.** The two write endpoints (`PUT /api/providers` +and `POST /api/providers/preset/:id/enable`) declare +`authCredentials` and gate **hard** on their sub-item +(`updateUserModelProvider` / `createUserModelProvider`). A provider +that declares the sub-item absent answers +`501 {ok:false, code:"engine_capability_not_supported", …}` rather than +acknowledging a configuration the engine will never read. On the +default `acp` transport no provider is registered yet, so the gate +reports `unregistered-transport` and the write proceeds. The three read +endpoints declare the same capability and gate **soft** — they report +degradation and keep serving. + +**Legacy migration.** While the engine store carries no migration +marker, the deprecated `providers.json` is still the authority: webui +folds it into the store, losslessly, on the next read, and stamps the +marker on success — after which the file is never read again. A failed +migration (an unparseable `config.yaml`, a write that could not +complete) leaves the store untouched and the old format readable, and +the next read retries. Field-by-field equivalence is pinned by +`packages/webui/test/lib/engine/provider-migration.test.js`. **Request** ```json @@ -2059,15 +2121,27 @@ required. { "ok": true, "providers": [ /* masked view, same shape as GET */ ], - "path": "/home/you/.mcode-webui/providers.json" + "path": "/home/you/.minimax/config.yaml", + "engineSync": { "ok": true, "written": true, "keys": ["openai_compat"] } } ``` +- `path` is the file this handler wrote: the engine's `config.yaml`. + It used to be `~/.mcode-webui/providers.json`. +- `engineSync` reports the store write itself. `written: false` means + the document would have come out unchanged (a no-op PUT does not + re-chmod a file an operator just hand-edited). It is `ok: true` + whenever the store accepted the write. - `400 BAD_BODY` — invalid provider shape, unknown protocol, or validation failure (each error carries a human-readable `error` string with the offending field). -- `500 WRITE_FAILED` — disk I/O failure (the in-memory state did - not change; the operator should retry). +- `500 WRITE_FAILED` — the store refused or could not perform the + write. Two causes, and the second is the one that matters: a + `config.yaml` that does not parse is **refused, never + overwritten**, because rewriting it would destroy every engine + setting the store does not own. In both cases the previous + document is intact, the next `GET` returns the catalogue the client + already had, and the operator can retry. ### `POST /api/providers/test` diff --git a/packages/webui/docs/API.zh-CN.md b/packages/webui/docs/API.zh-CN.md index 05d0ffc2..ce2c289b 100644 --- a/packages/webui/docs/API.zh-CN.md +++ b/packages/webui/docs/API.zh-CN.md @@ -175,6 +175,16 @@ stdin。 `"session-busy"`（另一个客户端正在跑这个会话）或 `"at-capacity"` （服务端已达 `MAX_CONCURRENT`，即 `/api/health` 里报的 `maxConcurrent`） +**409 对这条消息是终态，而且它不是失败。** 这个回合不会交给引擎，`›` 行 +不会写入，落库记录里也不会有任何东西——被拒的发送不可能被半途应用，也不可能 +出现「引擎跑了、转录却丢了」的情形。这里没有发送队列，契约就是「被拒，回合 +结束后再发」。 + +`error` 是面向用户的句子（composer 逐字渲染它），`reason` 是稳定的机读键； +按 `reason` 分支。`cid-busy` 与 `session-busy` 的句子说明「本会话正在跑一个 +回合，这条消息未送达」，而不是复述内部 detail——后者写的是「另一个窗口」， +对「同一个标签页隔一会儿再发一次」这个常见情形是错的。 + ### `POST /api/stop` 取消当前运行。先尝试通过 acp 调用 `session/cancel` @@ -1812,8 +1822,14 @@ Multipart 文件上传。保存到 `MCODE_WEBUI_UPLOAD_DIR` 并返回 现网配置来自哪个文件。 分层解析顺序：`MCODE_WEBUI_MODELS_CONFIG` 环境变量 → cwd 下的 -`models.json` → 用户级 `~/.mcode-webui/providers.json`（PUT 的写入 -目标）。同 id 的 provider 做深合并；模型按 id 去重，高层胜出。 +`models.json` → 引擎的 `<引擎数据目录>/config.yaml` 里的 +`custom_provider` 节点（PUT 的写入目标）。同 id 的 provider 做深 +合并；模型按 id 去重，高层胜出。 + +env 与 cwd 两层由部署方拥有，任何 handler 都不写。第三层过去是 +webui 自己的文件（`~/.mcode-webui/providers.json`），现在是引擎 +自己的 provider 存储；那个文件已**废弃**，详见下文 `PUT /api/providers` +一节。 **响应 200** ```json @@ -1856,14 +1872,30 @@ Multipart 文件上传。保存到 `MCODE_WEBUI_UPLOAD_DIR` 并返回 与 `scripts/check-docs-alignment.mjs` 一起把这条规则钉死：无论 密钥来自哪一层，明文 key 都绝不允许出现在任何 `/api/providers*` 响应中。 +- `sources.user` 与 `userPath` 仍然指向**已废弃**的 + `~/.mcode-webui/providers.json`。字段没变，取值也没变：两者的 + 文档语义都是「服务端解析了哪些文件」，运维排查 provider 缺失时 + 仍然需要知道该看哪里。变的是答案——该文件只在迁移完成前被读取， + 此后不再被写入。真正的目录在引擎存储里，`GET /api/models` 也 + 是从那里读的。 +**路径按服务端解析出的形态上报，不做二次解析。** `sources.cwd` 是 +`/models.json`，而 `process.cwd()` 是内核返回的工作 +目录——在 macOS 上它是完全解析后的形态，因此从 `/var` 下启动的服务会 +上报 `/private/var/...`。这正是「你到底读了哪个文件」的正确答案； +写入侧用的是同一个解析器，所以响应里指名的文件就是 `PUT` 会落到的 +文件。`sources.user` 与 `userPath` 直接来自 `MCODE_WEBUI_DATA_DIR`， +按配置原样上报。 + - `MCODE_WEBUI_MODELS_CONFIG` 未设置时 `sources.env` 为 `null`； 此时 `sources.cwd` 也从层级集合中省略（环境变量覆盖的就是 cwd 那个文件）。 ### `PUT /api/providers` -校验并持久化一份 v2 provider 配置到用户级文件 -（`~/.mcode-webui/providers.json`，即本 handler 写入的文件）。 +校验并持久化一份 v2 provider 配置到**引擎的 provider 存储**—— +`<引擎数据目录>/config.yaml` 的 `custom_provider` 节点，文件权限 +`0600`。env / cwd 两层由部署方拥有，本 handler 不写；已废弃的 +`~/.mcode-webui/providers.json` 同样不写。 env / cwd 两层归部署方所有，永远不在这里被写。 handler 通过 rename 原子写入（磁盘上不会出现半写文件），下一次 @@ -1895,14 +1927,40 @@ handler 通过 rename 原子写入（磁盘上不会出现半写文件），下 { "ok": true, "providers": [ /* 掩码视图，形态与 GET 相同 */ ], - "path": "/home/you/.mcode-webui/providers.json" + "path": "/home/you/.minimax/config.yaml", + "engineSync": { "ok": true, "written": true, "keys": ["openai_compat"] } } ``` +- `path` 是本 handler 实际写入的文件：引擎的 `config.yaml`。它 + 过去是 `~/.mcode-webui/providers.json`。 +- `engineSync` 报告这次存储写入本身。`written: false` 表示文档 + 内容不会变化——空转的 PUT 不会去重设运维刚手工编辑过的文件权限。 + 只要存储接受了写入，它就是 `ok: true`。 + +**能力门控。** 两个写端点（`PUT /api/providers` 与 +`POST /api/providers/preset/:id/enable`）声明 `authCredentials` +能力，并对自己的子项（`updateUserModelProvider` / +`createUserModelProvider`）做**硬**门控。声明缺失该子项的 provider +会得到 `501 {ok:false, code:"engine_capability_not_supported", …}`， +而不是确认一份引擎永远不会读取的配置。在默认的 `acp` 传输下尚未 +注册任何 provider，门控报告 `unregistered-transport`，写入照常进行。 +三个读端点声明同一能力，做**软**门控——只报告降级，继续服务。 + +**存量迁移。** 引擎存储里没有迁移标记时，已废弃的 +`providers.json` 仍然是权威来源：webui 会在每次读取时尝试把它 +无损折叠进存储，成功后写入标记，该文件此后再不被读取。迁移失败 +（引擎配置无法解析、写入失败）时存储保持原样，旧格式继续可读， +下一次读取会重试。目录字段逐项等价由 +`packages/webui/test/lib/engine/provider-migration.test.js` 钉死。 + - `400 BAD_BODY` —— provider 形态非法、协议未知，或校验失败 （每条错误都带一条可读的 `error` 文本，指出出问题的字段）。 -- `500 WRITE_FAILED` —— 磁盘 I/O 失败（内存中的状态没有变化； - 运维应重试）。 +- `500 WRITE_FAILED` —— 存储拒绝或未能完成写入。两种成因，其 + 中第二种才是重点：无法解析的 `config.yaml` 会被**拒绝，绝不覆盖**， + 因为覆盖会连带毁掉存储并不拥有的全部引擎配置。两种情况下前一份 + 文档都保持完整，随后的 `GET` 返回客户端原本就有的目录，运维可以 + 直接重试。 ### `POST /api/providers/test` @@ -2018,8 +2076,11 @@ SSE 事件，让每个已连接客户端刷新目录。下一次 `/api/models` ``` - `400 UNKNOWN_PRESET` —— `:id` 不是已知模板。 -- `500 WRITE_FAILED` —— 磁盘 I/O 失败（内存中的状态没有变化； - 运维应重试）。 +- `500 WRITE_FAILED` —— 存储拒绝或未能完成写入。两种成因，其 + 中第二种才是重点：无法解析的 `config.yaml` 会被**拒绝，绝不覆盖**， + 因为覆盖会连带毁掉存储并不拥有的全部引擎配置。两种情况下前一份 + 文档都保持完整，随后的 `GET` 返回客户端原本就有的目录，运维可以 + 直接重试。 --- diff --git a/packages/webui/docs/ARCHITECTURE.md b/packages/webui/docs/ARCHITECTURE.md index a2f1eba0..01a4b5c3 100644 --- a/packages/webui/docs/ARCHITECTURE.md +++ b/packages/webui/docs/ARCHITECTURE.md @@ -608,7 +608,7 @@ is what lets it be re-exported from `engine/index.js` at all. static imports, because `routes/model.js` already imported all four **before** M3-B4 and the server's boot cost is therefore exactly what it was. They reach `@mavis/shared/local-runtime-paths` (via `lib/config.js`) -and `js-yaml` (via `engine-provider-sync.js`), so the module is deliberately +and `js-yaml` (via `engine/provider-store.js`), so the module is deliberately **not** re-exported from `engine/index.js`: making the shared facade — the one import site the whole server shares, and the one `routes/plugins.js` must stay light through — heavier than it has ever been would buy nothing. diff --git a/packages/webui/docs/ARCHITECTURE.zh-CN.md b/packages/webui/docs/ARCHITECTURE.zh-CN.md index 1779293a..ef01569f 100644 --- a/packages/webui/docs/ARCHITECTURE.zh-CN.md +++ b/packages/webui/docs/ARCHITECTURE.zh-CN.md @@ -556,7 +556,7 @@ handler 层测试因此保持封闭。 `lib/engine-catalogue.js`、`lib/models.js`、`lib/providers-config.js`—— 是静态 import，因为 M3-B4 之前 `routes/model.js` 就静态 import 了这四个， 所以 server 的启动成本分文未增。但它们会经 `lib/config.js` 抵达 -`@mavis/shared/local-runtime-paths`、经 `engine-provider-sync.js` 抵达 +`@mavis/shared/local-runtime-paths`、经 `engine/provider-store.js` 抵达 `js-yaml`，所以这个模块**刻意没有**从 `engine/index.js` 转发导出：让 共享门面——整个 server 唯一的共享 import 站点，也是 `routes/plugins.js` 必须保持轻量的那个——比它历来更重，换不来任何东西。 diff --git a/packages/webui/server/engine/mode-writes.js b/packages/webui/server/engine/mode-writes.js index 073049e1..dec264c3 100644 --- a/packages/webui/server/engine/mode-writes.js +++ b/packages/webui/server/engine/mode-writes.js @@ -50,18 +50,29 @@ // plan through the questionnaire mechanism; it has no write. // // - #68 names `authCredentials` · `setConfigOption` — the GENERIC -// config option, per the plan (§3a, row 68). The two config ids -// webui's own controls depend on, `model` and `permissionMode`, are -// NOT the generic write, and the plan requires them to survive it -// ("通用 configId 真 501；两个常用 id 桥接"). So the gate's -// sub-item is a function of the request: the two bridged ids ask -// for their own sub-item and pass a provider that denies the +// config option, per the plan (§3a, row 68). The three config ids +// webui's own controls depend on, `model`, `permissionMode` and +// `thinkingEffort`, are NOT the generic write, and the plan requires +// them to survive it ("通用 configId 真 501；常用 id 桥接"). So the +// gate's sub-item is a function of the request: a bridged id asks +// for its own sub-item and passes a provider that denies the // generic one, and every other config id asks for `setConfigOption` // and gets the 501. `MODE_WRITE_BRIDGED_CONFIG_IDS` is that table, // exported because the frontend needs the same names to decide // which controls to hide (see `webapp/lib/engine-capabilities.ts`, // and the tripwire test that pins the two tables to each other). // +// M3-B14 added the third id, `thinkingEffort` → `setThinkingEffort`. +// It is the same decision the first two took, for the reason #58 +// forced: the effort write rides the GENERIC config-option channel +// on the wire, so without a bridge a provider that denies the +// generic write would make the thinking-effort control the one +// endpoint in the family that 501s. The name is a forward contract +// — NEITHER audited surface has a `setThinkingEffort` method today, +// and the snapshot audit now says so out loud rather than leaving the +// bridge unverified (see `test/lib/engine/capability-snapshot.test.js` +// and `engine/model-writes.js` KNOWN DEBT 1). +// // What the two 501s on these routes now are, and why they must not be // confused. B7 recorded the same collision for #70 and this batch adds // two more instances of it, so it is worth stating flatly: @@ -159,20 +170,29 @@ export const MODE_WRITE_ENDPOINTS = Object.freeze({ }); /** - * The two config ids that survive a provider denying the GENERIC + * The three config ids that survive a provider denying the GENERIC * config-option write, and the sub-item each one asks for instead. * * The plan (§3a, row 68) is explicit that the generic `configId` has * nowhere to be delivered under a provider with no generic write, while - * these two have dedicated equivalents — "两个常用 id 桥接到 + * these have dedicated equivalents — "常用 id 桥接到 * `selectModel`/`setPermissionMode`". Naming the sub-items rather than * quietly widening the gate is what keeps the 501 honest: a provider * that declares `authCredentials` partial with `missing: - * ["setConfigOption"]` says "I have the dedicated model and permission - * writers but not a generic one", and the gate reads exactly that. + * ["setConfigOption"]` says "I have the dedicated model, permission + * and thinking-effort writers but not a generic one", and the gate + * reads exactly that. + * + * `thinkingEffort` → `setThinkingEffort` arrived in M3-B14, and it is + * the one entry whose sub-item NO audited host carries (the other two + * were verified present by reflection before B10 named them). The name + * is therefore a forward contract with the engine, not a description of + * today's host, and the snapshot audit records that gap explicitly + * rather than letting the bridge be an exemption nothing checks. The + * consequence for a client is stated in the KNOWN DEBT section. * * Exported because the frontend asks the same question about the same - * two controls, and two hand-maintained copies of a set of engine + * three controls, and two hand-maintained copies of a set of engine * sub-item names is a drift waiting to happen. The tripwire test in * `webapp/test/engine-capabilities-degradation.test.ts` reads this * table out of the server source and fails if the two ever disagree. @@ -182,6 +202,7 @@ export const MODE_WRITE_ENDPOINTS = Object.freeze({ export const MODE_WRITE_BRIDGED_CONFIG_IDS = Object.freeze({ model: "selectModel", permissionMode: "setPermissionMode", + thinkingEffort: "setThinkingEffort", }); /** @@ -192,7 +213,7 @@ export const MODE_WRITE_BRIDGED_CONFIG_IDS = Object.freeze({ * everything else asks for the generic one. An `undefined` or * non-bridged config id is the generic case, which is the safe * direction — a name nobody recognised must not quietly inherit the - * exemption reserved for the two ids this batch audited. + * exemption reserved for the three ids this family audited. * * @param {string} endpoint A key of MODE_WRITE_ENDPOINTS. * @param {string} [configId] #68 only. @@ -476,6 +497,12 @@ export async function setEngineSessionConfigOption(options = {}) { // Until then the safest thing is that the exemption is narrow: // two named ids, never a prefix, never a default. // +// CLOSED BY M3-B10 for these two names: both were verified present +// by reflection on a booted host and added to `REQUIRED_METHODS`, +// so the audit now checks them on both surfaces. What reopened the +// question is the third id — see item 5, which is the same debt +// with a different answer for a different reason. +// // 3. `/api/set-model` AND `/api/permissions` CALL THE SAME RPC // WRAPPER AND ARE NOT GATED. `routes/model.js` reaches // `lib/mcode-rpc.js#setConfigOption` directly for `model`, @@ -489,6 +516,12 @@ export async function setEngineSessionConfigOption(options = {}) { // bridge it as a third id or to accept the 501 with a frontend // degradation; this batch does not decide it for it. // +// CLOSED BY M3-B14: a human picked branch (a) — bridge it. #58 and +// #59 are now gated, in `engine/model-writes.js`, and the gate is +// deliberately not this family's `assertModeWriteCapability`: the +// two endpoints are not mode-write endpoints, and #58's gate turns +// on which CHANNEL its plan took, which a config id cannot say. +// // 4. `setMode` IS THE ONLY SUB-ITEM #67 ASKS FOR, AND THE MATRIX // HAS NO ROW FOR IT. `toolSkillInvocation` is the plan's home for // session mode control (§3a, row 67) and it is a real @@ -498,3 +531,30 @@ export async function setEngineSessionConfigOption(options = {}) { // plan mode". If M4's provider work ever grows a mode row, #67 // moves to it and nothing else in this file changes except the // one string in `MODE_WRITE_ENDPOINTS`. +// +// 5. `THINKING_EFFORT` IS BRIDGED, AND #68's 501 FOR THAT CONFIG ID +// IS GONE WITH IT. This is the one behaviour change M3-B14 makes +// to this file, and it is a consequence of the bridge rather than a +// separate decision: `#68 {"key":"thinkingEffort"}` used to ask for +// the generic `setConfigOption` and answer 501 under a provider +// that denies it. It now asks for `setThinkingEffort` and is +// delivered, exactly like `model` and `permissionMode` have been +// since B9. +// +// The cost is the honesty of the name. `selectModel` and +// `setPermissionMode` are methods the audited host HAS; +// `setThinkingEffort` is one neither audited surface has, and the +// snapshot audit records that as an explicit "unimplemented" fact +// rather than leaving the third bridge unverified. So under a +// provider that denies the generic write, #68 with +// `key:"thinkingEffort"` now forwards a call the provider cannot +// serve — it will answer with whatever its own dedicated-writer +// path says, which today means the engine is asked directly. +// +// Nothing in the shipped webapp calls #68 (item 1), so there is no +// client to break, and the change makes the two endpoints agree: +// a config id cannot be delivered on #58 and refused on #68 for +// the same provider. The alternative — leaving `thinkingEffort` +// generic on #68 while bridging it on #58 — would have kept a 501 +// that the control is now hidden from, i.e. a status no user could +// ever reach. diff --git a/packages/webui/server/engine/model-reads.js b/packages/webui/server/engine/model-reads.js index c7eb163e..26d4c3e9 100644 --- a/packages/webui/server/engine/model-reads.js +++ b/packages/webui/server/engine/model-reads.js @@ -73,7 +73,7 @@ // server's boot cost is exactly what it was. What they must not do is // reach `@mavis/*` or `js-yaml` through the SHARED facade — and they // do reach `@mavis/shared/local-runtime-paths` (via `lib/config.js`) -// and `js-yaml` (via `engine-provider-sync.js`). That is why this module +// and `js-yaml` (via `engine/provider-store.js`, since M3-B11). That is why this module // is deliberately NOT re-exported from `engine/index.js`, and why // `routes/model.js` imports it directly: `test/lib/engine/host-facade.test.js` // guards `engine/index.js` and `routes/plugins.js` against exactly that diff --git a/packages/webui/server/engine/model-writes.js b/packages/webui/server/engine/model-writes.js index 6bf29e38..4865f28e 100644 --- a/packages/webui/server/engine/model-writes.js +++ b/packages/webui/server/engine/model-writes.js @@ -25,6 +25,7 @@ // | the two `set_config_option` pushes | `pushEngineModelSelection` (here) | // | permission mode → label / engine value | `resolvePermissionSelection` (here) | // | the permission-mode engine push | `pushEnginePermissionMode` (here) | +// | the #58 / #59 capability gate | `assertModelWriteCapability` (here, M3-B14) | // | body parsing, the 400s, the 200 | `routes/model.js` | // | `cs.model` / `cs.permissions` writes | `routes/model.js` (B9's rule, reused) | // | `pushStateFor` and the response body | `routes/model.js` | @@ -37,28 +38,249 @@ // route owns the WRITE (`cs.configOptions` is webui's own view, mutated in // place exactly as before, and only on the same conditions as before). // +// M3-B14: the two endpoints this family owns are now GATED, and the +// gate is not one call. +// +// #59 is the simple half: it writes exactly one config id +// (`permissionMode`), so its gate asks for `authCredentials` +// · `setPermissionMode` unconditionally, and it changes nothing — no +// registered provider declares that sub-item missing, and the shipped +// UI already hides the permission selector under exactly that +// declaration. +// +// #58 is the half that needed a human decision, and the decision +// (recorded in KNOWN DEBT 1 below) was to bridge `thinkingEffort` as a +// THIRD config id so the effort write is a named sub-item rather than a +// generic one. Having made it named, the obvious next step — hang the +// gate on the endpoint — is exactly wrong, and the reason is the two +// channels `planModelSelectionPush` already documents: +// +// VARIANT — a switchable builtin. ONE `model` push carries the +// model AND the on/off level, because such a model has no +// engine effort vocabulary at all. The level rides the +// MODEL capability. Gating it on an effort sub-item would +// 501 a model switch for a capability the switch does not +// use, so it is NOT gated. +// +// EFFORT — a model push, then a `thinkingEffort` push. The second +// one is a STANDALONE effort write on its own config id, +// and it is the only thing in #58 that needs +// `setThinkingEffort`. Gated. +// +// EFFORT, model-only — the request named a model and no level, so +// there is no effort write to gate. A pure model switch +// must NOT be collaterally 501'd by an absent effort +// channel: that would remove a working half of the +// endpoint (and, in the UI, a working model picker) to +// protect a control that is hidden separately anyway. +// +// So `assertModelWriteCapability` takes the plan's answer, not the +// request's fields: `Boolean(plan.thinkingPush)` is the single +// predicate, and it is read AFTER planning so the variant channel is +// `not-applicable` by construction rather than by a second condition +// somebody has to keep in sync. The reverse half is pinned by a test +// in both directions — a provider that denies the effort writer +// answers 501 for an effort write and 200 for a model-only pick on the +// SAME provider and the SAME session. +// // What this file deliberately does NOT do: // -// - It does not gate either endpoint. The decision belongs to a human, -// and the KNOWN DEBT section at the bottom costs both branches: the -// push is one call site per endpoint, so arming either gate is one -// line and nothing else in this file moves. +// - It does not gate `contextWindow`. There is no engine write to +// gate (KNOWN DEBT 3). // - It does not build a host. There is no host on this path. -// - It does not own the transport table or the `501` mapping. B9 owns -// those for #67/#68, and this batch does not duplicate them. +// - It does not own the transport table's other families or the `501` +// mapping. B9 owns those for #67/#68, and `engine/index.js` owns +// the error → HTTP mapping for every family; this batch adds no +// third copy of either. // // Boot-path weight. `routes/model.js` imports this module directly rather // than through `engine/index.js`, and that is the same call B4 made for // `model-reads.js`: this module statically imports `lib/engine-catalogue.js` // (which reaches `js-yaml`), so re-exporting it from the facade index would // make `engine/index.js` heavier than the rest of the server's one shared -// import site. The server's own boot cost is unchanged — every module -// involved was already on it through this route. `lib/mcode-rpc.js` (and -// with it the ACP client) is reached through `await import()` inside the -// two data-plane functions, so the same rule every other engine family -// follows holds here too. +// import site. M3-B14 adds two static imports for the gate — +// `engine/capabilities.js` and `engine/index.js` — and both are +// DECLARATION modules that `app.js` already loads for +// `GET /api/engine-capabilities`, so the server's own boot cost is +// unchanged. `lib/mcode-rpc.js` (and with it the ACP client) is reached +// through `await import()` inside the data-plane functions, so the same +// rule every other engine family follows holds here too. import { resolveModelId, variantChannelFor } from "../lib/engine-catalogue.js"; +import { assertEngineCapability } from "./capabilities.js"; +import { DEFAULT_ENGINE_PROVIDER_ID, getEngineProvider } from "./index.js"; + +// --------------------------------------------------------------------------- +// The gate +// --------------------------------------------------------------------------- + +/** + * Transport → registered engine provider id. Absent means "no provider + * claims this transport yet" (M4), NOT "the capability is + * unavailable" — and the two answer differently on purpose, the same + * split `mode-writes.js#providerByTransport`, + * `session-writes.js#providerByTransport` and five read families draw. + * + * Built per call rather than frozen at module scope, for the same + * reason they give: `engine/index.js` re-exports `mode-writes.js`, so a + * module-level table would read `DEFAULT_ENGINE_PROVIDER_ID` while that + * binding is still in its temporal dead zone on a cold + * `import("./engine/index.js")`. Every consumer of the table is a + * function anyway. + * + * @returns {Readonly>} + */ +function providerByTransport() { + return Object.freeze({ runtime: DEFAULT_ENGINE_PROVIDER_ID }); +} + +/** + * What each endpoint's gate asks for. + * + * A literal rather than a read of `MODE_WRITE_BRIDGED_CONFIG_IDS` at + * module scope, because the same temporal-dead-zone rule applies: this + * module's body must not read a binding that `engine/index.js` might + * still be evaluating. The two tables are the same fact, so the suite + * asserts they are the same fact by VALUE + * (`test/lib/engine/model-writes.test.js` compares every pair) — a + * stronger pin than the frontend's source tripwire, and one that fails + * as a diff rather than as a missing grep. + * + * `subItem: null` is not "any sub-item" and never reaches + * `assertEngineCapability`; it is #58's "this request shape asks for no + * gate", and `resolveModelWriteSubItem` is what turns it into one. + * + * @type {Readonly>} + */ +export const MODEL_WRITE_ENDPOINTS = Object.freeze({ + "POST /api/set-model": Object.freeze({ + capability: "authCredentials", + subItem: "setThinkingEffort", + enforcement: "hard", + }), + "POST /api/permissions": Object.freeze({ + capability: "authCredentials", + subItem: "setPermissionMode", + enforcement: "hard", + }), +}); + +/** + * Which sub-item an endpoint's gate asks for, given the shape of the + * write it is about to make. + * + * #59 writes exactly one config id, so its answer never varies — not + * because the table says so but because nothing about a permission-mode + * change can be a different kind of write. + * + * #58's answer is a function of the PLAN, and specifically of + * `plan.thinkingPush`: that field is non-null only on the effort + * channel with a non-empty level, i.e. exactly the case where a + * standalone effort write is about to happen. The variant channel + * always has it null — the level rides the model push there, and it + * rides the MODEL capability — and a model-only effort-channel pick + * has it null because there is no effort write at all. + * + * The predicate is therefore read after planning, and it is a single + * one. A version that also tested `plan.channel === "effort"` would be + * a second condition describing the same fact, and the failure mode of + * getting it wrong is silent: the gate simply stops firing. + * + * @param {string} endpoint A key of MODEL_WRITE_ENDPOINTS. + * @param {boolean} [effortWrite] #58 only: is a standalone + * `thinkingEffort` push part of this request? + * @returns {string|null} The sub-item, or null when this shape is + * deliberately ungated. + */ +export function resolveModelWriteSubItem(endpoint, effortWrite = false) { + const need = MODEL_WRITE_ENDPOINTS[endpoint]; + if (need === undefined) { + const err = new Error( + `resolveModelWriteSubItem: "${endpoint}" is not part of the model-write family ` + + `(known: ${Object.keys(MODEL_WRITE_ENDPOINTS).join(", ")})`, + ); + err.code = "unknown_model_write_endpoint"; + throw err; + } + if (endpoint !== "POST /api/set-model") return need.subItem; + return effortWrite ? need.subItem : null; +} + +/** + * Resolve the provider that answers the model-write family on + * `transport`, or `null` when none is registered yet. + * + * @param {string} transport One of the `MCODE_WEBUI_TRANSPORT` values. + * @returns {{id: string, transport: string, capabilities: object}|null} + */ +export function resolveModelWriteProvider(transport) { + const providerId = providerByTransport()[transport]; + if (!providerId) return null; + return getEngineProvider(providerId); +} + +/** + * HARD gate for both endpoints. Throws + * `EngineCapabilityNotSupportedError` for a declared `none`, and for a + * `partial` naming the sub-item this particular write needs, which the + * router maps to 501 with `engineCapabilityHttpResponse`'s payload. + * + * Hard for the same reason #67 and #68 are: there is no webui-side + * meaning left to answer with once the engine write is gone. A model + * webui recorded and the engine never selected is not a model the user + * is on, and a thinking level the engine never accepted is not a + * level the user chose. `routes/model.js` still writes `cs.model` and + * answers 200 for the shapes this gate deliberately does not cover — + * see the module header for why those are not collaterally gated. + * + * Three answers, and the difference between them is the point: + * + * `not-applicable` — this write shape asks for no gate. The + * model-only pick and the variant channel. + * Reported, never thrown, so a reader can + * tell "not checked" from "checked and + * passed". + * `unregistered-transport` — no provider claims this transport yet + * (M4). The pre-gate behaviour, and under + * the default `acp` transport that is what + * every request sees. + * `checked` — the declaration was consulted and allows + * the write. + * + * @param {string} endpoint A key of MODEL_WRITE_ENDPOINTS. + * @param {string} transport The active transport. + * @param {boolean} [effortWrite] #58 only. + * @returns {{endpoint: string, gate: string, provider: string|null, + * capability: string, subItem: string|null, enforcement: "hard"}} + */ +export function assertModelWriteCapability(endpoint, transport, effortWrite = false) { + const need = MODEL_WRITE_ENDPOINTS[endpoint]; + if (need === undefined) { + // Caller confusion, not an engine limitation. A plain Error, so a + // typo in webui's own key can never be reported to a user as an + // engine limitation. + const err = new Error( + `assertModelWriteCapability: "${endpoint}" is not part of the model-write family ` + + `(known: ${Object.keys(MODEL_WRITE_ENDPOINTS).join(", ")})`, + ); + err.code = "unknown_model_write_endpoint"; + throw err; + } + const subItem = resolveModelWriteSubItem(endpoint, effortWrite); + const base = { + endpoint, + provider: null, + capability: need.capability, + subItem, + enforcement: need.enforcement, + }; + if (subItem === null) return { ...base, gate: "not-applicable" }; + const provider = resolveModelWriteProvider(transport); + if (!provider) return { ...base, gate: "unregistered-transport" }; + assertEngineCapability(provider.capabilities, need.capability, provider.id, subItem); + return { ...base, gate: "checked", provider: provider.id }; +} + /** * #58's "no session yet" warning — the record-only path. @@ -349,15 +571,25 @@ export async function resolvePermissionSelection(mode) { * 1. NO SESSION → answer with the local-only warning and stop. The pick * is recorded by the route and re-applied on the next boot by * `applyRecordedModel`; there is nothing to push and nothing to say - * about `mcodeSynced` beyond false. + * about `mcodeSynced` beyond false. This returns BEFORE the gate, + * and that order is deliberate: a request that will not reach the + * engine cannot be a fake success (`mcodeSynced: false` plus a + * warning says exactly what happened), so there is nothing for the + * capability to be honest about. * 2. PLAN. `variantChannelFor` reads the engine's materialised builtin * tree; a plan comes back for a switchable builtin and null for * everything else. - * 3. PUSH, in the plan's order. The first failure sets the warning; a + * 3. GATE (M3-B14). Armed only when `plan.thinkingPush` is non-null — + * the standalone effort write. Throws for a provider that declares + * the dedicated effort writer absent; the router answers 501. For + * every other shape this is `not-applicable` and the push proceeds, + * which is what keeps a pure model switch off the effort channel's + * gate. See the module header for why that half is not optional. + * 4. PUSH, in the plan's order. The first failure sets the warning; a * second failure on the effort channel only escalates when the * warning is still the untouched default, so a model rejection is * not overwritten by the effort rejection it caused. - * 4. MIRROR DECISION, returned rather than applied (see the module + * 5. MIRROR DECISION, returned rather than applied (see the module * header). * * `mcodeSynced` reports the MODEL push only, and is false for a @@ -372,10 +604,12 @@ export async function resolvePermissionSelection(mode) { * @param {string} [options.modelId] * @param {boolean} [options.thinkingWasProvided] * @param {string} [options.thinking] + * @param {string} [options.transport] Overrides `MCODE_WEBUI_TRANSPORT` + * for the gate only; the push itself is transport-agnostic. * @returns {Promise<{channel: string, mcodeSynced: boolean, * thinkingSynced: boolean, warning: string|null, * thinkingMirror: {kind: "set", value: string}|{kind: "clear"}|null, - * plan: object}>} + * gate: object, plan: object}>} */ export async function pushEngineModelSelection(options = {}) { const { cs, cid, modelId = "", thinkingWasProvided = false, thinking = "" } = options; @@ -387,12 +621,28 @@ export async function pushEngineModelSelection(options = {}) { thinkingSynced: false, warning: NO_SESSION_MODEL_WARNING, thinkingMirror: null, + gate: { gate: "not-applicable", endpoint: "POST /api/set-model", subItem: null }, plan: null, }; } - const [rpc] = await Promise.all([import("../lib/mcode-rpc.js")]); + const [rpc, config] = await Promise.all([ + import("../lib/mcode-rpc.js"), + import("../lib/config.js"), + ]); + const transport = options.transport || config.MCODE_WEBUI_TRANSPORT; const variantPlan = variantChannelFor(modelSelectionTarget(cs, modelId)); const plan = planModelSelectionPush({ cs, modelId, thinkingWasProvided, thinking, variantPlan }); + // The one predicate that decides whether this endpoint is gated at + // all, and it is read off the PLAN rather than off the request: a + // request that carried a level can still plan no effort push (a + // cleared level, or a switchable builtin that folds the level into + // the model push), and those are exactly the cases that must not be + // collaterally refused. + const gate = assertModelWriteCapability( + "POST /api/set-model", + transport, + Boolean(plan.thinkingPush), + ); let mcodeSynced = false; let thinkingSynced = false; @@ -403,7 +653,7 @@ export async function pushEngineModelSelection(options = {}) { mcodeSynced = plan.reportsModelSynced ? r.ok : false; thinkingSynced = Boolean(r.ok) && plan.carriedThinking; if (!r.ok) warning = r.error; - return { channel: "variant", mcodeSynced, thinkingSynced, warning, thinkingMirror: null, plan }; + return { channel: "variant", mcodeSynced, thinkingSynced, warning, thinkingMirror: null, gate, plan }; } if (plan.modelPush) { @@ -438,7 +688,7 @@ export async function pushEngineModelSelection(options = {}) { : thinkingWasProvided && !thinking && modelId ? { kind: "clear" } : null; - return { channel: "effort", mcodeSynced, thinkingSynced, warning, thinkingMirror, plan }; + return { channel: "effort", mcodeSynced, thinkingSynced, warning, thinkingMirror, gate, plan }; } /** @@ -457,81 +707,109 @@ export async function pushEngineModelSelection(options = {}) { * "Full access" and pushes nothing — and the guard is the difference * between "the engine is in this mode" and "we hope it is". * + * The GATE (M3-B14) runs before the push and only when a push is + * actually going to happen, for the same reason #58's returns before + * its own: a request that records a local pick and reports + * `mcodeSynced: false` is already truthful, so there is no fake + * success for a capability gate to prevent. This endpoint's gate is the + * unconditional case — one config id, one sub-item, no channel to + * reason about — and it is behaviourally inert today: no registered + * provider lists `setPermissionMode` as missing, and the snapshot audit + * proves both providers really carry the method. + * * @param {object} options * @param {object} options.cs Client state; only `mcodeSessionId` is read. * @param {string} options.mcodeValue From `resolvePermissionSelection`. * @param {string} [options.cid] - * @returns {Promise<{mcodeSynced: boolean, warning: string|null}>} + * @param {string} [options.transport] Overrides `MCODE_WEBUI_TRANSPORT` + * for the gate only. + * @returns {Promise<{mcodeSynced: boolean, warning: string|null, gate: object}>} */ export async function pushEnginePermissionMode(options = {}) { const { cs, cid, mcodeValue } = options; const sid = cs && cs.mcodeSessionId; + // The two shapes that stop here also stop at the gate, and the + // reported reason has to say which of the two it was — "not + // applicable" and "not checked because there was nothing to check" + // are the same outcome for the caller and different facts for a + // reader of a log. + const ungated = { gate: "not-applicable", endpoint: "POST /api/permissions", subItem: null }; if (!sid) { - return { mcodeSynced: false, warning: NO_SESSION_PERMISSION_WARNING }; + return { mcodeSynced: false, warning: NO_SESSION_PERMISSION_WARNING, gate: ungated }; } if (!mcodeValue) { - return { mcodeSynced: false, warning: null }; + return { mcodeSynced: false, warning: null, gate: ungated }; } - const [rpc] = await Promise.all([import("../lib/mcode-rpc.js")]); + const [rpc, config] = await Promise.all([ + import("../lib/mcode-rpc.js"), + import("../lib/config.js"), + ]); + const transport = options.transport || config.MCODE_WEBUI_TRANSPORT; + const gate = assertModelWriteCapability("POST /api/permissions", transport); const r = await rpc.setConfigOption(sid, "permissionMode", mcodeValue, cid); - return { mcodeSynced: Boolean(r.ok), warning: r.ok ? null : r.error }; + return { mcodeSynced: Boolean(r.ok), warning: r.ok ? null : r.error, gate }; } // --------------------------------------------------------------------------- // KNOWN DEBT // --------------------------------------------------------------------------- // -// 1. NEITHER ENDPOINT IS GATED, AND THAT IS A DECISION LEFT OPEN FOR A -// HUMAN — not an oversight. B9's gate already exempts exactly the two -// config ids these endpoints write (`model` → `selectModel`, -// `permissionMode` → `setPermissionMode`, see -// `MODE_WRITE_BRIDGED_CONFIG_IDS` in `engine/mode-writes.js`), so -// both sub-items are known names and neither needs rediscovering. -// What stops the gate from being switched on here is one more config -// id, and it is #58's: +// 1. RESOLVED IN M3-B14 — BRANCH (a), AND THE NAME IS A FORWARD +// CONTRACT. Kept as the record of what was decided and why, because +// the next reader's first question is "why is there a third id". // -// - #59 /api/permissions writes `permissionMode` ONLY. Gating it -// hard on `authCredentials.setPermissionMode` is behaviourally -// inert today (no registered provider lists that sub-item as -// missing, and the snapshot audit now proves both providers -// really have the method) and is safe against the shipped UI, -// which already hides the permission selector under exactly that -// declaration (`webapp/lib/engine-capabilities.ts` + -// `composer.tsx`). The change is one -// `assertEngineCapability(...)` call before the push. +// B9's gate already exempted two of the three config ids these +// endpoints write (`model` → `selectModel`, `permissionMode` → +// `setPermissionMode`, see `MODE_WRITE_BRIDGED_CONFIG_IDS` in +// `engine/mode-writes.js`). What stopped the gate being switched on +// was the third: // -// - #58 /api/set-model ALSO writes `thinkingEffort`, and -// `thinkingEffort` is a GENERIC config id — the one the plan -// (§3a, row 68) says has nowhere to be delivered under a -// provider with no generic write. Gating #58 the same way makes -// the thinking-effort control answer 501 for the same reason #68 +// - #59 writes `permissionMode` ONLY, so its gate is +// `authCredentials.setPermissionMode` unconditionally. +// - #58 ALSO writes `thinkingEffort`, and `thinkingEffort` was a +// GENERIC config id — the one the plan (§3a, row 68) says has +// nowhere to be delivered under a provider with no generic +// write. Gating #58 the same way would have made the +// thinking-effort control answer 501 for the same reason #68 // does for an unrecognised id. // -// Two branches, both costed, neither chosen here: -// -// (a) BRIDGE `thinkingEffort` as a THIRD id in -// `MODE_WRITE_BRIDGED_CONFIG_IDS`, pointed at a sub-item that -// means "the dedicated thinking-effort writer". Cost: a third -// name in a table the frontend mirrors, and a third declaration -// the snapshot audit must then prove exists on both surfaces -// (today's probe found no `setThinkingEffort` / -// `selectThinkingEffort` on either, so the name would have to -// be agreed with the engine team first). Benefit: #58 becomes -// gateable on the same table as #59, and the two controls stay -// symmetric. +// The two branches were costed and a human picked (a): bridge +// `thinkingEffort` as a THIRD id, so the effort write has a +// sub-item of its own instead of riding the generic one. Both +// endpoints are now gated and the behaviour change is small and +// deliberate: under a provider that declares the dedicated effort +// writer missing, the thinking-effort CONTROL is hidden (the +// mirror table drives it) while a pure model switch still answers +// 200 — see item 4. // -// (b) ACCEPT the 501 and degrade the UI. Cost: the thinking-effort -// control disappears for any provider that denies the generic -// config write — which, under M4's ACP provider, is most of -// them — and `#58` loses a working half to keep an enrichment. -// `webapp/lib/engine-capabilities.ts` would need a third -// bridged id for the effort control to follow the same -// fail-open rule rather than a 501 at click time. +// The open half, and it is the honest one: the sub-item is named +// `setThinkingEffort`, and NEITHER audited surface has a method by +// that name. B10's probe found no `setThinkingEffort` or +// `selectThinkingEffort` on the adapter or the CliService, and +// M3-B14 re-ran the probe by reflection on a real booted host and +// got the same answer. The name was chosen for symmetry with +// `setPermissionMode` (both are `set_config_option` writes on a +// config id) rather than `selectThinkingEffort` (which would have +// mirrored `selectModel`, a picker call, not a setter), and it is +// a BET ON THE ENGINE, not a description of today's host. // -// Until a human picks one, #58 keeps its pre-B10 behaviour, and this -// module stays gate-ready: the push is already a single call site per -// endpoint, so arming either gate is one line in the executor. +// So the bridge is recorded, not verified: the snapshot audit +// carries `setThinkingEffort` in a new `unimplemented` list, which +// asserts the method is absent on both surfaces and turns the audit +// RED the moment a surface grows it. That is the direction that +// matters — when the engine team lands the dedicated writer, the +// audit goes red, the declaration is re-audited, the name moves +// from `unimplemented` to `methods`, and the control comes back on +// its own. Nothing has to remember to check. // +// What is NOT done, deliberately: no provider's `authCredentials` +// declaration was edited to list `setThinkingEffort` in `missing`. +// Listing it would make every provider refuse the effort write and +// remove the control for every user today, which is branch (b)'s +// cost, not this one's. The gate reads the declaration; the +// declaration describes the surface; the surface really has no such +// method; therefore the gate is inert, and that is the truthful +// state of the world rather than a faked one. // 2. B9's KNOWN DEBT 2 (the bridge naming sub-items no audited host was // proven to have) IS CLOSED BY THIS BATCH, and the evidence is in // `test/lib/engine/capability-snapshot.test.js`: `selectModel` and @@ -542,6 +820,10 @@ export async function pushEnginePermissionMode(options = {}) { // this batch, so its own debt text is left as written; this entry is // the closure record. // +// M3-B14 opens a smaller version of the same question for the third +// id — see item 1 — and handles it with the `unimplemented` list +// rather than by moving a name into `methods`. +// // 3. `contextWindow` IS RECORDED AND NEVER PUSHED. The engine's ACP // surface has no channel for it (`session/set_config_option` accepts // exactly three config ids and the model wire encoding has no context @@ -552,3 +834,22 @@ export async function pushEnginePermissionMode(options = {}) { // request reaches the engine. Wiring it is engine-side work; the seam // is `planModelSelectionPush`'s output, which a future engine // channel would extend with a third push. +// +// 4. #58'S GATE IS ARMED ON THE EFFORT CHANNEL ONLY, AND THAT IS A +// PRODUCT DECISION AS MUCH AS A TECHNICAL ONE. A provider that +// declares `setThinkingEffort` missing answers 501 for an effort +// write and 200 for a model-only pick on the SAME session. The +// alternative — one gate for the whole endpoint — would have +// removed the model picker and #58's entire working half to protect +// an enrichment. It is recorded here because the asymmetry reads as +// an oversight from the route, and because "simplify the gate to one +// check" is the refactor that would silently turn it into a +// regression. The predicate is `Boolean(plan.thinkingPush)` and +// nothing else; the suite pins both directions against one provider. +// +// 5. `#68` NO LONGER 501s FOR `key: "thinkingEffort"`. It is a +// consequence of bridging the id, recorded in both KNOWN DEBT +// sections because the two endpoints write the same config id and a +// reader looking at only one of them should not conclude they behave +// differently. Nothing in the shipped webapp calls #68, so there is +// no client to break; the change makes the two endpoints agree. diff --git a/packages/webui/server/engine/provider-reads.js b/packages/webui/server/engine/provider-reads.js new file mode 100644 index 00000000..65b7d7b0 --- /dev/null +++ b/packages/webui/server/engine/provider-reads.js @@ -0,0 +1,350 @@ +// webui/server/engine/provider-reads.js +// +// Migration step M3, batch B11 (= plan item A5, read half): the +// PROVIDER CATALOGUE READ family — +// +// #62 GET /api/providers — the masked catalogue + layers +// #64 POST /api/providers/test — connectivity probe +// #65 GET /api/providers/presets — the preset gallery +// +// The write half is `provider-writes.js`; the storage both halves share +// is `provider-store.js`. This module is the part that decides WHICH of +// two files is the catalogue, and that decision is the batch. +// +// --------------------------------------------------------------------- +// Before, and after +// --------------------------------------------------------------------- +// +// BEFORE: `providers.json` was the catalogue and `config.yaml` was a +// lossy projection of it. #62 read the first, the engine read the +// second, and the two were kept in agreement by a double write that +// could disagree with itself — the YAML write ran second, could fail, +// and left the first already updated. +// +// AFTER: `config.yaml#custom_provider` is the catalogue, carrying +// each provider's webui record beside its engine fields, and +// `providers.json` is a deprecated source read only until the store +// carries the migration marker. The env and cwd layers are untouched: +// they are deployment-owned files webui has never written and the +// plan does not put them in scope. +// +// The two authorities, and why a failed migration is not a failure of +// the endpoint: +// +// marker present → the store answers; the deprecated file is not read +// at all. +// marker absent → the deprecated file answers and the store +// contributes nothing. A migration is attempted once +// per read, and its outcome is invisible to the +// response, because the response the operator sees +// is the one they saw before this batch. That is the +// fallback contract in full: the old format stays +// readable, and no state is ever half-consumed, +// because the migration's only durable effects (the +// records and the marker) ride the SAME atomic +// rename. +// +// #64 and #65 are named here for the GATE, not for a data plane: the +// probe is a network call webui makes from its own process and the +// gallery is a local template list, so neither reads the store. They +// belong to the family because each answers "is this deployment able to +// manage providers", and a provider that cannot is still better served +// by a working local probe than by a 501 that says nothing about the +// credential the operator pasted. KNOWN DEBT 1 costs the branch that +// would let the engine answer #64. +// +// --------------------------------------------------------------------- +// Gate policy: SOFT, for all three +// --------------------------------------------------------------------- +// +// #62 and #65 are reads whose subject webui owns outright; a +// provider that declared no provider surface would leave the +// catalogue and the gallery perfectly well defined. Gating them hard +// would delete a working endpoint over an enrichment — the +// `session-export.js` argument, reused rather than re-argued. #64 is +// a read too, and a stricter one, for the same reason. The 501 +// machinery stays unused by this family and the suite pins that. +// +// Boot-path weight. `routes/providers.js` imports this module +// directly, NOT through `engine/index.js`, for the reason +// `model-reads.js` set: this module reaches `js-yaml` (through +// `provider-store.js`), and `engine/index.js` is the one import site +// the whole server shares. + +import { + loadProvidersConfig, + loadUserLevelProviders, + SCHEMA_VERSION, +} from "../lib/providers-config.js"; +import { + migrateLegacyProviderStore, + readProviderStore, + userLevelFileExists, +} from "./provider-store.js"; +import { DEFAULT_ENGINE_PROVIDER_ID, getEngineProvider } from "./index.js"; + +/** + * The declaration this family's engine-facing half needs. + * + * `subItem` names the ENGINE method that would eventually serve the + * endpoint, not the one webui calls today. For #62 and #65 that is the + * read pair (`listUserModelProviders` / `listProviderPresets`); for + * #64 it is the engine's tester, which is a different method on a + * different shape — see KNOWN DEBT 1. + * + * @type {Readonly>} + */ +export const PROVIDER_READ_ENDPOINTS = Object.freeze({ + "GET /api/providers": Object.freeze({ + capability: "authCredentials", + subItem: "listUserModelProviders", + enforcement: "soft", + }), + "POST /api/providers/test": Object.freeze({ + capability: "authCredentials", + subItem: "testUserModelProvider", + enforcement: "soft", + }), + "GET /api/providers/presets": Object.freeze({ + capability: "authCredentials", + subItem: "listProviderPresets", + enforcement: "soft", + }), +}); + +/** + * Transport → registered engine provider id. Absent means "no provider + * claims this transport yet" (M4), NOT "the capability is + * unavailable" — the same distinction every sibling family draws, and + * for the same reason: one of them is a deployment gap and the other + * is an engine limitation, and they answer with different statuses. + * + * Built per call rather than frozen at module scope, because + * `engine/index.js` re-exports this module and a module-level table + * would read `DEFAULT_ENGINE_PROVIDER_ID` while that binding is still + * in its temporal dead zone on a cold `import("./engine/index.js")`. + * + * @returns {Readonly>} + */ +function providerByTransport() { + return Object.freeze({ runtime: DEFAULT_ENGINE_PROVIDER_ID }); +} + +/** + * Resolve the provider that answers the provider-read family on + * `transport`, or `null` when none is registered yet. + * + * @param {string} transport + * @returns {{id: string, transport: string, capabilities: object}|null} + */ +export function resolveProviderReadProvider(transport) { + const providerId = providerByTransport()[transport]; + if (!providerId) return null; + return getEngineProvider(providerId); +} + +/** + * SOFT gate. Reports; never throws. A `none`, or a `partial` naming + * this endpoint's sub-item, comes back as `degraded: true` with the + * declaration's own `reason` — the same degradation record + * `summarizeUnavailableCapabilities` produces and the same one the + * frontend already renders from `/api/engine-capabilities`. + * + * @param {string} endpoint A key of PROVIDER_READ_ENDPOINTS. + * @param {string} transport + * @returns {{endpoint: string, provider: string|null, capability: string, + * subItem: string, enforcement: "soft", gate: string, degraded: boolean, + * reason: string|null}} + */ +export function checkProviderReadCapability(endpoint, transport) { + const need = PROVIDER_READ_ENDPOINTS[endpoint]; + if (need === undefined) { + const err = new Error( + `checkProviderReadCapability: "${endpoint}" is not part of the provider-read family ` + + `(known: ${Object.keys(PROVIDER_READ_ENDPOINTS).join(", ")})`, + ); + err.code = "unknown_provider_read_endpoint"; + throw err; + } + const provider = resolveProviderReadProvider(transport); + if (!provider) { + return { + endpoint, + provider: null, + capability: need.capability, + subItem: need.subItem, + enforcement: need.enforcement, + gate: "unregistered-transport", + degraded: false, + reason: null, + }; + } + const entry = provider.capabilities[need.capability]; + const missing = entry && Array.isArray(entry.missing) ? entry.missing : []; + const degraded = + !entry || entry.level === "none" || (entry.level === "partial" && missing.includes(need.subItem)); + return { + endpoint, + provider: provider.id, + capability: need.capability, + subItem: need.subItem, + enforcement: need.enforcement, + gate: "checked", + degraded, + reason: degraded && entry && entry.reason ? entry.reason : null, + }; +} + +/** + * #62 — the resolved provider catalogue, whichever file is currently + * the authority. + * + * The order of the steps is the contract: + * + * 1. Read the store. If it carries the migration marker, its records + * ARE the catalogue and the deprecated file is never opened. + * 2. Otherwise, if the deprecated file exists, attempt the migration + * once and re-read the store. A failure here is NOT an error for + * the caller: the last branch answers from the deprecated file + * exactly as the pre-B11 route did. + * 3. Merge. The user layer (store records, or the legacy records on + * the fallback path) goes UNDER the cwd and env layers, which + * keep their existing precedence and their existing + * re-read-per-call behaviour. + * + * A store that cannot be read at all (an unparseable `config.yaml`) + * takes the same fallback: the deprecated file answers, because a + * syntactically broken engine config must not take the provider dialog + * down with it. What the write path does about that file is the write + * path's problem, and it refuses to overwrite it. + * + * `userLevelFileExists` is what keeps a GET from ever writing: a fresh + * install has no deprecated file, so there is nothing to migrate, and + * polling #62 must not be what gives a machine its first + * `config.yaml`. + * + * @param {{configPath?: string}} [opts] + * @returns {Promise<{ + * version: number, + * providers: object[], + * sources: {env: string|null, cwd: string|null, user: string}, + * userPath: string, + * storePath: string, + * catalogueSource: "engine-store"|"legacy-file", + * migration: {attempted: boolean, migrated: boolean, count: number, + * code: string|null, error: string|null}, + * }>} + */ +export async function readEngineProviderCatalogue(opts = {}) { + const store = readProviderStore(opts); + if (store.ok && store.migrationDone) { + return catalogueFrom(store.records, store, { attempted: false, migrated: false, count: 0 }); + } + if (!userLevelFileExists()) { + return catalogueFrom([], store, { attempted: false, migrated: false, count: 0 }); + } + const legacy = loadUserLevelProviders(); + const migration = await migrateLegacyProviderStore(legacy, opts); + if (migration.ok && migration.migrated) { + const after = readProviderStore(opts); + if (after.ok && after.migrationDone) { + return catalogueFrom(after.records, after, { + attempted: true, + migrated: true, + count: migration.count, + }); + } + } + // Every remaining branch is the fallback: the migration failed, or it + // was already done by a concurrent read, or the store turned out not + // to be readable. The deprecated file answers, unchanged in format, + // and the reason travels with the result for the route's log line. + return catalogueFrom(legacy, store, { + attempted: true, + migrated: false, + count: 0, + code: migration.ok ? null : migration.code, + error: migration.ok ? null : migration.error, + }); +} + +/** + * Run the layer merge for a resolved user layer. The env and cwd + * layers come from `loadProvidersConfig`, which is where their + * precedence and their per-call re-read live; this wrapper only decides + * which providers take the user layer's place, and reports which file + * won. + * + * @param {object[]} userLayer + * @param {object} store A `readProviderStore` result, for the paths. + * @param {object} migration + * @returns {object} + */ +function catalogueFrom(userLayer, store, migration) { + const cfg = loadProvidersConfig({ userLayer }); + return { + version: SCHEMA_VERSION, + providers: cfg.providers, + sources: cfg.sources, + userPath: cfg.sources.user, + storePath: store.configPath, + catalogueSource: store.ok && store.migrationDone ? "engine-store" : "legacy-file", + migration: { + attempted: migration.attempted, + migrated: migration.migrated, + count: migration.count || 0, + code: migration.code || null, + error: migration.error || null, + }, + }; +} + +// --------------------------------------------------------------------------- +// KNOWN DEBT +// --------------------------------------------------------------------------- +// +// 1. #64'S SUB-ITEM NAMES A METHOD THAT CANNOT ANSWER IT, AND THE +// GATE IS STILL WORTH ARMING. The engine's tester is +// `testUserModelProvider(providerId)` — keyed on a PERSISTED +// provider. #64 tests an UNSAVED candidate: the body carries the +// protocol, the key, the baseURL and the headers of a form the +// operator has not submitted yet, and the endpoint's whole +// contract is "does THIS work". There is no id to hand the engine +// yet, so the sub-item can only ever be aspirational. +// +// Two branches, both costed, neither chosen here: +// +// (a) PERSIST-THEN-TEST. Materialise the candidate, ask the +// engine, roll the store back. Cost: a write on a read-only +// endpoint, a window in which another tab's #62 sees a +// half-configured provider, and a rollback that can fail — a +// "Test" button that can lose a concurrent edit is worse +// than one that runs its own fetch. +// +// (b) GIVE THE ENGINE AN UNSAVED-CANDIDATE TESTER, e.g. +// `testUserModelProviderCandidate(input)` taking the same +// shape `createUserModelProvider` does. Cost: an engine API +// change, which is M4's to negotiate, and a capability +// sub-item the snapshot audit would then have to prove. +// +// Until one is chosen the probe stays webui-local, which is also +// the only option that keeps its two load-bearing properties: the +// local key-format check runs BEFORE any network call, and the +// apiKey is sent to the configured baseURL and nowhere else. +// +// 2. #65'S PRESET GALLERY IS STILL WEBUI'S OWN TEMPLATE LIST, and the +// engine has a different one. The two are not the same taxonomy — +// the engine's `McodeProviderTemplate` and webui's +// `PROVIDER_PRESETS` disagree on what a template carries — so the +// plan's "两套模板对齐" regression note is NOT closed by this batch, +// only made visible: the gate now names `listProviderPresets`, so +// a provider that declines to serve presets says so instead of the +// two lists quietly disagreeing. Merging them is an engine-side +// taxonomy decision, recorded here rather than guessed at. +// +// 3. THE ENV AND CWD LAYERS WERE LEFT ALONE ON PURPOSE. They are +// deployment-owned files webui has never written, they keep their +// precedence, and folding them into the store would mean webui +// writing files it does not own. Their records are normalised into +// the same shape, so a future merge is a matter of moving the +// READ, not of changing a schema. diff --git a/packages/webui/server/engine/provider-store.js b/packages/webui/server/engine/provider-store.js new file mode 100644 index 00000000..210bd111 --- /dev/null +++ b/packages/webui/server/engine/provider-store.js @@ -0,0 +1,873 @@ +// webui/server/engine/provider-store.js +// +// Migration step M3, batch B11 (= plan item A5): the SINGLE provider +// store. This module replaces `lib/engine-provider-sync.js` and deletes +// the dual-source arrangement it used to paper over. +// +// --------------------------------------------------------------------- +// What A5 actually was, and what this file is +// --------------------------------------------------------------------- +// +// Before this batch webui kept TWO files describing the same thing: +// +// 1. `/providers.json` — the webui v2 catalogue. +// Ordered, list-shaped, lossless (it holds `preset`, `enabled`, +// the gemini/openai protocol distinction and `coding-plan` +// auth, none of which the engine shape can express). +// 2. `/config.yaml` — the engine's own +// `custom_provider` tree. A DERIVED projection, written by +// `lib/engine-provider-sync.js` on every PUT, carrying +// `_webui_owned` markers so the merge could tell "webui wrote +// this" from "an operator typed this in by hand". +// +// The projection was lossy in both directions, and the loss was +// invisible precisely because nothing read the lossy side back: +// `enabled: false`, `auth.type: "coding-plan"`, `preset` and the +// gemini-vs-openai protocol distinction were dropped on the way to +// the engine and never came back; the ordering of the catalogue came +// from the file that was about to stop being authoritative. +// +// After this batch there is ONE authority for webui-managed +// providers — the engine's `custom_provider` tree — and each +// webui-managed entry carries the webui v2 record alongside its +// engine fields, so the consolidation costs the schema nothing: +// +// custom_provider: +// my-gateway: +// name: My Gateway +// kind: custom +// enabled: true +// api: openai-completions +// options: { apiKey, baseURL, authMode, headers? } +// models: { glm-5.3: { limit, thinking, modalities } } +// _webui_owned: true ← ownership marker (unchanged) +// _webui_provider: { … } ← the lossless v2 record (new) +// +// The engine ignores both marker fields: it parses `config.yaml` +// through js-yaml with no schema rejection and its consumers read +// named fields (`packages/config/src/byok-config.ts`). That is the +// same argument `_webui_owned` already made, and it is why the +// engine's own writer (`updateLocalByokConfig`) can be pointed at +// this file later without a migration of its own. +// +// --------------------------------------------------------------------- +// The migration, and why it can never lose data +// --------------------------------------------------------------------- +// +// `/providers.json` is DEPRECATED, not deleted. It is +// read exactly once per process — by the one-shot migration — and +// only while the store carries no migration marker. The marker +// (`_webui_provider_migration` at the top level of `config.yaml`) is +// what closes the file for good, and it is a top-level marker rather +// than an inference ("the tree has webui entries") for one concrete +// reason: a user who DELETES every provider through the UI leaves a +// tree with no webui entries, and an inferred marker would make the +// stale legacy file authoritative again — resurrecting providers the +// operator had just removed. +// +// The failure path is the other half of the contract. Every step of +// the migration is a pure plan followed by ONE atomic `tmp + rename` +// of the whole `config.yaml`; if any of them fails the file is not +// touched and the marker is not written, so the next read falls back +// to the legacy file in its original format. There is no state in +// which the legacy file has been half-consumed. +// +// --------------------------------------------------------------------- +// What the route must still own +// --------------------------------------------------------------------- +// +// Body parsing, HTTP statuses, masking, the `providers.updated` SSE +// broadcast and the response shapes all stay in +// `routes/providers.js`. This module answers three questions only: +// what the catalogue is (`readProviderStore`), what the next write +// should look like (`buildProviderStoreWrite`, pure), and how the +// write lands (`commitProviderStoreWrite`, one atomic rename). + +import { writeFile, rename, mkdir, chmod, rm } from "node:fs/promises"; +import { dirname, join } from "node:path"; +import { homedir } from "node:os"; +import { existsSync, readFileSync } from "node:fs"; +import yaml from "js-yaml"; +import { randomBytes } from "node:crypto"; + +import { getUserLevelPath, normaliseProvider } from "../lib/providers-config.js"; + +// ===================================================================== +// Markers +// ===================================================================== + +/** + * Ownership marker. Every entry the webui writes carries + * `_webui_owned: true`; an operator who adds a provider through the + * engine CLI (`mcode provider add`) does not, and the two ownerships + * are told apart by this field alone. Unchanged from the module this + * file replaces — renaming it would orphan every operator-managed + * entry on the next write. + * + * @type {string} + */ +export const WEBUI_OWNED_MARKER = "_webui_owned"; + +/** + * The webui v2 record, embedded on every webui-owned entry. THIS is + * what makes the storage consolidation lossless: the engine fields + * beside it are the projection the runtime consumes, and this one is + * the record the catalogue API serialises. A provider the projection + * cannot express (disabled, coding-plan, a gemini endpoint) is still + * fully present here, so merging the two sources into one file drops + * nothing that either source used to hold. + * + * @type {string} + */ +export const WEBUI_PROVIDER_MARKER = "_webui_provider"; + +/** + * Top-level marker meaning "the legacy `providers.json` has been + * folded in; do not read it again". Absent means the opposite. See + * the module header for why this cannot be inferred from the tree. + * + * @type {string} + */ +export const PROVIDER_STORE_MIGRATION_MARKER = "_webui_provider_migration"; + +/** Schema version of the migration marker itself. */ +export const PROVIDER_STORE_MIGRATION_SCHEMA = 1; + +// ===================================================================== +// Engine location +// ===================================================================== + +/** + * Resolve the engine's data directory. + * + * The engine resolves its own data dir via `packages/config/src/config.ts`: + * MINIMAX_DATA_DIR || MAVIS_DATA_DIR || ~/.minimax + * Mirrored verbatim so a webui-managed write lands in the directory + * the engine subprocess reads on next spawn. Read at CALL time, never + * at module scope, so a test (or a deployment) can point it elsewhere + * between two operations. + * + * @returns {string} + */ +export function resolveEngineDataDir() { + const env = process.env.MINIMAX_DATA_DIR?.trim() || process.env.MAVIS_DATA_DIR?.trim() || ""; + if (env) return env; + return join(homedir(), ".minimax"); +} + +/** + * The engine config file this store lives in. The single source for + * the path — `lib/engine-catalogue.js` reads the engine's BUILTIN + * provider tree from the same file and used to import it from + * `lib/engine-provider-sync.js`. + * + * @returns {string} + */ +export function getEngineConfigPath() { + return join(resolveEngineDataDir(), "config.yaml"); +} + +// ===================================================================== +// Pure projection: webui v2 record → engine custom_provider entry +// ===================================================================== + +// engine provider api formats — must match `MODEL_PROVIDER_APIS` in +// packages/local-runtime-v2/src/service/model-system/identity.ts (the +// engine rejects anything outside this set at `normalizeApiFormat`). +const WEBUI_PROTOCOL_TO_ENGINE_API = { + openai: "openai-completions", + anthropic: "anthropic-messages", + // gemini has no engine-native api; the OpenAI-compat endpoint is the + // usual `byok` target. Engine does not have a Gemini-specific format. + gemini: "openai-completions", +}; + +// Reserved engine keys — must NOT collide with the existing engine's +// internal provider ids (which would either shadow `minimax` or land in +// `RESERVED_CUSTOM_PROVIDER_KEYS` and be dropped). Mirrored from +// packages/config/src/byok-config.ts. +const RESERVED_ENGINE_KEYS = new Set(["minimax", "minimax_api", "provider", "custom_provider"]); + +const PROVIDER_KEY_REGEX = /^[A-Za-z0-9][A-Za-z0-9_.-]*$/; + +/** + * Pure: a webui v2 id → an engine-safe provider key. "" when the id + * cannot be expressed as one. + * + * @param {string} id + * @returns {string} + */ +export function providerKeyFromId(id) { + const trimmed = (id || "").trim(); + if (!trimmed) return ""; + if (!PROVIDER_KEY_REGEX.test(trimmed)) return ""; + if (RESERVED_ENGINE_KEYS.has(trimmed)) { + return `${trimmed}-byok`; + } + return trimmed; +} + +/** + * Pure: webui model id → engine-safe model key. + * + * The engine accepts `/` inside model keys: the wire form + * `formatModelKey(, )` uses `/` only as the + * *structural* separator, and `parseSourceQualifiedModelKey` splits on + * the FIRST one, so `deepseek/x` survives as a single string. Upstream + * catalogues carry namespace-style ids like `z-ai/glm-5.3`, and + * rejecting them here dropped those models from the engine sync. + * + * Everything else the engine's YAML parser or custom_provider lookup + * would choke on (whitespace, control codes, YAML structural tokens) is + * still rejected, so the store never lands an unparseable entry. + * + * @param {string} id + * @returns {string} + */ +export function modelKeyFromId(id) { + const trimmed = (id || "").trim(); + if (!trimmed) return ""; + if (/[\s:#{}\[\]@&*!|>'"%`,]/.test(trimmed)) return ""; + // Must not start with `-` (YAML list) or `&` / `*` (anchors). + if (/^[-&*]/.test(trimmed)) return ""; + return trimmed; +} + +/** + * Pure: a normalised v2 record → the engine's own field set, or `{}` when the + * record is ineligible for the engine projection. + * + * Ineligible, and each for a reason the engine states itself: + * - `auth.type === "coding-plan"` — the engine runs coding plans + * through its own OAuth / Codex / Claude Code flows, which are + * outside the byok projection; + * - `enabled: false` — an operator who switched a provider off must + * not see it advertised by `listByokRuntimeModels`; + * - an empty apiKey or an empty baseURL — the engine's + * `createUserProvider` rejects a key-less entry, and with no + * baseURL there is nothing to call; + * - a protocol outside the map, or an id that is not a legal engine + * key even after the reserved-key rewrite. + * + * Ineligibility is a statement about the ENGINE view only. The record + * itself is preserved in full on the entry (see WEBUI_PROVIDER_MARKER), + * which is why one of these costs the catalogue nothing. + * + * Emptiness, not truthiness, on the headers: `normaliseProvider` always + * materialises `auth.headers` (absent → {}), and `{}` is TRUTHY, so a + * truthiness test emits `headers: {}` for every provider that never + * configured one. The runtime merges `options.headers` into every + * upstream request (`local-runtime-v2/.../catalog/provider-views.ts` → + * `mergeProviderHeaders`), so an empty map is noise, not signal. + * + * @param {object} record A normalised webui v2 provider record. + * @returns {object} Engine fields, or `{}` when ineligible. + */ +export function projectRecordToEngine(record) { + if (!record || typeof record !== "object") return {}; + if (record.auth && record.auth.type === "coding-plan") return {}; + if (record.enabled === false) return {}; + const apiKey = typeof record.auth?.apiKey === "string" ? record.auth.apiKey.trim() : ""; + const baseURL = typeof record.auth?.baseURL === "string" ? record.auth.baseURL.trim() : ""; + if (!apiKey || !baseURL) return {}; + const api = WEBUI_PROTOCOL_TO_ENGINE_API[(record.protocol || "openai").trim()]; + if (!api) return {}; + const key = providerKeyFromId(record.id); + if (!key) return {}; + const customHeaders = + record.auth?.headers && typeof record.auth.headers === "object" && Object.keys(record.auth.headers).length > 0 + ? { ...record.auth.headers } + : null; + const name = + typeof record.label === "string" && record.label.trim() ? record.label.trim() : key; + const models = {}; + for (const m of record.models || []) { + const modelKey = modelKeyFromId(m.id); + if (!modelKey) continue; + const engineModel = {}; + if (typeof m.label === "string" && m.label.trim() && m.label.trim() !== modelKey) { + engineModel.name = m.label.trim(); + } + if (typeof m.contextLimit === "number" && m.contextLimit > 0) { + engineModel.limit = { context: m.contextLimit }; + } + if (Array.isArray(m.thinkingLevels) && m.thinkingLevels.length > 0) { + engineModel.thinking = { effortOptions: [...m.thinkingLevels] }; + } + if (Array.isArray(m.modalities) && m.modalities.length > 0) { + engineModel.modalities = { input: [...m.modalities] }; + } + models[modelKey] = engineModel; + } + return { + name, + kind: "custom", + enabled: true, + api, + options: { + apiKey, + baseURL, + authMode: "api-key", + ...(customHeaders ? { headers: customHeaders } : {}), + }, + ...(Object.keys(models).length > 0 ? { models } : {}), + }; +} + +// ===================================================================== +// Pure projection (reverse): engine entry → webui v2 record +// ===================================================================== + +/** + * Pure: engine api format → the webui protocol that maps onto it. + * The map is many-to-one in the forward direction (gemini and openai + * both project to `openai-completions`), so a record reconstructed + * from engine fields ALONE can only be as precise as that map allows: + * `openai-completions` reads back as `openai`. The forward record on + * the entry is what preserves the distinction; this reverse map is the + * fallback for entries written before the record marker existed, and + * the module header's "lossless" claim is scoped to records this + * batch writes, not to engine files authored by hand. + * + * @param {string} api + * @returns {string} + */ +export function protocolFromEngineApi(api) { + if (api === "anthropic-messages") return "anthropic"; + return "openai"; +} + +/** + * Pure: one engine `custom_provider` entry → a normalised webui v2 + * record, or `null` when the entry is not webui-managed. + * + * Two paths, in order: + * 1. `_webui_provider` — the record written alongside the entry. + * Exact, and the only path that can express a disabled provider, + * a coding-plan auth, a `preset`, or the gemini protocol. + * 2. Reconstruction from the engine fields, for entries the + * pre-B11 double-write left behind. Lossy by the map above; it + * exists so an operator who has been running the webui since + * ticket 05 does not come back to an empty catalogue. + * + * The returned record is re-normalised on the way out, so a + * hand-edited or stale `_webui_provider` cannot put a malformed + * record on the wire. + * + * @param {string} key + * @param {object} entry + * @returns {object|null} A normalised v2 provider record. + */ +export function recordFromEngineEntry(key, entry) { + if (!entry || typeof entry !== "object") return null; + if (entry[WEBUI_OWNED_MARKER] !== true) return null; + const embedded = entry[WEBUI_PROVIDER_MARKER]; + if (embedded && typeof embedded === "object") { + const norm = normaliseProvider(embedded); + if (norm.ok) return norm.value; + } + const reconstructed = reconstructRecord(key, entry); + if (!reconstructed) return null; + const norm = normaliseProvider(reconstructed); + return norm.ok ? norm.value : null; +} + +/** + * Pure: reconstruct a v2 record from an entry's engine fields. + * + * The three things a reconstruction cannot know are read off the + * entry in the only honest way available: a `name` equal to the key + * is the projection's own fallback for an empty label, so the + * reconstructed label is the key; models that carried no + * `thinking.effortOptions` / `modalities.input` had none; and the + * protocol is whatever the api format maps back to. + * + * @param {string} key + * @param {object} entry + * @returns {object|null} + */ +function reconstructRecord(key, entry) { + const options = entry.options && typeof entry.options === "object" ? entry.options : {}; + const apiKey = typeof options.apiKey === "string" ? options.apiKey : ""; + if (!apiKey) return null; + const models = []; + for (const [modelKey, model] of Object.entries(entry.models || {})) { + if (!model || typeof model !== "object") continue; + models.push({ + id: modelKey, + label: typeof model.name === "string" && model.name ? model.name : modelKey, + ...(model.limit && typeof model.limit.context === "number" && model.limit.context > 0 + ? { contextLimit: model.limit.context } + : {}), + ...(model.thinking && Array.isArray(model.thinking.effortOptions) && + model.thinking.effortOptions.length > 0 + ? { thinkingLevels: [...model.thinking.effortOptions] } + : {}), + ...(model.modalities && Array.isArray(model.modalities.input) && + model.modalities.input.length > 0 + ? { modalities: [...model.modalities.input] } + : {}), + }); + } + return { + id: key, + label: typeof entry.name === "string" && entry.name ? entry.name : key, + enabled: entry.enabled !== false, + protocol: protocolFromEngineApi(entry.api), + auth: { + type: "byok", + apiKey, + baseURL: typeof options.baseURL === "string" ? options.baseURL : "", + headers: { ...(options.headers || {}) }, + }, + models, + }; +} + +// ===================================================================== +// Store read +// ===================================================================== + +/** + * Read the engine config file. Returns the raw document, or `null` + * when the file does not exist, or `null` with `unreadable: true` + * when it exists but does not parse. + * + * A missing file is not an error — the writer creates it. An + * unparseable one IS, and the caller must not overwrite it: the + * operator's own section would be lost to a `{}` rewrite. + * + * @param {string} configPath + * @returns {{ok: true, raw: object, exists: boolean}|{ok: false, code: string, error: string}} + */ +export function readEngineConfigRaw(configPath) { + if (!existsSync(configPath)) return { ok: true, raw: {}, exists: false }; + let parsed; + try { + parsed = yaml.load(readFileSync(configPath, "utf8")); + } catch (e) { + return { + ok: false, + code: "ENGINE_CONFIG_UNREADABLE", + error: e && e.message ? e.message : String(e), + }; + } + if (parsed === null || parsed === undefined) return { ok: true, raw: {}, exists: true }; + if (typeof parsed !== "object" || Array.isArray(parsed)) { + return { + ok: false, + code: "ENGINE_CONFIG_UNREADABLE", + error: "engine config is not a YAML mapping", + }; + } + return { ok: true, raw: parsed, exists: true }; +} + +/** + * Read the provider store. + * + * Returns the webui-managed records in store order (the order they + * were written, which is the order the catalogue API has always + * returned them in), whether the legacy file is still open, and — when + * it is — the legacy records, so the caller can fall back without a + * second read. + * + * The two authorities, and which one wins: + * + * migration marker PRESENT → the store is authoritative. The + * legacy file is not touched, not even stat()ed. + * migration marker ABSENT → the legacy file is authoritative and + * the store contributes nothing. This is the pre-B11 state (a + * `config.yaml` written by the old double-write carries + * `_webui_owned` markers but no migration marker, and its + * projection is lossy) and the required fallback when a + * migration attempt failed. + * + * @param {{configPath?: string, legacyProviders?: object[]|null}} [opts] + * `legacyProviders` is injected rather than read here so this + * module stays free of the webui data dir; the caller reads + * the deprecated file (see `lib/providers-config.js`). + * @returns {{ + * ok: boolean, + * code?: string, + * error?: string, + * raw: object, + * tree: object, + * records: object[], + * migrationDone: boolean, + * legacyProviders: object[]|null, + * configPath: string, + * }} + */ +export function readProviderStore(opts = {}) { + const configPath = opts.configPath || getEngineConfigPath(); + const read = readEngineConfigRaw(configPath); + if (!read.ok) { + return { + ok: false, + code: read.code, + error: read.error, + raw: null, + tree: null, + records: [], + migrationDone: true, + legacyProviders: opts.legacyProviders || null, + configPath, + }; + } + const raw = read.raw; + const tree = + raw.custom_provider && typeof raw.custom_provider === "object" && !Array.isArray(raw.custom_provider) + ? raw.custom_provider + : {}; + const migrationDone = Boolean(raw[PROVIDER_STORE_MIGRATION_MARKER]); + const records = migrationDone ? providerRecordsFromTree(tree) : []; + return { + ok: true, + raw, + tree, + records, + migrationDone, + legacyProviders: opts.legacyProviders || null, + configPath, + }; +} + +/** + * Pure: the webui-managed records of a `custom_provider` tree, in + * key order. Entries the webui does not own are skipped (they are + * never in the catalogue) and entries whose record cannot be + * normalised are skipped rather than surfaced half-formed. + * + * @param {object} tree + * @returns {object[]} + */ +export function providerRecordsFromTree(tree) { + const out = []; + for (const [key, entry] of Object.entries(tree || {})) { + const record = recordFromEngineEntry(key, entry); + if (record) out.push(record); + } + return out; +} + +// ===================================================================== +// Store write — plan (pure) then commit (one atomic rename) +// ===================================================================== + +/** + * Pure: the next `custom_provider` map for a provider list. + * + * The ownership rule, unchanged from the double-write it replaces and + * the reason foreign entries are safe: + * + * eligible webui keys ∩ existing keys → UPDATE in place + * existing keys ∖ webui keys, `_webui_owned: true` → DELETE + * existing keys ∖ webui keys, marker absent or false → PRESERVE + * webui keys ∖ existing keys → ADD + * + * "webui keys" is every record the caller passes, NOT only the + * projectable ones. A provider the engine cannot express still + * occupies its key with a marker and its record and no engine fields, + * so switching a provider off no longer removes it from the store the + * way the old sync did. + * + * Order is load-bearing and deliberate: webui records come FIRST, in + * the caller's list order, and preserved foreign entries follow. The + * catalogue API returns that order verbatim, and the order an operator + * sees in the dialog has always been the order they PUT. + * + * @param {object} existingTree The store's current `custom_provider`. + * @param {object[]} records Normalised v2 records to persist. + * @returns {{tree: object, keys: string[], preserved: string[], records: string[]}} + */ +export function buildProviderStoreWrite(existingTree, records) { + const tree = {}; + const keys = []; + const persisted = []; + for (const record of records || []) { + if (!record || typeof record !== "object") continue; + const projected = projectRecordToEngine(record); + // Every record gets a home. `normaliseProvider` enforces a + // stricter id grammar than a store key needs, so an id that cannot + // be an engine key can only arrive from a direct engine-module + // caller — but dropping it would be the one loss this batch cannot + // have, and the record is the whole point of the store. The + // fallback key is derived from the index, which is stable for a + // given list order, and an id that DOES map gets its real key. + const storeKey = providerKeyFromId(record.id) || `p${keys.length}`; + delete tree[storeKey]; + tree[storeKey] = { + ...projected, + [WEBUI_OWNED_MARKER]: true, + [WEBUI_PROVIDER_MARKER]: normaliseRecordForStore(record), + }; + keys.push(storeKey); + persisted.push(record.id); + } + for (const [key, entry] of Object.entries(existingTree || {})) { + if (!entry || typeof entry !== "object") continue; + if (entry[WEBUI_OWNED_MARKER] === true) continue; // owned and no longer listed → deleted + if (Object.prototype.hasOwnProperty.call(tree, key)) continue; // already re-added as a record + tree[key] = entry; + } + const preserved = Object.keys(tree).filter( + (k) => tree[k][WEBUI_OWNED_MARKER] !== true, + ); + return { tree, keys, preserved, records: persisted }; +} + +/** + * Pure: normalise a record for embedding. Returns `null` rather than + * throwing for a record the schema rejects — the caller is mid-write + * and the alternative to a null record is a lost provider. + * + * @param {object} record + * @returns {object|null} + */ +function normaliseRecordForStore(record) { + const norm = normaliseProvider(record); + return norm.ok ? norm.value : null; +} + +/** + * Commit a store write: ONE atomic `tmp + rename` of the whole + * `config.yaml`, mode 0600. + * + * Atomicity is the whole point of this function, and it is + * structural rather than best-effort: the previous arrangement wrote + * `providers.json` first and `config.yaml` second, so a failure in + * between left the two files disagreeing and a retry could not tell + * which one the operator was looking at. Here there is one file and + * one rename, so a failed write leaves the previous document exactly + * as it was — a reader either sees the old catalogue or the new one, + * never a mixture, and never a truncated YAML document. + * + * Mode 0600 because the document carries plaintext apiKeys; the + * engine's own `updateLocalByokConfig` does the same + * (`packages/config/src/local-model-provider-write.ts`). + * + * @param {object} options + * @param {string} options.configPath + * @param {object} options.raw The document read before planning. + * @param {object[]} options.records Normalised v2 records to persist. + * @param {boolean} [options.migrated] Stamp the migration marker + * (i.e. this write also closes the legacy `providers.json`). + * @returns {Promise<{ok: boolean, written: boolean, keys: string[], + * preserved: string[], code?: string, error?: string}>} + */ +export async function commitProviderStoreWrite(options = {}) { + const configPath = options.configPath || getEngineConfigPath(); + try { + // Re-read rather than trust the caller's `raw`. The plan is + // built on what the caller saw, but the DECISION to write at all + // must be made from what is on disk right now: a config.yaml that + // became unparseable between the caller's read and this call + // would otherwise be overwritten, and overwriting it destroys every + // section the store does not own. One extra small read per write is + // the cheapest insurance in this module. + const disk = readEngineConfigRaw(configPath); + if (!disk.ok) { + return { + ok: false, + written: false, + keys: [], + preserved: [], + code: "ENGINE_STORE_UNREADABLE", + error: disk.error, + }; + } + const plan = buildProviderStoreWrite( + options.tree || {}, + options.records || [], + ); + const next = { ...(options.raw || options.diskRaw || disk.raw), custom_provider: plan.tree }; + if (options.migrated) { + next[PROVIDER_STORE_MIGRATION_MARKER] = { + schema: PROVIDER_STORE_MIGRATION_SCHEMA, + at: new Date().toISOString(), + }; + } + // Skip the write when the document would come out identical: a + // no-op PUT should not touch mtime, and should not re-chmod a file + // an operator just hand-edited. + if (sameEngineConfigDocument(options.raw || disk.raw, next) && !options.migrated) { + return { ok: true, written: false, keys: plan.keys, preserved: plan.preserved }; + } + await atomicWriteYaml0600(configPath, next); + return { ok: true, written: true, keys: plan.keys, preserved: plan.preserved }; + } catch (e) { + return { + ok: false, + written: false, + keys: [], + preserved: [], + code: "ENGINE_STORE_WRITE_FAILED", + error: e && e.message ? e.message : String(e), + }; + } +} + +/** + * Pure: would writing `next` change the document on disk? Compared on + * the YAML text, not on the object, because that is what a reader + * actually observes — and because object comparison would call a + * re-ordered `custom_provider` a change when the engine does not care. + * + * @param {object|undefined} raw + * @param {object} next + * @returns {boolean} + */ +function sameEngineConfigDocument(raw, next) { + if (!raw) return false; + const dump = (o) => yaml.dump(o, { indent: 2, lineWidth: -1, noRefs: true }); + return dump(raw) === dump(next); +} + +/** + * Atomic YAML write + 0600 permission pin. + * + * Two-step: write the new content to a tmp file (mode 0600), then + * rename. The rename preserves POSIX mode, but we chmod the target + * afterwards as belt-and-suspenders (some filesystems and Windows + * edge cases drop the mode on rename). + * + * @param {string} configPath + * @param {object} object + */ +export async function atomicWriteYaml0600(configPath, object) { + await mkdir(dirname(configPath), { recursive: true }); + const tmp = join(dirname(configPath), `.config-tmp-${randomBytes(6).toString("hex")}`); + // mode 0600 — owner read/write only. The file carries plaintext + // apiKeys; any looser mode would expose them to other users on the + // host. + await writeFile(tmp, yaml.dump(object, { indent: 2, lineWidth: -1, noRefs: true }), { + encoding: "utf8", + mode: 0o600, + }); + try { + await chmod(tmp, 0o600); + await rename(tmp, configPath); + await chmod(configPath, 0o600); + } catch (e) { + // A rename that fails leaves the tmp file behind, and that file + // carries every plaintext apiKey in the catalogue at mode 0600 in + // the engine data dir — one leaked copy per failed write, none of + // them ever read. The old double write had the same gap and only + // ever tested the success path; this batch is the one that makes + // the write atomic, so it is also the one that has to clean up + // after itself when it cannot. + await rm(tmp, { force: true }).catch(() => {}); + throw e; + } +} + +// ===================================================================== +// One-shot migration of the deprecated `providers.json` +// ===================================================================== + +/** + * Does the deprecated `providers.json` exist? + * + * The read path asks this before it considers migrating, and that is + * the whole reason it exists: there is nothing to migrate on a fresh + * install, and a GET must never be what gives a machine its first + * `config.yaml`. A file that exists but does not parse still counts as + * present — the migration then runs, reads zero records out of it, and + * stamps the marker, which is the correct outcome for a corrupt file + * (the operator gets an empty catalogue they can rebuild rather than + * a permanently-failing one). + * + * @returns {boolean} + */ +export function userLevelFileExists() { + return existsSync(getUserLevelPath()); +} + +/** + * In-flight migration promise, module scope. The migration is + * triggered from the READ path, and a webui with several tabs polling + * `/api/providers` would otherwise start one write per request. The + * memo is cleared on settle, so a FAILED migration retries on the next + * read — which is the fallback contract, not a bug: the store is + * untouched, the marker is unwritten, and the legacy file is still + * the authority until a later attempt succeeds. + * + * @type {Promise|null} + */ +let migrationInFlight = null; + +/** + * Fold the deprecated `providers.json` into the store, once. + * + * Safe to call from every read. It is a no-op when the marker is + * already present (the common case after the first PUT), and when the + * legacy file is absent or empty it stamps the marker with an empty + * catalogue so a fresh install stops looking for the file. + * + * Never throws. A failure returns a structured result and leaves both + * files exactly as they were; the caller answers from the legacy file + * in that case, which is the pre-B11 behaviour. + * + * @param {object[]} legacyProviders Normalised records read from the + * deprecated file. The caller owns the read so this module + * never learns the webui data dir's layout. + * @param {{configPath?: string}} [opts] + * @returns {Promise<{ok: boolean, migrated: boolean, count: number, + * code?: string, error?: string}>} + */ +export async function migrateLegacyProviderStore(legacyProviders, opts = {}) { + if (migrationInFlight) return migrationInFlight; + const run = (async () => { + const configPath = opts.configPath || getEngineConfigPath(); + const read = readEngineConfigRaw(configPath); + if (!read.ok) { + return { + ok: false, + migrated: false, + count: 0, + code: read.code, + error: read.error, + }; + } + const raw = read.raw; + if (raw[PROVIDER_STORE_MIGRATION_MARKER]) { + return { ok: true, migrated: false, count: 0 }; + } + const tree = + raw.custom_provider && + typeof raw.custom_provider === "object" && + !Array.isArray(raw.custom_provider) + ? raw.custom_provider + : {}; + const records = legacyProviders || []; + const result = await commitProviderStoreWrite({ + configPath, + raw, + tree, + records, + migrated: true, + }); + if (!result.ok) { + return { ok: false, migrated: false, count: 0, code: result.code, error: result.error }; + } + return { ok: true, migrated: true, count: result.keys.length }; + })(); + migrationInFlight = run; + try { + return await run; + } finally { + migrationInFlight = null; + } +} + +/** + * Test-only: drop the in-flight migration memo. A suite that drives + * a failing migration and then a succeeding one needs the second + * attempt to actually run. + * + * @returns {void} + */ +export function _resetProviderStoreMigration() { + migrationInFlight = null; +} diff --git a/packages/webui/server/engine/provider-writes.js b/packages/webui/server/engine/provider-writes.js new file mode 100644 index 00000000..fef0c18f --- /dev/null +++ b/packages/webui/server/engine/provider-writes.js @@ -0,0 +1,280 @@ +// webui/server/engine/provider-writes.js +// +// Migration step M3, batch B11 (= plan item A5, write half): the +// PROVIDER CATALOGUE WRITE family — +// +// #63 PUT /api/providers — replace the catalogue +// #66 POST /api/providers/preset/:id/enable — materialise a preset +// +// The read half is `provider-reads.js`; the storage both halves share +// is `provider-store.js`. +// +// --------------------------------------------------------------------- +// Why this is the batch's HARD-gated half +// --------------------------------------------------------------------- +// +// Before this batch, #63 wrote `providers.json` FIRST and projected +// it into the engine's `config.yaml` SECOND. That is two durable +// writes with no transaction between them, and the order was chosen +// so the projection could never advertise something the catalogue +// did not have — which is a real property, bought with a worse one: +// when the second write failed, the first had already landed, the +// response was 200 with a `warning`, and the operator's next edit was +// computed from a file the engine had never seen. The catalogue and +// the engine were allowed to disagree, permanently, with a warning +// nobody was required to read. +// +// There is one file now and one rename, so the disagreement cannot +// be constructed. What is left to decide is what an engine that +// cannot manage providers should answer, and the answer is 501: the +// endpoint's entire product is "the provider configuration is now +// this", a state the engine reads and webui does not. A 200 there +// would be #110's fake success in its purest form — a panel showing +// a key the runtime will never send. +// +// #66 shares the gate with #63 because it IS #63: it materialises a +// template and hands the result to the same commit. One gate, one +// commit, two routes. +// +// --------------------------------------------------------------------- +// What the route keeps +// --------------------------------------------------------------------- +// +// Body parsing, the 400s, the keep-key convention's PLACEMENT, the +// `providers.updated` SSE frame, the ACP singleton teardown, the +// response shape and the `engineSync` / `warning` fields all stay in +// `routes/providers.js`. This module owns the gate, the decision of +// which records the write persists, and the write itself. + +import { + commitProviderStoreWrite, + readProviderStore, +} from "./provider-store.js"; +import { applyKeepKeyConvention } from "../lib/providers-config.js"; +import { assertEngineCapability } from "./capabilities.js"; +import { DEFAULT_ENGINE_PROVIDER_ID, getEngineProvider } from "./index.js"; + +/** + * The declaration this family's engine-facing half needs. + * + * @type {Readonly>} + */ +export const PROVIDER_WRITE_ENDPOINTS = Object.freeze({ + "PUT /api/providers": Object.freeze({ + capability: "authCredentials", + subItem: "updateUserModelProvider", + enforcement: "hard", + }), + "POST /api/providers/preset/:id/enable": Object.freeze({ + capability: "authCredentials", + subItem: "createUserModelProvider", + enforcement: "hard", + }), +}); + +/** + * Transport → registered engine provider id. Absent means "no provider + * claims this transport yet" (M4), NOT "the capability is + * unavailable" — the distinction every sibling family draws, and the + * one that decides whether this endpoint answers 404-for-an-unknown- + * provider (a deployment question) or 501 (an engine limitation). + * + * Built per call, never frozen at module scope: `engine/index.js` + * re-exports this module, and a module-level table would read + * `DEFAULT_ENGINE_PROVIDER_ID` while that binding is still in its + * temporal dead zone on a cold `import("./engine/index.js")`. + * + * @returns {Readonly>} + */ +function providerByTransport() { + return Object.freeze({ runtime: DEFAULT_ENGINE_PROVIDER_ID }); +} + +/** + * Resolve the provider that answers the provider-write family on + * `transport`, or `null` when none is registered yet. + * + * @param {string} transport + * @returns {{id: string, transport: string, capabilities: object}|null} + */ +export function resolveProviderWriteProvider(transport) { + const providerId = providerByTransport()[transport]; + if (!providerId) return null; + return getEngineProvider(providerId); +} + +/** + * HARD gate for both endpoints. Throws + * `EngineCapabilityNotSupportedError` for a declared `none`, and for a + * `partial` naming this endpoint's sub-item; the router maps it to the + * shared 501 body from `errors.js#engineCapabilityHttpResponse`. + * + * An unregistered transport is NOT a 501. It returns + * `gate: "unregistered-transport"` and lets the write proceed, which is + * what every other family in this migration does and the reason M4 + * exists: the transport table is empty until M4, and a 501 that meant + * "nobody has written M4 yet" would be a lie about the engine. + * + * @param {string} endpoint A key of PROVIDER_WRITE_ENDPOINTS. + * @param {string} transport + * @returns {{endpoint: string, provider: string|null, capability: string, + * subItem: string, enforcement: "hard", gate: string}} + */ +export function assertProviderWriteCapability(endpoint, transport) { + const need = PROVIDER_WRITE_ENDPOINTS[endpoint]; + if (need === undefined) { + // Caller confusion, not an engine limitation: a plain Error, so a + // typo in webui's own key can never be reported to an operator as + // an engine limitation. + const err = new Error( + `assertProviderWriteCapability: "${endpoint}" is not part of the provider-write family ` + + `(known: ${Object.keys(PROVIDER_WRITE_ENDPOINTS).join(", ")})`, + ); + err.code = "unknown_provider_write_endpoint"; + throw err; + } + const base = { + endpoint, + provider: null, + capability: need.capability, + subItem: need.subItem, + enforcement: need.enforcement, + }; + const provider = resolveProviderWriteProvider(transport); + if (!provider) return { ...base, gate: "unregistered-transport" }; + assertEngineCapability(provider.capabilities, need.capability, provider.id, need.subItem); + return { ...base, gate: "checked", provider: provider.id }; +} + +/** + * The records a #63 body resolves to, ready to persist. + * + * Two rules, both pre-existing and both about WHICH key survives a + * round trip: + * + * 1. The keep-key convention. An incoming `auth.apiKey` that is + * empty OR absent means "do not change the existing key", and the + * previous value is copied onto the record before validation. + * `existing` must be the STORE's records, not the merged + * catalogue: a key sourced from the env or cwd layer is + * deployment-owned, and copying one into the store would pin a + * deployment secret to operator-managed disk where the env layer + * can no longer rotate it. + * 2. The whole catalogue is the body. There is no patch semantics, + * and there was none before this batch; a provider the body omits + * is a provider the operator removed. + * + * Pure — no IO, no clock, no store access — so the convention's scope + * is a thing a test can pin rather than a comment. + * + * @param {object[]} incoming The body's `providers`. + * @param {object[]} existing The store's current records. + * @returns {object[]} + */ +export function planProviderCatalogueWrite(incoming, existing) { + return applyKeepKeyConvention(existing || [], incoming || []); +} + +/** + * #63 / #66 — persist a provider catalogue to the store. + * + * The commit is ONE atomic rename of the whole `config.yaml`, and + * every outcome is a value: + * + * { ok: true, written, keys, preserved, records } — the store now + * holds exactly `records`, with the marker stamped (this is the + * write that also closes the deprecated `providers.json`). + * { ok: false, code: "ENGINE_STORE_UNREADABLE" } — `config.yaml` + * does not parse. The file is left exactly as it is, because + * overwriting it would destroy whatever the operator had in the + * sections this batch does not own. The route answers 500. + * { ok: false, code: "ENGINE_STORE_WRITE_FAILED" } — the write + * itself failed. The previous document is intact; the route + * answers 500. + * + * `records` is the persisted catalogue in the order it will be read + * back, which is the order the operator PUT — the store is a YAML + * mapping, and this is what keeps the catalogue's order stable across + * a round trip through it. + * + * @param {object} options + * @param {object[]} options.records Normalised records to persist. + * @param {string} [options.configPath] + * @returns {Promise<{ok: boolean, written?: boolean, keys?: string[], + * preserved?: string[], records?: object[], code?: string, error?: string}>} + */ +export async function commitProviderCatalogueWrite(options = {}) { + const store = readProviderStore(options); + if (!store.ok) { + return { + ok: false, + code: "ENGINE_STORE_UNREADABLE", + error: store.error, + }; + } + const result = await commitProviderStoreWrite({ + configPath: store.configPath, + raw: store.raw, + tree: store.tree, + records: options.records || [], + // Every store write stamps the marker. A PUT is a full + // replacement, so it has just made the deprecated file + // irrelevant whether or not one existed, and stamping it here is + // what makes "delete every provider" stick: without the marker a + // later read would go back to the deprecated file and resurrect + // what the operator removed. + migrated: true, + }); + if (!result.ok) return result; + return { + ok: true, + written: result.written, + keys: result.keys, + preserved: result.preserved, + records: options.records || [], + }; +} + +// --------------------------------------------------------------------------- +// KNOWN DEBT +// --------------------------------------------------------------------------- +// +// 1. THE ACP SINGLETON TEARDOWN IS STILL THE ROUTE'S, AND IT IS NOW +// A NO-OP FOR MOST DEPLOYMENTS. #63 used to call +// `shutdownMcodeAcpSingleton()` after a successful projection so +// the next catalogue call would re-read `config.yaml` into a +// fresh subprocess. That reason survives — the subprocess caches +// its config — but the call is a no-op under the `runtime` +// transport, where the host is the in-process one this module +// wrote to directly. It is left in place because the `acp` +// transport still spawns the child, and removing it on an +// assumption about M4's provider table is how the next batch +// inherits a stale-config bug nobody can reproduce. +// +// 2. THE ENGINE'S OWN WRITER (`updateLocalByokConfig`) IS STILL NOT +// USED. It would give the write a cross-process lock, which +// matters only when a `mcode provider` CLI command races a webui +// PUT — and it drags `js-yaml` plus `proper-lockfile` into the +// webui bundle for a one-way write webui makes rarely. The atomic +// rename this module uses is what makes the PUT atomic *within* +// the process, which is the failure the death line named. The +// cross-process case is real and unclaimed; the argument for +// leaving it is the same one the module it replaces recorded, and +// it is recorded here rather than silently re-decided. +// +// 3. A WEBUI PROVIDER WHOSE ENGINE KEY COLLIDES WITH A FOREIGN ENTRY +// OVERWRITES THAT ENTRY, exactly as the double-write it replaces +// did. `buildProviderStoreWrite` writes every record first and +// carries foreign entries after, so a foreign key that a webui +// provider also claims is lost. +// +// The obvious fix — suffix the webui key — is worse than the bug +// for a reason specific to this batch: the key IS the runtime id. +// `custom_provider:/` is what `applyRecordedModel` and +// B4's `resolveModelId` match a pre-session pick against, so a +// silent rename turns a user's already-chosen model into an +// unresolvable one. Choosing between "an operator's hand-written +// entry disappears" and "a recorded model stops resolving" is a +// product decision, not a refactor's. The pre-existing behaviour +// is preserved and pinned by a named test so the choice stays +// visible. diff --git a/packages/webui/server/lib/engine-catalogue.js b/packages/webui/server/lib/engine-catalogue.js index 82630a0a..811f5022 100644 --- a/packages/webui/server/lib/engine-catalogue.js +++ b/packages/webui/server/lib/engine-catalogue.js @@ -15,12 +15,15 @@ // providers (minimax-cn / deepseek-cn / zai-max / zai-pro / // kimi-taozi / opencode-go × 3 / nousresearch) with 30+ models. // -// Ticket 05 added the *write* half: webui's PUT handler now -// projects its providers.json into the engine's `custom_provider` -// tree (`server/lib/engine-provider-sync.js`) with the -// `_webui_owned: true` ownership marker. Foreign entries (added -// via `mcode provider add` or hand-edited by the operator) are -// preserved through every sync. +// Ticket 05 added the *write* half: webui projected its +// providers.json into the engine's `custom_provider` tree with the +// `_webui_owned: true` ownership marker. M3-B11 made that tree the +// STORE rather than a projection of a second file +// (`server/engine/provider-store.js`), so this module's read below +// is of the primary source rather than of a mirror. Foreign entries +// (added via `mcode provider add` or hand-edited by the operator) +// are still preserved through every write — the ownership rule did +// not move with the file. // // Ticket 06 closes the loop on the *read* half: the same tree is // also a catalogue source for `/api/models`. Webui-only fields @@ -92,7 +95,7 @@ import { existsSync, readFileSync } from "node:fs"; import yaml from "js-yaml"; -import { getEngineConfigPath } from "./engine-provider-sync.js"; +import { getEngineConfigPath } from "../engine/provider-store.js"; // ===================================================================== // Builtin-model thinking projection (ticket 36 — builtin-thinking-levels). diff --git a/packages/webui/server/lib/engine-provider-sync.js b/packages/webui/server/lib/engine-provider-sync.js deleted file mode 100644 index 1677e284..00000000 --- a/packages/webui/server/lib/engine-provider-sync.js +++ /dev/null @@ -1,550 +0,0 @@ -// webui/server/lib/engine-provider-sync.js -// Engine-side projection of webui's providers.json (ticket 05). -// -// Background — the root cause pinned by ticket 05: -// -// Webui keeps its own `providers.json` (a v2 catalogue, layered merge, -// keep-key convention) so the dialog can show provider groups, "enabled" -// toggles and "configured with key" greying without talking to the engine -// on every render. The engine has its own `custom_provider` registry -// (packages/local-runtime-v2/src/service/model-system/management/service-custom-provider-operations.ts) -// that is the ONLY source the `model` config option -// (packages/tui/src/acp/control-state.ts) advertises. Pre-ticket-05, the -// two never met: webui's PUT handler persisted its file, the engine kept -// its `custom_provider` from `config.yaml` independent of the webui — -// selecting a provider model in the dialog was UI-only and -// `applyRecordedModel` had nothing to match against, so the engine -// silently kept its default. -// -// Fix — write the engine's `custom_provider` shape right here so the engine -// sees the same providers the webui advertises: -// -// webui provider (v2) → engine custom_provider entry -// { id, label, protocol, auth, models } -// → { name, kind: 'custom', enabled, api, options: {apiKey, baseURL, authMode, headers?}, -// models: { modelId: { limit: {context}, thinking: {effortOptions}, modalities } } } -// -// Conversion rules (pinned by tests): -// - provider id → engine provider key (sluggified so the -// `custom_provider:/...` runtime id stays alphanumeric + dot + -// underscore + hyphen) -// - `auth.headers` → `options.headers`, copied verbatim and omitted -// when empty. The runtime merges these into every upstream request -// for the provider, which is what makes a header typed in the -// add-provider dialog actually take effect. -// - provider id → engine provider key (sluggified so the -// `custom_provider:/...` runtime id stays alphanumeric + dot + -// underscore + hyphen) -// - webui `enabled: false` AND/OR empty apiKey AND/OR missing baseURL → -// entry is OMITTED (the engine's `custom_provider` rejects apiKey-less -// `createUserProvider` and the `enabled: false` switch turns the entry -// invisible to `listByokRuntimeModels`) -// - protocol → api format: openai → openai-completions, -// anthropic → anthropic-messages, gemini → openai-completions (Gemini's -// OpenAI-compat endpoint is what `auth.type: 'byok'` callers point at; -// the engine does not have a native Gemini api format) -// - model id → engine model key; thinkingLevels → thinking.effortOptions, -// modalities → modalities.input, contextLimit → limit.context -// - auth.type === 'coding-plan' → SKIP (engine handles coding-plan via -// its own OAuth / Codex / Claude Code flows — out of scope for the -// byok projection) -// -// Ownership rule — ticket 05 acceptance (merge-over-replace): -// -// The webui's PUT does NOT replace the engine's whole `custom_provider` -// tree. A manually-added operator entry (e.g. via `mcode provider add` -// on the engine CLI) is FOREIGN to the webui and must survive an -// unrelated webui PUT. The hard destruction class this commit is -// closing: pre-fix sync, removing a webui provider OR running an -// empty-eligible-list sync would silently DROP a foreign entry the -// operator typed in by hand. -// -// Ownership is tracked per-entry by an opaque marker field: -// -// _webui_owned: true ← every entry webui writes carries this -// -// The engine ignores unknown fields (it parses via js-yaml with no -// schema-rejection; see `parseCustomProvidersConfig` in -// `packages/config/src/byok-config.ts`), so the marker is engine-safe. -// The sync algorithm: -// -// existing engine keys ∩ eligible webui keys → UPDATE in place -// existing engine keys ∖ eligible webui keys: -// _webui_owned === true → DELETE (webui owns it, -// operator removed the -// webui provider) -// _webui_owned !== true (or missing) → PRESERVE (foreign; -// operator owns it; -// webui leaves it alone) -// eligible webui keys ∖ existing engine keys → ADD (new provider) -// -// This means a foreign `manual-only` provider stays in the engine -// tree even after every webui PUT, even after the operator deletes -// every webui-managed provider. The webui NEVER deletes a foreign -// entry. The only way to delete a foreign entry is the engine CLI's -// own `mcode provider delete` (or hand-editing `config.yaml`). -// -// Write strategy: -// -// The engine reads `/config.yaml` (the same dir the -// webui already knows about — see server/lib/config.js#resolveDataDir). -// We do an atomic tmp+rename YAML write keyed on `custom_provider` -// only; we never touch the operator's other engine config -// (provider.*, defaultModel, etc.). The engine subprocess running the -// singleton client has a stale `getConfig()` cache after a write — -// the route handler then calls `shutdownMcodeAcpSingleton()` so the -// next operation spawns a fresh subprocess that reads the new file. -// Brand-new prompt subprocesses spawned by `runMcodeAcp` always pick -// up the latest config, so the rest of the system stays in lockstep. -// -// File permissions — ticket 05 acceptance (0600): -// -// config.yaml carries the apiKey as plaintext. umask-default 0664 -// would expose the key to every user on the host. The engine's own -// `updateLocalByokConfig` writes 0600 (see -// `packages/config/src/local-model-provider-write.ts`); the helper -// matches. The new tmp file is created 0600, the rename preserves -// the mode on POSIX, and a final chmod pins it for platforms where -// the rename semantics differ. -// -// We deliberately do NOT route through the engine's -// `updateLocalByokConfig` (`@mavis/config`) — pulling that into the -// webui bundle would drag in js-yaml + proper-lockfile just for a -// one-way write we do rarely, and the lockfile is meaningful only when -// multiple `mcode` subprocesses are racing the same file (which the -// webui does not do — `mcode acp` does not edit `config.yaml` at -// runtime, only the `mcode provider add` CLI does, and that flow -// cannot run concurrently with a webui PUT in the same process tree). - -import { writeFile, rename, mkdir, chmod } from "node:fs/promises"; -import { dirname, join } from "node:path"; -import { homedir } from "node:os"; -import { existsSync, readFileSync } from "node:fs"; -import yaml from "js-yaml"; -import { randomBytes } from "node:crypto"; - -import { applyKeepKeyConvention } from "./providers-config.js"; - -// engine provider api formats — must match `MODEL_PROVIDER_APIS` in -// packages/local-runtime-v2/src/service/model-system/identity.ts (the -// engine rejects anything outside this set at `normalizeApiFormat`). -const WEBUI_PROTOCOL_TO_ENGINE_API = { - openai: "openai-completions", - anthropic: "anthropic-messages", - // gemini has no engine-native api; the OpenAI-compat endpoint is the - // usual `byok` target. Engine does not have a Gemini-specific format. - gemini: "openai-completions", -}; - -// Reserved engine keys — must NOT collide with the existing engine's -// internal provider ids (which would either shadow `minimax` or land in -// `RESERVED_CUSTOM_PROVIDER_KEYS` and be dropped). Mirrored from -// packages/config/src/byok-config.ts. -const RESERVED_ENGINE_KEYS = new Set([ - "minimax", - "minimax_api", - "provider", - "custom_provider", -]); - -const PROVIDER_KEY_REGEX = /^[A-Za-z0-9][A-Za-z0-9_.-]*$/; - -/** - * Ownership marker field. Every entry the webui writes carries - * `_webui_owned: true`. The engine ignores it (js-yaml parses the whole - * record and the engine's downstream consumers read named fields only); - * the field is the on-disk fingerprint the sync algorithm uses to - * distinguish webui-managed entries from operator-managed ones. - * - * The constant is exported only so tests can assert against it without - * drifting if the marker ever changes (rename = data loss for every - * operator-managed entry on the next sync). Do not rename lightly. - */ -export const WEBUI_OWNED_MARKER = "_webui_owned"; - -/** - * Resolve the engine's data directory. - * - * The engine resolves its own data dir via `packages/config/src/config.ts`: - * MINIMAX_DATA_DIR || MAVIS_DATA_DIR || ~/.minimax - * We mirror that exact resolution here so a webui-managed PUT lands in the - * directory the engine subprocess will read on next spawn. The webui itself - * already uses the same env precedence in server/lib/config.js — the - * resolver is the same shape, kept here as a copy so the helper has no - * cross-package import surface. - */ -export function resolveEngineDataDir() { - const env = (process.env.MINIMAX_DATA_DIR?.trim() || - process.env.MAVIS_DATA_DIR?.trim() || - ""); - if (env) return env; - return join(homedir(), ".minimax"); -} - -export function getEngineConfigPath() { - return join(resolveEngineDataDir(), "config.yaml"); -} - -/** Pure: a webui v2 id → an engine-safe provider key. */ -export function providerKeyFromId(id) { - const trimmed = (id || "").trim(); - if (!trimmed) return ""; - if (!PROVIDER_KEY_REGEX.test(trimmed)) return ""; - if (RESERVED_ENGINE_KEYS.has(trimmed)) { - return `${trimmed}-byok`; - } - return trimmed; -} - -/** - * Pure: webui model id → engine-safe model key. - * - * The engine accepts `/` inside model keys (the wire form - * `formatModelKey(, ) = /` - * uses `/` only as the *structural* separator between provider and - * model — `parseSourceQualifiedModelKey` splits on the FIRST `/`, so a - * model id that contains `/` is preserved as a single string after - * the split). Upstream catalogues commonly carry namespace-style model - * ids like `deepseek/x` or `z-ai/glm-5.3`; rejecting them here would - * drop the model from the engine sync and leave the engine unable to - * match the user's pre-session pick on session boot. - * - * Ticket 09-02: the previous `^[A-Za-z0-9][A-Za-z0-9_.-]*$` rejected - * every model id containing `/`, which silently dropped those entries - * from the sync. The actual engine constraints are weaker (any - * non-empty trimmed string is accepted as a Record key in the YAML - * custom_provider tree) so we widen to allow `/`. - * - * Other unsafe characters (whitespace, control codes, YAML structural - * tokens like `:`, `{}`, `[]`, `#`, `&`, `*`, `!`, `|`, `>`, `'`, - * `"`, `%`, `@`, `\``) still cause the engine's YAML parser or its - * custom_provider lookup to fail — those are rejected here so the - * sync never lands an unparseable entry on disk. - */ -export function modelKeyFromId(id) { - const trimmed = (id || "").trim(); - if (!trimmed) return ""; - // Reject whitespace, YAML structural tokens, and anything else - // the engine's byok-config parser would misinterpret. `/` is the - // only "extra" character we allow (the wire-form separator). - if (/[\s:#{}\[\]@&*!|>'"%`,]/.test(trimmed)) return ""; - // The model key must not start with `-` (YAML lists) or `&`/`*` (anchors) - if (/^[-&*]/.test(trimmed)) return ""; - return trimmed; -} - -/** Pure: webui v2 → engine custom_provider entry. Returns null when ineligible. */ -export function toEngineCustomProvider(provider) { - if (!provider || typeof provider !== "object") return null; - // coding-plan providers go through the engine's OAuth / Codex / - // subscription flows — out of scope for the byok projection. - if (provider.auth && provider.auth.type === "coding-plan") return null; - if (provider.enabled === false) return null; - const apiKey = - typeof provider.auth?.apiKey === "string" ? provider.auth.apiKey.trim() : ""; - if (!apiKey) return null; - const baseURL = - typeof provider.auth?.baseURL === "string" ? provider.auth.baseURL.trim() : ""; - if (!baseURL) { - // No baseURL → engine has nothing to call. Skip (matches the dialog's - // "configured without baseURL" grey-out: same semantics as no key). - return null; - } - const protocol = - typeof provider.protocol === "string" ? provider.protocol.trim() : "openai"; - const api = WEBUI_PROTOCOL_TO_ENGINE_API[protocol]; - if (!api) return null; - // Copy into a fresh object: the engine config is compared by value - // on the next sync, and handing it a live reference to the parsed - // providers.json would let a later mutation write through. - // Emptiness, not truthiness: `normaliseProvider` always materialises - // `auth.headers` (absent -> {}), and `{}` is TRUTHY, so the original - // `customHeaders ? …` test emitted `headers: {}` for every provider - // that never configured one. The end-to-end run caught this; the unit - // fixture, whose `auth` simply has no `headers` key at all, cannot. - const customHeaders = - provider.auth?.headers && - typeof provider.auth.headers === "object" && - !Array.isArray(provider.auth.headers) && - Object.keys(provider.auth.headers).length > 0 - ? { ...provider.auth.headers } - : null; - const providerKey = providerKeyFromId(provider.id); - if (!providerKey) return null; - const name = - typeof provider.label === "string" && provider.label.trim() - ? provider.label.trim() - : providerKey; - const models = {}; - if (Array.isArray(provider.models)) { - for (const m of provider.models) { - if (!m || typeof m !== "object") continue; - const modelKey = modelKeyFromId(m.id); - if (!modelKey) continue; - const engineModel = {}; - if ( - typeof m.label === "string" && - m.label.trim() && - m.label.trim() !== modelKey - ) { - engineModel.name = m.label.trim(); - } - if (typeof m.contextLimit === "number" && m.contextLimit > 0) { - engineModel.limit = { context: m.contextLimit }; - } - if ( - Array.isArray(m.thinkingLevels) && - m.thinkingLevels.length > 0 && - m.thinkingLevels.every((x) => typeof x === "string" && x.length > 0) - ) { - engineModel.thinking = { effortOptions: [...m.thinkingLevels] }; - } - if ( - Array.isArray(m.modalities) && - m.modalities.length > 0 && - m.modalities.every((x) => typeof x === "string" && x.length > 0) - ) { - engineModel.modalities = { input: [...m.modalities] }; - } - models[modelKey] = engineModel; - } - } - return { - key: providerKey, - entry: { - name, - kind: "custom", - enabled: true, - api, - options: { - apiKey, - baseURL, - authMode: "api-key", - // Custom headers are already validated by - // `normalizeCustomHeaders` on the PUT path, so this copy only - // has to be faithful. It is the load-bearing line for the - // whole feature: the runtime merges `options.headers` into - // every upstream request for this provider - // (`local-runtime-v2/.../catalog/provider-views.ts:218` → - // `mergeProviderHeaders(provider.options?.headers, ...)`), so - // without it a header the operator typed and read back would - // be stored, displayed, and never sent. Emitted only when - // non-empty so an untouched provider's engine entry keeps the - // exact shape it had before this field existed. - ...(customHeaders ? { headers: customHeaders } : {}), - }, - ...(Object.keys(models).length > 0 ? { models } : {}), - }, - }; -} - -/** - * Read the engine's existing `config.yaml` so a sync can preserve - * `provider.*`, `defaultModel`, and any operator-managed sections the - * webui must not touch. - * - * Returns a plain object (possibly empty). A missing file is not an - * error — the writer creates it. - */ -function readEngineConfigRaw(configPath) { - if (!existsSync(configPath)) return {}; - try { - const raw = readFileSync(configPath, "utf8"); - const parsed = yaml.load(raw); - if (!parsed || typeof parsed !== "object" || Array.isArray(parsed)) return {}; - return parsed; - } catch { - // YAML parse error — the engine will surface this on its next read; - // we'd rather write a broken-than-empty file than drop the operator's - // section. Bail out as "no-op" so the route can answer with a clear - // structured error. - return null; - } -} - -/** - * Atomic YAML write + 0600 permission pin. - * - * Two-step: write the new content to a tmp file (mode 0600), then - * rename. The rename preserves POSIX mode, but we chmod the target - * afterwards as belt-and-suspenders (some filesystems and Windows - * edge cases drop the mode on rename). The engine's own - * `updateLocalByokConfig` does the same dance — see - * `packages/config/src/local-model-provider-write.ts`. - */ -async function atomicWriteYaml0600(configPath, object) { - await mkdir(dirname(configPath), { recursive: true }); - const tmp = join( - dirname(configPath), - `.config-tmp-${randomBytes(6).toString("hex")}`, - ); - // mode 0600 — owner read/write only. The file carries plaintext - // apiKeys; any looser mode would expose them to other users on the - // host. - await writeFile(tmp, yaml.dump(object, { indent: 2, lineWidth: -1, noRefs: true }), { - encoding: "utf8", - mode: 0o600, - }); - await chmod(tmp, 0o600); - await rename(tmp, configPath); - await chmod(configPath, 0o600); -} - -/** - * Is this engine entry webui-owned (vs operator/foreign)? - * - * The marker is set on every entry the webui writes. Operators who add - * custom providers via the engine CLI never set it, so the marker - * distinguishes the two ownerships on disk. - */ -function isWebuiOwned(entry) { - return !!(entry && typeof entry === "object" && entry[WEBUI_OWNED_MARKER] === true); -} - -/** - * Build the next `custom_provider` map by merging the eligible webui - * projection over the existing engine tree (see the file-level - * ownership rule). Pure: no IO, no writes. - * - * Algorithm: - * 1. eligible webui keys ⊂ existing engine keys → UPDATE in place - * (the webui entry replaces the engine entry; we still carry - * `_webui_owned: true` so the next sync treats it as webui). - * 2. existing engine keys ∖ eligible webui keys: - * marker === true → DELETE - * marker !== true → PRESERVE (foreign, operator-owned) - * 3. eligible webui keys ∖ existing engine keys → ADD - * - * The returned map carries the ownership marker on every webui-owned - * entry; foreign entries are passed through verbatim (including their - * original structure — we never edit a foreign entry's fields). - */ -function mergeCustomProviderTree(existingCustom, eligible) { - const next = {}; - // Carry forward any foreign entries that the engine already has. - // We do this first so the eligibility-driven UPDATE/DELETE pass - // below only touches webui-owned keys. - for (const [key, entry] of Object.entries(existingCustom || {})) { - if (!entry || typeof entry !== "object") continue; - if (!isWebuiOwned(entry)) { - next[key] = entry; - } - } - const eligibleKeys = new Set(); - for (const { key, entry } of eligible) { - eligibleKeys.add(key); - // Strip the existing entry (if any) — we'll replace it below with - // the new webui projection. The marker on the new entry will be - // preserved across sync cycles. - delete next[key]; - // Build the new entry with the ownership marker set. We stamp - // the marker AFTER cloning so we don't mutate the input (the - // route caches the eligible list across calls). - next[key] = { ...entry, [WEBUI_OWNED_MARKER]: true }; - } - // No further work: webui-owned entries that are no longer eligible - // were omitted from `next` (we never re-add them), so the DELETE - // step is implicit. - void eligibleKeys; - return next; -} - -/** - * Project a list of webui v2 providers into the engine's `custom_provider` - * tree and write the engine's `config.yaml` atomically (mode 0600). - * - * Ownership rule (see file header): the merge preserves operator / - * foreign entries — only entries webui wrote (the `_webui_owned` - * marker is the on-disk fingerprint) are added / updated / removed. - * Foreign entries survive every webui PUT. - * - * Returns: - * - { ok: true, written: true|false, keys: [providerKey, ...], - * preserved: [foreignKey, ...] } - * `written: false` means there was nothing eligible to write (the - * webui's catalogue is empty or every provider was ineligible); the - * foreign set is still reported so the route can log it. `keys` - * lists the webui keys the sync touched (added/updated). `preserved` - * lists the foreign keys that survived untouched. - * - { ok: false, code: 'ENGINE_SYNC_FAILED', error: string } - * - * Never throws — surfaces every failure as a structured result so the - * route handler can attach the error to the response without try/catch. - */ -export async function syncProvidersToEngine(providers, opts = {}) { - const configPath = opts.configPath || getEngineConfigPath(); - try { - const eligible = []; - for (const p of providers || []) { - const out = toEngineCustomProvider(p); - if (out) eligible.push(out); - } - const existing = readEngineConfigRaw(configPath); - if (existing === null) { - return { - ok: false, - code: "ENGINE_SYNC_FAILED", - error: `engine config at ${configPath} is unreadable (YAML parse error)`, - }; - } - // Preserve every operator-owned section; only `custom_provider` is - // touched. The engine's `defaultModel` (when pointing at a - // `custom_provider:/`) is left to the operator. - const next = { ...existing }; - const existingCustom = - existing.custom_provider && typeof existing.custom_provider === "object" - ? existing.custom_provider - : {}; - const merged = mergeCustomProviderTree(existingCustom, eligible); - // List foreign keys we kept untouched, for the route response / - // log. Owned entries (webui + foreign-derived from marker) are - // excluded — only the operator-managed ones we preserved go here. - const preserved = []; - for (const key of Object.keys(merged)) { - const e = merged[key]; - if (!isWebuiOwned(e)) preserved.push(key); - } - if ( - eligible.length === 0 && - Object.keys(merged).length === Object.keys(existingCustom).length && - Object.keys(merged).every((k) => existingCustom[k] === merged[k]) - ) { - // Nothing eligible AND the merged tree is byte-identical to - // the existing one (no webui-owned entries changed, no - // foreign entries added/removed). Skip the write — the engine's - // view of the world is unchanged, and a no-op write would - // still touch mtime / chmod. - return { ok: true, written: false, keys: [], preserved }; - } - next.custom_provider = merged; - await atomicWriteYaml0600(configPath, next); - return { - ok: true, - written: true, - keys: eligible.map((e) => e.key), - preserved, - }; - } catch (e) { - return { - ok: false, - code: "ENGINE_SYNC_FAILED", - error: e && e.message ? e.message : String(e), - }; - } -} - -/** - * Compatibility wrapper for the routes that already have the raw PUT body - * in hand (they apply keep-key convention themselves for the user-level - * file write). This wraps the body in the same shape the route would pass - * to `syncProvidersToEngine` after normalisation, so the sync sees the - * same provider list the engine should advertise. - */ -export async function syncProvidersFromPutBody(parsedBody, existingUserLevel, opts) { - const incoming = Array.isArray(parsedBody?.providers) ? parsedBody.providers : null; - if (incoming === null) { - return { ok: false, code: "BAD_BODY", error: "providers must be an array" }; - } - const resolved = applyKeepKeyConvention(existingUserLevel || [], incoming); - return syncProvidersToEngine(resolved, opts); -} \ No newline at end of file diff --git a/packages/webui/server/lib/mcode-acp.js b/packages/webui/server/lib/mcode-acp.js index c70a0f00..d5d0b6f4 100644 --- a/packages/webui/server/lib/mcode-acp.js +++ b/packages/webui/server/lib/mcode-acp.js @@ -22,6 +22,7 @@ import { pushAlert, getCidsByMcodeSession, updateRunSid, + moveRunSession, } from "./state-bus.js"; import { applyMavisUsageToCs } from "./mavis-usage.js"; import { mcodePermissionToWebui } from "./mcode-rpc.js"; @@ -272,6 +273,26 @@ function matchesModelId(recorded, engineCurrent, modelOption) { * `resolveModelId` covers this case before the name-match runs. */ +/** + * The engine's stderr tail, shaped for an alert's `data`. + * + * The engine announces its own failures on stderr and dies; the webui is + * the one that raises the crash alert. Without carrying the tail across, + * every engine failure collapses to `mcode acp exited (code=1)` — an + * operator cannot act on an exit code, only on the engine's line + * (`agent_name_conflict_migration_failed:lock`, a config parse error, a + * missing binary). The client already bounds and truncates it; see + * `McodeAcpClient#stderrTail` in packages/webui/acp.mjs. + * + * Returns `{}` — not `{ stderrTail: "" }` — when the engine said nothing, + * so a silent failure produces byte-identical alert data to what it + * produced before this helper existed. + */ +function acpStderrData(client) { + const tail = client && typeof client.stderrTail === "string" ? client.stderrTail : ""; + return tail ? { stderrTail: tail } : {}; +} + // Exported for unit tests (test/lib/mcode-acp-note.test.js extends to // cover applyRecordedModel's resolution logic). The pre-session model // apply needs to handle three input forms without regressing, so the @@ -397,6 +418,24 @@ export async function runMcodeAcp(content, opts = {}) { } catch (e) { console.warn(`[webui] bindDraftToMcodeSid: ${e.message}`); } + // P16 — the claim follows the conversation's identity. A first + // turn's draft record is promoted to the engine `mvs_` id right + // here, and `cs.sessionId` follows it, so the key the run was + // claimed under is no longer the key the view presents. Without + // re-keying, a send arriving a moment later presents a key no + // live run holds: `beginRun` cannot see the running turn, acks + // the send, and hands a second turn to an engine session that is + // already executing — while the new turn's echo lands in a + // `cs.chat`/record the run-mirror then writes over, so the user + // sees a message the engine ran and the webui has no record of + // (UAT 2026-10-03 16点轮 异常 #1). Re-keyed HERE, at the only + // instant the identity changes, and not in `handleSend`, which + // cannot observe it. `moveRunSession` records the retired key as + // an alias, so the `endRun(cid, runSessionId)` in the caller's + // `finally` still finds and releases this claim. + if (stillViewingAtBind && sid && cs.sessionId !== owningWebuiSessionId) { + moveRunSession(cid, owningWebuiSessionId, cs.sessionId); + } // First-turn session-busy guard: `handleSend` claimed the run with // `beginRun(cid, cs.mcodeSessionId, cs.sessionId)` BEFORE this turn // existed, so on @@ -431,7 +470,7 @@ export async function runMcodeAcp(content, opts = {}) { src: "mcode-acp", cid: cid || null, sessionId: sid || null, - data: { phase: "start-or-load" }, + data: { phase: "start-or-load", ...acpStderrData(client) }, }); return { status: "failed", @@ -1330,7 +1369,7 @@ function streamAcpPrompt( src: "mcode-acp", cid: cid || null, sessionId: sid || null, - data: { phase: "promise-catch" }, + data: { phase: "promise-catch", ...acpStderrData(client) }, }); finalize(); }); @@ -1475,6 +1514,16 @@ export async function runMcodeRuntime(content, opts = {}) { } catch (e) { console.warn(`[runtime-send] bindDraftToMcodeSid: ${e.message}`); } + // P16 — the claim follows the conversation's identity, same instant and + // same reason as the ACP path above (see the full note there): the + // promotion rewrote `cs.sessionId`, and a run still claimed under the + // retired draft key is invisible to `beginRun`, so a send arriving now + // is acked and handed to an already-busy engine session. Both + // transports must do this; a guard that exists on one only is the same + // hole with a different transport name. + if (stillViewingAtBind && cs.sessionId !== owningWebuiSessionId) { + moveRunSession(cid, owningWebuiSessionId, cs.sessionId); + } // First-turn session-busy guard, backfilled at the same moment as the // ACP path: `handleSend` claimed the run before this turn existed, so // on a session's first turn the claim was registered with diff --git a/packages/webui/server/lib/providers-config.js b/packages/webui/server/lib/providers-config.js index a624aece..4c5c1ca1 100644 --- a/packages/webui/server/lib/providers-config.js +++ b/packages/webui/server/lib/providers-config.js @@ -68,9 +68,9 @@ // repeated reads of the same path on the same tick are coalesced by // the routes themselves (handleGetProviders / handleGetModels). -import { existsSync, readFileSync, writeFileSync, renameSync, mkdirSync } from "node:fs"; +import { existsSync, readFileSync } from "node:fs"; import { homedir } from "node:os"; -import { join, dirname } from "node:path"; +import { join } from "node:path"; // ===================================================================== // Constants @@ -194,21 +194,11 @@ function safeReadJson(path) { } } -/** - * Atomic write: write `.tmp` then rename to ``. A half-written - * file on disk would be a config-load hazard the next PUT reads back into. - */ -function atomicWriteJson(path, value) { - mkdirSync(dirname(path), { recursive: true }); - const tmp = `${path}.tmp`; - writeFileSync(tmp, JSON.stringify(value, null, 2), "utf8"); - renameSync(tmp, path); -} - // ===================================================================== // Validation / normalisation // ===================================================================== + function str(v, fallback = "") { return typeof v === "string" ? v : fallback; } @@ -484,15 +474,30 @@ function mergeProvider(lower, higher) { * The cwd layer is intentionally skipped when `MCODE_WEBUI_MODELS_CONFIG` * is set (env layer "is" the cwd path; two layers pointing at the same * file would double-count). + * + * `opts.userLayer` (batch B11) REPLACES the user layer with an + * already-resolved provider list, which is how the engine's provider + * store takes over as the authority while the env and cwd layers keep + * their existing precedence, their existing per-call re-read, and their + * existing "deployment-owned, never written" property. The default — + * no `opts` — is the deprecated user file, so this module stays + * usable (and testable) on its own. + * + * @param {{userLayer?: object[]}} [opts] + * @returns {{version: number, providers: object[], sources: object}} */ -export function loadProvidersConfig() { +export function loadProvidersConfig(opts = {}) { const envPath = process.env.MCODE_WEBUI_MODELS_CONFIG; const cwdPath = envPath ? null : join(process.cwd(), "models.json"); const userPath = getUserLevelPath(); const envLayer = envPath ? readLayer(envPath) : null; const cwdLayer = cwdPath ? readLayer(cwdPath) : null; - const userLayer = existsSync(userPath) ? readLayer(userPath) : null; + const userLayer = Array.isArray(opts.userLayer) + ? { providers: opts.userLayer } + : existsSync(userPath) + ? readLayer(userPath) + : null; const layers = [userLayer, cwdLayer, envLayer]; // lowest -> highest priority const sources = { @@ -767,45 +772,23 @@ export async function testProvider({ protocol, auth, timeoutMs }) { } // ===================================================================== -// Persisted PUT (user-level write) +// Persistence — MOVED (batch B11) // ===================================================================== - -/** - * Validate-and-persist the incoming PUT body to the user-level file. - * Returns the persisted (normalised) config on success; on failure a - * `{ ok: false, error }` shape with a per-field message so the API - * can answer 400 without leaking internal stack traces. - * - * Note: the PUT handler is the ONLY write path for the user-level - * file. The env / cwd layers are deployment-owned and never written. - */ -export function writeProvidersConfig(parsed) { - if (!parsed || typeof parsed !== "object") { - return { ok: false, code: "BAD_BODY", error: "body is not an object" }; - } - const norm = normaliseConfig(parsed); - if (!norm) { - return { ok: false, code: "BAD_BODY", error: "no providers in body" }; - } - if (norm.warnings && norm.warnings.length > 0) { - return { - ok: false, - code: "BAD_BODY", - error: norm.warnings.join("; "), - }; - } - const path = getUserLevelPath(); - try { - atomicWriteJson(path, { version: SCHEMA_VERSION, providers: norm.providers }); - } catch (e) { - return { - ok: false, - code: "WRITE_FAILED", - error: e && e.message ? e.message : String(e), - }; - } - return { ok: true, path, providers: norm.providers }; -} +// +// `writeProvidersConfig` and its `atomicWriteJson` helper used to live +// here. They are gone with the dual-source arrangement they served: +// `providers.json` is no longer written by anything, and the store that +// replaced it is written by `engine/provider-store.js` with a different +// shape (YAML, mode 0600, one rename), a different ownership rule +// (foreign engine entries survive) and a different failure surface (an +// unreadable engine config is refused rather than overwritten). +// +// What this module still owns, and why it is the right owner: the +// SCHEMA. Normalisation, validation, masking, the layered resolution +// and the connectivity probe are all still about what a provider +// record MEANS, and a write target that changed does not change any of +// them. `loadProvidersConfig({userLayer})` is the seam the new store +// reads through. /** * Used by tests / routes that want to assert "plaintext key was never diff --git a/packages/webui/server/lib/state-bus.js b/packages/webui/server/lib/state-bus.js index f526b9e1..cc643edb 100644 --- a/packages/webui/server/lib/state-bus.js +++ b/packages/webui/server/lib/state-bus.js @@ -795,11 +795,30 @@ const runsByCid = new Map(); // cid -> Map const runsBySid = new Map(); // engineSessionId -> run let runCount = 0; // live turns across every cid — the real resource count -/** The run registry entry for one conversation of one tab, or null. */ +/** + * The run registry entry for one conversation of one tab, or null. + * + * A conversation's key is NOT stable: a first turn's draft record is + * promoted to the engine identity mid-turn (`promoteDraftToMcodeSid` + * rewrites the record id and follows it on `cs.sessionId`), and + * `moveRunSession` re-registers the claim under the new key. The key the + * claim was TAKEN under therefore stops matching the view, and a second + * send into that conversation would look like a fresh conversation. The + * entry keeps the retired keys in `aliases` for exactly that reason — a + * send carrying a key this run has already held is a send into a + * conversation that is already running, and must be refused (P16: the + * guard missing that let a second turn be acked and handed to the engine + * while its echo was lost from the persisted record). + */ function runFor(key, webuiSessionId) { const m = runsByCid.get(key); if (!m) return null; - return m.get(webuiSessionId) || null; + const direct = m.get(webuiSessionId) || null; + if (direct) return direct; + for (const entry of m.values()) { + if (entry.aliases && entry.aliases.has(webuiSessionId)) return entry; + } + return null; } /** True when any conversation of this tab is streaming to the engine. */ @@ -869,7 +888,7 @@ export function beginRun(cid, sid, webuiSessionId = null) { limit: MAX_CONCURRENT, }; } - const run = { cid: key, webuiSessionId: wsid, sid: sid || null, startedAt: Date.now(), bufferSid: null }; + const run = { cid: key, webuiSessionId: wsid, sid: sid || null, startedAt: Date.now(), bufferSid: null, aliases: new Set() }; const m = runsByCid.get(key) || new Map(); m.set(wsid, run); runsByCid.set(key, m); @@ -1103,7 +1122,7 @@ export function endRun(cid, webuiSessionId = null) { const entry = runFor(key, wsid); if (!entry) return; const m = runsByCid.get(key); - m.delete(wsid); + m.delete(entry.webuiSessionId); if (m.size === 0) runsByCid.delete(key); runCount -= 1; // Only drop the sid claim if this very run still owns it — a later run on @@ -1123,6 +1142,14 @@ export function endRun(cid, webuiSessionId = null) { * * Idempotent, and a no-op when `to` is already claimed by another run. * + * The retired `from` key is remembered on the entry (`aliases`, see + * `runFor`): a turn that outlives its own conversation key — the draft + * record it was claimed under is promoted to the engine `mvs_` id while + * the turn runs — must still be findable by the key `handleSend`'s + * `finally` releases, and by the key the NEXT send presents. Without the + * alias the release silently misses and the claim leaks until the process + * ends, refusing every later send in that conversation. + * * @returns {boolean} true when the run now lives under `to` */ export function moveRunSession(cid, from, to) { @@ -1132,9 +1159,13 @@ export function moveRunSession(cid, from, to) { if (fromKey === toKey) return runFor(key, toKey) !== null; const entry = runFor(key, fromKey); if (!entry) return false; - if (runFor(key, toKey)) return false; - runsByCid.get(key).delete(fromKey); + // A key this very entry retired is not a collision — moving back onto + // it is the same run, and refusing it would strand the claim under a key + // the view no longer uses. + if (runFor(key, toKey) && runFor(key, toKey) !== entry) return false; + runsByCid.get(key).delete(entry.webuiSessionId); runsByCid.get(key).set(toKey, entry); + entry.aliases.add(entry.webuiSessionId); entry.webuiSessionId = toKey; return true; } diff --git a/packages/webui/server/routes/chat.js b/packages/webui/server/routes/chat.js index 96e45071..3991b32f 100644 --- a/packages/webui/server/routes/chat.js +++ b/packages/webui/server/routes/chat.js @@ -179,11 +179,23 @@ export async function handleSend(req, res, ctx) { let runSessionId = (cs && cs.sessionId) || null; const claim = beginRun(cid, cs && cs.mcodeSessionId, runSessionId); if (!claim.ok) { + // A refusal is a decision, and the browser shows this `error` string + // verbatim in the composer's banner. The internal `detail` above is + // written for the server log ("another window" is wrong for the + // common case — the very same tab sending again a moment later), so + // the user-facing field carries the decision and the next action + // instead, and `reason` stays the stable machine-readable key. This + // refusal is TERMINAL for the send: nothing below this point runs, so + // the engine is never handed a prompt the webui will not record + // (P16 — a refused send must never reach the engine). + const busy = claim.reason === "cid-busy" || claim.reason === "session-busy"; res.writeHead(409, { "Content-Type": "application/json; charset=utf-8" }); return res.end( JSON.stringify({ ok: false, - error: claim.detail, + error: busy + ? "This conversation is already running a turn. The message was NOT delivered — wait for the turn to finish, then send it again." + : claim.detail, reason: claim.reason, ...(claim.reason === "at-capacity" ? { running: claim.running, limit: claim.limit } diff --git a/packages/webui/server/routes/providers.js b/packages/webui/server/routes/providers.js index dedfddd8..4fdb6ed5 100644 --- a/packages/webui/server/routes/providers.js +++ b/packages/webui/server/routes/providers.js @@ -2,77 +2,133 @@ // GET /api/providers, PUT /api/providers, POST /api/providers/test, // GET /api/providers/presets, POST /api/providers/preset/:id/enable // -// Provider configuration v2 — the management surface behind the -// schema and layered-resolution contract in -// `lib/providers-config.js`. The routes: +// The provider management surface, and — since batch B11 — a THIN one. +// The endpoint contracts, the wire shapes, the masking rule, the SSE +// frame and the connectivity probe all live where they always did. The +// three things that moved out are the ones that were never really this +// route's business: // -// GET /api/providers — full (masked) catalogue -// + resolved layers + sources. -// PUT /api/providers — validate + persist to -// user-level file + reload -// + SSE broadcast. -// POST /api/providers/test — local key format check -// first, then a protocol- -// minimal connectivity probe. -// GET /api/providers/presets — built-in preset -// templates, each with an -// `enabled` flag indicating -// whether the preset id is -// already configured. -// POST /api/providers/preset/:id/enable — materialise a preset -// template into the -// user-level file as -// enabled (PUT semantics + -// hot apply). +// 1. WHICH FILE IS THE CATALOGUE, and what happens when the other one +// cannot be read. That decision — including the one-shot migration +// of the deprecated `providers.json` and the fallback to it when +// the migration fails — is `engine/provider-reads.js`. +// 2. THE GATE. Each of the five endpoints declares the engine +// capability it needs, and the two write endpoints gate HARD +// because their whole product is a state the engine reads and +// webui does not (`engine/provider-writes.js`). +// 3. THE WRITE. One atomic rename of the engine's `config.yaml`, +// with the ownership rule that keeps an operator's hand-written +// custom providers alive (`engine/provider-store.js`). // -// Security contract (pinned by tests): +// --------------------------------------------------------------------- +// Security contract (unchanged, and pinned by tests) +// --------------------------------------------------------------------- // - apiKey is masked in EVERY response path. The public shape is -// `auth: { type, hasKey, apiKeyMasked, baseURL }`. The route -// never returns the plaintext key, the masked form is the ONLY -// shape an apiKey can take on the wire. -// - The PUT handler writes the user-level file via atomic -// rename; the env / cwd layers are deployment-owned and never -// written by this handler. +// `auth: { type, hasKey, apiKeyMasked, baseURL }`. The route never +// returns the plaintext key; the masked form is the ONLY shape an +// apiKey can take on the wire. // - The probe handler rejects malformed keys locally — no network // call is made when `validateKeyFormat` returns `{ ok: false }`. -// - Probe requests send the apiKey ONLY to the configured -// baseURL; a structured error is returned when no baseURL is -// configured for the protocol. +// - Probe requests send the apiKey ONLY to the configured baseURL; +// a structured error is returned when no baseURL is configured. +// - `auth.headers` are NOT masked, by decision: they are +// operator-authored routing configuration, not a credential the +// server substitutes. +// +// --------------------------------------------------------------------- +// Storage contract (CHANGED by this batch, documented in both docs) +// --------------------------------------------------------------------- +// +// The catalogue now lives in the engine's `config.yaml` +// (`custom_provider`), not in `/providers.json`. The +// deprecated file is read until the store carries its migration +// marker, and is never written again; a migration that fails leaves +// it in charge, so the operator keeps the catalogue they had. The +// response keeps its `sources` and `userPath` fields and their +// values, because an operator diagnosing a missing provider needs to +// be told which file the server resolved — that question now has a +// different answer, and the bilingual docs carry it. // -// Hot-reload semantics: -// - PUT triggers `pushProvidersUpdated()`, which broadcasts a -// named `providers.updated` SSE event with the masked payload -// so the UI can refresh its catalogue without an extra round -// trip. The next `GET /api/models` reads the same layers and -// picks up the change immediately (the user-level file is -// re-read on every call — no in-process cache to invalidate). +// Hot-reload semantics: the store is re-read on every call, so the next +// `GET /api/models` picks a change up immediately, and the +// `providers.updated` SSE broadcast is how the UI learns about it +// without polling. import { Readable } from "node:stream"; import { - loadProvidersConfig, publicView, - writeProvidersConfig, testProvider as runProbe, getUserLevelPath, - applyKeepKeyConvention, loadUserLevelProviders, - normaliseProvider, + normaliseConfig, } from "../lib/providers-config.js"; -import { - syncProvidersToEngine, - syncProvidersFromPutBody, -} from "../lib/engine-provider-sync.js"; import { PROVIDER_PRESETS, publicPresetView, presetToMaterialised, getPresetById, } from "../lib/provider-presets.js"; +import { + assertProviderWriteCapability, + commitProviderCatalogueWrite, + planProviderCatalogueWrite, +} from "../engine/provider-writes.js"; +import { + checkProviderReadCapability, + readEngineProviderCatalogue, +} from "../engine/provider-reads.js"; +import { getEngineConfigPath, readProviderStore } from "../engine/provider-store.js"; import { pushStateFor, sseByCid } from "../lib/state-bus.js"; import { readJson } from "../lib/read-json.js"; import { shutdownMcodeAcpSingleton } from "../lib/acp-client.js"; +/** + * The active transport. Read through a function so a test can move it + * between two calls and so the module-scope import cost stays zero — + * the same rule every other gated route follows. + * + * @returns {string} + */ +function activeTransport() { + return process.env.MCODE_WEBUI_TRANSPORT || "acp"; +} + +/** + * Normalise a PUT body into records, or explain why it cannot be. + * Split out from the handler because the answer decides a 400 and a + * store write, and a route that inlines both makes the two look like + * one decision when they are two. + * + * @param {object} parsed The parsed body. + * @returns {{ok: true, records: object[]}|{ok: false, code: string, error: string}} + */ +export function planCatalogueFromBody(parsed) { + const norm = normaliseConfig(parsed); + if (!norm) return { ok: false, code: "BAD_BODY", error: "no providers in body" }; + if (norm.warnings && norm.warnings.length > 0) { + return { ok: false, code: "BAD_BODY", error: norm.warnings.join("; ") }; + } + return { ok: true, records: norm.providers }; +} + +/** + * Answer a plan failure. Every refusal on the write path is a 400 with + * the same body, and the one store-level failure that is not the + * operator's fault is a 500 — the split the pre-B11 handler made, kept + * exactly so the status a client sees for a bad body does not move. + * + * @param {{code: string, error: string}} failure + * @param {object} res + * @returns {number} The status written. + */ +function writePlanFailure(failure, res) { + const status = failure.code === "WRITE_FAILED" ? 500 : 400; + res.writeHead(status, { "Content-Type": "application/json; charset=utf-8" }); + res.end(JSON.stringify({ ok: false, code: failure.code, error: failure.error })); + return status; +} + /** * GET /api/providers — masked catalogue + resolved-layer summary. * @@ -84,14 +140,21 @@ import { shutdownMcodeAcpSingleton } from "../lib/acp-client.js"; * sources: { env, cwd, user }, // absolute paths (env is the * // MCODE_WEBUI_MODELS_CONFIG * // override or null) - * userPath: "..." // user-level file path + * userPath: "..." // the deprecated user-level file * } * - * `sources` is documented (not redacted) — operators need to see - * which file the server actually read. + * `sources` is documented (not redacted) — operators need to see which + * files the server actually resolved. + * + * The gate is soft, so it is called for its report and nothing else; + * the route does not branch on it. That is deliberate: a degraded + * provider still serves a well-defined catalogue, and hiding the + * endpoint would remove a working UI over a declaration about who + * would eventually answer it. */ -export function handleGetProviders(_req, res, _ctx) { - const cfg = loadProvidersConfig(); +export async function handleGetProviders(_req, res, _ctx) { + checkProviderReadCapability("GET /api/providers", activeTransport()); + const cfg = await readEngineProviderCatalogue(); res.writeHead(200, { "Content-Type": "application/json; charset=utf-8" }); return res.end( JSON.stringify({ @@ -105,21 +168,25 @@ export function handleGetProviders(_req, res, _ctx) { } /** - * PUT /api/providers — validate-and-persist to user-level file. + * PUT /api/providers — validate and persist the catalogue. * * Body shape (v2): * { version: 2, providers: [ { id, label, protocol, auth, models, ... } ] } * - * Behaviour: + * Behaviour, and the one line of it that is new: * - 400 + structured error when any provider fails validation. - * - 500 + structured error when the atomic write fails. + * - 500 + structured error when the store refuses the write (an + * unreadable `config.yaml`, or an I/O failure). The store is + * refused rather than overwritten in the first case, so a + * syntactically broken engine config does not take the operator's + * other engine settings with it. * - 200 + the masked response on success. - * - Always broadcasts `providers.updated` after a successful write - * so every connected SSE client refreshes its catalogue. + * - Always broadcasts `providers.updated` after a successful write so + * every connected SSE client refreshes its catalogue. * * The body size is bounded by `lib/read-json.js` (the shared body * reader); a too-large payload is answered by the Hono capture with - * 413 — same answer every other route returns. + * 413 — the same answer every other route returns. */ export async function handlePutProviders(req, res, _ctx) { const parsed = await readJson(req); @@ -129,115 +196,100 @@ export async function handlePutProviders(req, res, _ctx) { JSON.stringify({ ok: false, code: "BAD_BODY", error: "body must be a JSON object" }), ); } - // Keep-existing-key convention (ticket 03 cross-branch API note): -// `auth.apiKey` empty OR absent on an incoming provider means "don't -// change the existing key". We copy the user-level file's apiKey -// onto those records before validation, so the masked placeholder -// the UI sends back (and an absent-field body) does not silently -// wipe the plaintext on every edit. See -// lib/providers-config.js#applyKeepKeyConvention. -// -// Layer scope: the "previous key" lookup reads the user-level file -// ONLY (`loadUserLevelProviders`), not the merged catalogue. Without -// this scoping, editing a provider whose key is sourced from the env -// or cwd layer would materialise the deployment secret into the -// user-level file — once written there, the deployment layer can no -// longer rotate it. The merged view still wins for the engine -// (`loadProvidersConfig` priority order), so the visible behaviour -// for the operator is unchanged: an env-defined key still wins at -// read time even after the user edits the provider. -// -// Only applied when `parsed.providers` is actually an array — a missing -// or non-array providers list is an error the original validation -// surfaces as BAD_BODY, and we must not change that behaviour. -const incomingProviders = Array.isArray(parsed.providers) ? parsed.providers : null; -const existingUserLevel = loadUserLevelProviders(); -const toWrite = - incomingProviders === null - ? parsed - : { - ...parsed, - providers: applyKeepKeyConvention(existingUserLevel, incomingProviders), - }; -const result = writeProvidersConfig(toWrite); + // HARD gate. The catalogue the operator is about to see is read by + // the engine, so a provider that cannot manage providers cannot + // truthfully answer 200 here — see the module header in + // `engine/provider-writes.js`. Placed AFTER the body check on + // purpose: a malformed body is the caller's mistake and is a 400 + // whichever engine is registered, and B9 established that ordering + // for the write family. + assertProviderWriteCapability("PUT /api/providers", activeTransport()); + // The keep-key convention (ticket 03): an empty or absent + // `auth.apiKey` means "do not change the existing key", and the + // previous value is carried over before validation. The lookup is + // scoped to the STORE's own records — not the merged catalogue — so + // editing a provider whose key comes from the env or cwd layer does + // not materialise a deployment secret into the operator's file. The + // merged view still wins at read time, so nothing changes for the + // operator. + // Scoped to the STORE's own records, never the merged catalogue. + // This is not a stylistic choice: the merged view carries the env and + // cwd layers, whose keys are deployment-owned, and the convention + // would then copy one of them into the operator-owned store where + // the env layer can no longer rotate it. An existing test pinned the + // pre-B11 scoping and went red the moment this line reached for the + // merged view. + const store = readProviderStore(); + const existing = store.ok ? store.records : loadUserLevelProviders(); + const toWrite = Array.isArray(parsed.providers) + ? { ...parsed, providers: planProviderCatalogueWrite(parsed.providers, existing) } + : parsed; + // Validation runs on the CONVENTION-APPLIED body, never on the raw + // one — the pre-B11 order, and the reason it matters: the + // convention can only replace an empty or absent key with a stored + // one, so validating first would reject a body whose key is about to + // become valid. A `providers` field that is not an array reaches + // `normaliseConfig` untouched and is refused there, exactly as + // before. + const finalPlan = planCatalogueFromBody(toWrite); + if (!finalPlan.ok) return writePlanFailure(finalPlan, res); + const result = await commitProviderCatalogueWrite({ records: finalPlan.records }); if (!result.ok) { - const status = result.code === "WRITE_FAILED" ? 500 : 400; - res.writeHead(status, { "Content-Type": "application/json; charset=utf-8" }); - return res.end( - JSON.stringify({ ok: false, code: result.code, error: result.error }), - ); + // 500 with the store's own code. This is the atomicity death line + // made visible: a refused write means the previous document is + // still the whole truth, so the client's next GET returns the + // catalogue it already had, not a mixture. + return writePlanFailure({ code: "WRITE_FAILED", error: result.error }, res); } - // ticket 05: project the same providers into the engine's - // `custom_provider` tree so the engine's `model` config option - // (packages/tui/src/acp/control-state.ts) advertises them and - // `applyRecordedModel` can resolve them. We run the sync AFTER - // the user-level file is durable so a sync failure cannot leave the - // engine advertising something the user-level file does not have. - // Surface the error in the response (acceptance criterion 1) but - // keep the response status 200 — the user-level write succeeded, - // the dialog refresh reflects the new catalogue, and the operator - // can retry the sync on the next PUT. The `engineSync` field lets - // the UI surface a non-blocking warning. - const engineSync = await syncProvidersToEngine(result.providers); - if (engineSync.ok) { - // Tear down the singleton subprocess so the next operation - // spawns a fresh one that reads the new config.yaml. Brand-new - // prompt subprocesses spawned by `runMcodeAcp` already pick up - // the latest config; this is only about the singleton used for - // session/list, commands probe, and account status. - shutdownMcodeAcpSingleton(); - } - // Reload + broadcast. `loadProvidersConfig()` re-reads the file on - // every call (no in-process cache), so a follow-up GET already - // sees the change. The SSE push is the mechanism the UI uses to - // notice WITHOUT polling. - pushProvidersUpdated(); + // Tear down the singleton subprocess so the next catalogue operation + // spawns a fresh one that reads the new config. Brand-new prompt + // subprocesses spawned by `runMcodeAcp` already pick up the latest + // config; this is only about the singleton used for session/list, + // the commands probe and account status. (A no-op under the runtime + // transport — see KNOWN DEBT 1 in `engine/provider-writes.js`.) + shutdownMcodeAcpSingleton(); + // Reload + broadcast. The store is re-read on every call (there is no + // in-process cache), so a follow-up GET already sees the change. The + // SSE push is the mechanism the UI uses to notice WITHOUT polling. + await pushProvidersUpdated(); // The state-bus push keeps the existing snapshot contract intact - // (UI's general "refresh from /api/state" hint) — model selectors - // also re-fetch /api/models because the broadcast carries the - // masked providers in `event: providers.updated`. + // (the UI's general "refresh from /api/state" hint) — model + // selectors also re-fetch /api/models because the broadcast carries + // the masked providers in `event: providers.updated`. pushStateFor("__broadcast__"); res.writeHead(200, { "Content-Type": "application/json; charset=utf-8" }); return res.end( JSON.stringify({ ok: true, - providers: result.providers.map(publicView), - path: result.path, - ...(engineSync.ok - ? { - engineSync: { - ok: true, - written: engineSync.written, - keys: engineSync.keys, - }, - } - : { - engineSync: { - ok: false, - code: engineSync.code, - error: engineSync.error, - }, - warning: `engine config sync failed: ${engineSync.error}`, - }), + providers: result.records.map(publicView), + path: getEngineConfigPath(), + engineSync: { ok: true, written: result.written, keys: result.keys }, }), ); } /** - * POST /api/providers/test — per-protocol minimal connectivity - * probe. + * POST /api/providers/test — per-protocol minimal connectivity probe. * * Body shape: * { protocol: "openai|anthropic|gemini", auth: { type, apiKey, baseURL } } * - * Order of checks: - * 1. protocol whitelist (no network for unknown protocols). - * 2. local key format (no network for malformed keys). + * Order of checks, and both orders are contracts: + * 1. protocol whitelist (no network for an unknown protocol); + * 2. local key format (no network for a malformed key); * 3. fetch with the configured baseURL (or the protocol default). * * `baseURL` in the request body is honoured so a UI "test this - * endpoint" button can exercise a custom URL without going through - * the persisted config. + * endpoint" button can exercise a custom URL without going through the + * persisted config, and `auth.headers` travel WITH the probe, because a + * probe that omitted them would answer a question about a request the + * provider will never receive. + * + * Unchanged by this batch: the probe does not touch the store, does not + * need a host, and does not need the gate to be armed — the gate is + * called for its report, and the endpoint answers either way. KNOWN + * DEBT 1 in `engine/provider-reads.js` costs the branch that would let + * the engine answer it instead. */ export async function handleTestProvider(req, res, _ctx) { const parsed = await readJson(req); @@ -247,27 +299,24 @@ export async function handleTestProvider(req, res, _ctx) { JSON.stringify({ ok: false, code: "BAD_BODY", error: "body must be a JSON object" }), ); } + checkProviderReadCapability("POST /api/providers/test", activeTransport()); const protocol = typeof parsed.protocol === "string" ? parsed.protocol : ""; const authRaw = parsed.auth && typeof parsed.auth === "object" ? parsed.auth : {}; - // The request body's `baseURL` (when provided) is the probe - // target; persisted auth.baseURL is the fallback. Tests pass a - // fake URL to confirm structured errors without a real network - // call. + // The request body's `baseURL` (when provided) is the probe target; + // persisted auth.baseURL is the fallback. Tests pass a fake URL to + // confirm structured errors without a real network call. const auth = { type: typeof authRaw.type === "string" ? authRaw.type : "byok", apiKey: typeof authRaw.apiKey === "string" ? authRaw.apiKey : "", baseURL: typeof authRaw.baseURL === "string" ? authRaw.baseURL : "", - // Custom headers (webui-parity ticket 85). Forwarded verbatim; - // `probe()` re-validates them through `normalizeCustomHeaders` - // because THIS route rebuilds `auth` by hand and therefore never - // passes through the PUT normaliser. Without this line the dialog - // sends them, the route drops them, and the probe silently answers - // a question about a request the provider will never receive. + // Custom headers (webui-parity ticket 85) are forwarded verbatim + // and re-validated by the prober, because THIS route rebuilds + // `auth` by hand and never passes through the PUT normaliser. headers: authRaw.headers, }; - // Optional timeout override (ms) — surfaces from the request - // body so a UI "quick test" can fire a short probe. Unspecified - // defaults to the lib's 8s. + // Optional timeout override (ms) — surfaces from the request body so + // a UI "quick test" can fire a short probe. Unspecified defaults to + // the lib's 8s. const timeoutMs = typeof parsed.timeoutMs === "number" && parsed.timeoutMs > 0 ? Math.min(parsed.timeoutMs, 8000) @@ -294,13 +343,13 @@ export async function handleTestProvider(req, res, _ctx) { // --------------------------------------------------------------------- // SSE broadcast — the named event every connected client receives // after a PUT (so the UI can refresh the catalogue without polling). -// The frame carries the masked providers payload; apiKey NEVER -// appears in cleartext (publicView is the only serialiser on this -// path, by design). +// The frame carries the masked providers payload; apiKey NEVER appears +// in cleartext (publicView is the only serialiser on this path, by +// design). // --------------------------------------------------------------------- -function pushProvidersUpdated() { - const cfg = loadProvidersConfig(); +async function pushProvidersUpdated() { + const cfg = await readEngineProviderCatalogue(); const frame = `event: providers.updated\ndata: ${JSON.stringify({ version: cfg.version, providers: cfg.providers.map(publicView), @@ -314,11 +363,13 @@ function pushProvidersUpdated() { /** * Test-only helper: returns the SSE frame that would be emitted on - * PUT, without writing to any client. Used by tests that want to - * assert the masked shape directly. + * PUT, without writing to any client. Used by tests that want to assert + * the masked shape directly. + * + * @returns {Promise} */ -export function _peekProvidersUpdatedFrame() { - const cfg = loadProvidersConfig(); +export async function _peekProvidersUpdatedFrame() { + const cfg = await readEngineProviderCatalogue(); return `event: providers.updated\ndata: ${JSON.stringify({ version: cfg.version, providers: cfg.providers.map(publicView), @@ -326,8 +377,11 @@ export function _peekProvidersUpdatedFrame() { } /** - * Test-only helper: returns the raw response stream shape used by - * the test endpoint when it builds a fake request body. + * Test-only helper: returns the raw response stream shape used by the + * test endpoint when it builds a fake request body. + * + * @param {unknown} body + * @returns {Readable} */ export function _bodyReadable(body) { return Readable.from([Buffer.from(JSON.stringify(body), "utf8")]); @@ -339,26 +393,27 @@ export function _bodyReadable(body) { // GET /api/providers/presets — preset gallery. // POST /api/providers/preset/:id/enable — one-click materialise. // -// The GET response carries each preset's `enabled` flag — true when -// a provider with the same id is already in the configured -// catalogue. The UI uses that flag to render "Enabled" / "Enable" -// buttons without a second round-trip. +// The GET response carries each preset's `enabled` flag — true when a +// provider with the same id is already in the configured catalogue, so +// the UI renders "Enabled" / "Enable" without a second round trip. // // The POST enable handler: // 1. resolves the template by id (400 if unknown); -// 2. re-reads the current user-level catalogue; -// 3. if a provider with the same id is already configured, returns -// 409 with the existing record (idempotent semantics — calling -// enable twice is a no-op + informational response); -// 4. otherwise prepends (or appends) the materialised template to -// the existing user-level catalogue and writes the file via -// `writeProvidersConfig` (which runs the same validation -// gate as a manual PUT); -// 5. triggers the same `providers.updated` SSE broadcast as a PUT, -// so every connected client refreshes its catalogue. +// 2. re-reads the current catalogue; +// 3. if a provider with the same id is already configured, answers +// 200 with the existing record (idempotent — enabling twice is a +// no-op plus an informational field); +// 4. otherwise prepends the materialised template and commits through +// the SAME write path as #63, so the persisted store passes the +// same validation gate and the same atomic rename; +// 5. broadcasts `providers.updated`, so every connected client +// refreshes its catalogue. // -// `apiKey` is deliberately left empty on materialisation — the -// user must supply it after the template is enabled. +// `apiKey` is deliberately left empty on materialisation — the operator +// must supply it after the template is enabled. That record is stored +// with no engine projection (an entry with no key has nothing to call), +// and it comes back on the next read because the store keeps the webui +// record beside the engine fields. // ===================================================================== /** @@ -372,8 +427,9 @@ export function _bodyReadable(body) { * enabledIds: [ "zhipu", "claude-code", ... ] * } */ -export function handleGetPresets(_req, res, _ctx) { - const cfg = loadProvidersConfig(); +export async function handleGetPresets(_req, res, _ctx) { + checkProviderReadCapability("GET /api/providers/presets", activeTransport()); + const cfg = await readEngineProviderCatalogue(); const configuredIds = new Set(cfg.providers.map((p) => p.id)); const presets = PROVIDER_PRESETS.map((p) => ({ ...publicPresetView(p), @@ -385,9 +441,7 @@ export function handleGetPresets(_req, res, _ctx) { ok: true, version: cfg.version, presets, - enabledIds: [...configuredIds].filter((id) => - PROVIDER_PRESETS.some((p) => p.id === id), - ), + enabledIds: [...configuredIds].filter((id) => PROVIDER_PRESETS.some((p) => p.id === id)), }), ); } @@ -398,22 +452,20 @@ export function handleGetPresets(_req, res, _ctx) { * Behaviour: * - 400 when `id` does not name a known preset. * - 200 (idempotent) when the preset is already configured; the - * response carries the existing (masked) provider record so - * the UI can re-show it. - * - 200 when the template was newly enabled; the response - * carries the materialised (masked) provider record. + * response carries the existing (masked) provider record so the UI + * can re-show it without a second GET. + * - 200 when the template was newly enabled; the response carries the + * materialised (masked) provider record. * - * Either way, a `providers.updated` SSE event is broadcast so - * every connected client refreshes its catalogue. The handler - * uses `writeProvidersConfig` (the same path as PUT) so the - * persisted file passes the same v2 validation gate and the - * layered-resolution hot reload applies on the next - * /api/providers GET. + * Either way a `providers.updated` SSE event is broadcast so every + * connected client refreshes its catalogue. The commit goes through + * `commitProviderCatalogueWrite`, the same path as #63, so the store + * passes the same validation gate and the same atomic rename and the + * layered resolution applies on the next read. */ export async function handleEnablePreset(req, res, _ctx, params = {}) { const id = - (params && typeof params.id === "string" && params.id) || - extractIdFromUrl(req.url); + (params && typeof params.id === "string" && params.id) || extractIdFromUrl(req.url); const tpl = getPresetById(id); if (!tpl) { res.writeHead(400, { "Content-Type": "application/json; charset=utf-8" }); @@ -425,16 +477,15 @@ export async function handleEnablePreset(req, res, _ctx, params = {}) { }), ); } - - // Read the current user-level file. `writeProvidersConfig` - // writes the WHOLE catalogue (it owns the file), so we have - // to merge with whatever is already there before calling it. - const cfg = loadProvidersConfig(); + assertProviderWriteCapability("POST /api/providers/preset/:id/enable", activeTransport()); + // Read the current catalogue. The write owns the WHOLE store, so we + // have to merge with whatever is already there before committing. + const cfg = await readEngineProviderCatalogue(); const existing = cfg.providers.find((p) => p.id === tpl.id); if (existing) { // Idempotent: the preset is already configured. Surface the - // existing masked record so the caller can re-render it - // without a second GET. + // existing masked record so the caller can re-render it without a + // second GET. pushStateFor("__broadcast__"); res.writeHead(200, { "Content-Type": "application/json; charset=utf-8" }); return res.end( @@ -446,91 +497,51 @@ export async function handleEnablePreset(req, res, _ctx, params = {}) { ); } - // New materialisation. Prepend the preset so the UI's - // "enable" action keeps the preset visible at the top of the - // provider list; the rest of the user-level catalogue is - // preserved verbatim. - const materialised = presetToMaterialised(tpl.id); - const nextProviders = [materialised, ...cfg.providers]; - // Defensive validation — `writeProvidersConfig` would catch a - // bad shape, but a structured error here makes the failure - // mode obvious in the route test. - for (const p of nextProviders) { - const r = normaliseProvider(p); - if (!r.ok) { - res.writeHead(500, { "Content-Type": "application/json; charset=utf-8" }); - return res.end( - JSON.stringify({ - ok: false, - code: "MATERIALISE_FAILED", - error: r.error, - }), - ); - } - } - - const result = writeProvidersConfig({ + // New materialisation. Prepend the preset so the UI's "enable" action + // keeps the preset visible at the top of the provider list; the rest + // of the catalogue is preserved verbatim. + const nextRecords = planCatalogueFromBody({ version: 2, - providers: nextProviders, + providers: [presetToMaterialised(tpl.id), ...cfg.providers], }); - if (!result.ok) { - const status = result.code === "WRITE_FAILED" ? 500 : 400; - res.writeHead(status, { "Content-Type": "application/json; charset=utf-8" }); + if (!nextRecords.ok) { + res.writeHead(500, { "Content-Type": "application/json; charset=utf-8" }); return res.end( - JSON.stringify({ ok: false, code: result.code, error: result.error }), + JSON.stringify({ ok: false, code: "MATERIALISE_FAILED", error: nextRecords.error }), ); } - // ticket 05: project to the engine's custom_provider tree as - // well. The preset itself lands without an apiKey (the user must - // supply one), so the sync sees an "enabled without key" record - // and correctly skips it — but the same shape runs through the - // PUT path's logic when the user later supplies a key and saves - // again. We still call the sync so a non-preset byok provider the - // user already has flows through with no behaviour change. - const engineSync = await syncProvidersToEngine(result.providers); - if (engineSync.ok) { - shutdownMcodeAcpSingleton(); + const result = await commitProviderCatalogueWrite({ records: nextRecords.records }); + if (!result.ok) { + return writePlanFailure({ code: "WRITE_FAILED", error: result.error }, res); } - // Broadcast — same SSE event PUT uses. The UI's model picker + shutdownMcodeAcpSingleton(); + // Broadcast — the same SSE event #63 uses. The UI's model picker // re-fetches /api/models after this, picking up the new // template-driven entries. - pushProvidersUpdated(); + await pushProvidersUpdated(); pushStateFor("__broadcast__"); - // Find the persisted record for the response body. - const persisted = result.providers.find((p) => p.id === tpl.id); + const persisted = result.records.find((p) => p.id === tpl.id); res.writeHead(200, { "Content-Type": "application/json; charset=utf-8" }); return res.end( JSON.stringify({ ok: true, alreadyEnabled: false, provider: publicView(persisted), - path: result.path, - ...(engineSync.ok - ? { - engineSync: { - ok: true, - written: engineSync.written, - keys: engineSync.keys, - }, - } - : { - engineSync: { - ok: false, - code: engineSync.code, - error: engineSync.error, - }, - warning: `engine config sync failed: ${engineSync.error}`, - }), + path: getEngineConfigPath(), + engineSync: { ok: true, written: result.written, keys: result.keys }, }), ); } /** - * Pull `:id` out of `req.url` as a fallback when the Hono layer - * didn't already pass `params`. Kept defensive: the Hono handler - * always supplies params, but legacy callers / unit tests that - * synthesise a raw `req` URL may not. + * Pull `:id` out of `req.url` as a fallback when the Hono layer didn't + * already pass `params`. Kept defensive: the Hono handler always + * supplies params, but legacy callers / unit tests that synthesise a + * raw `req` URL may not. + * + * @param {string} reqUrl + * @returns {string} */ function extractIdFromUrl(reqUrl) { if (typeof reqUrl !== "string") return ""; diff --git a/packages/webui/test/lib/acp-stderr-tail.test.js b/packages/webui/test/lib/acp-stderr-tail.test.js new file mode 100644 index 00000000..09d8ea69 --- /dev/null +++ b/packages/webui/test/lib/acp-stderr-tail.test.js @@ -0,0 +1,305 @@ +// webui/test/lib/acp-stderr-tail.test.js +// +// D2 — the engine subprocess's stderr used to reach nobody. +// +// `packages/webui/acp.mjs` forwarded stderr to the server's own stderr +// only under `this.debug`, so a production webui watched the engine die +// with `agent_name_conflict_migration_failed:lock` on the pipe and +// surfaced a single actionable-looking non-action: `mcode acp exited +// (code=1)`. Exit codes do not say which lock; the engine's line does. +// +// The fix carries a bounded tail of that stream inside the failure +// alert's `data.stderrTail`. These assertions run against REAL fake +// engine subprocesses (a stubbed `McodeAcpClient` would prove only that +// the stub's own buffer works) and against the real `runMcodeAcp`, so +// the bytes cross a genuine pipe, a genuine `spawn`, and a genuine +// `pushAlert`. +// +// What is pinned here, and why each half matters: +// * the tail survives `debug: false` — the whole point of the fix; +// * it is BOUNDED and marked when truncated, so a chatty engine cannot +// turn a 200-byte alert into a 200KB one nor hide the failure behind +// its own earlier noise; +// * a silent engine leaves the alert's `data` byte-identical to what it +// was before the field existed (absent, not `""`); +// * a clean code-0 run raises no error alert at all. + +import { test, describe, before, after, beforeEach, afterEach } from "node:test"; +import assert from "node:assert/strict"; +import { writeFileSync } from "node:fs"; +import { join, resolve } from "node:path"; +import { pathToFileURL } from "node:url"; + +import { mkTmpDir, rmTmpDir } from "../helpers/tmp.js"; + +const WEBUI_DIR = resolve(import.meta.dirname, "..", ".."); +const absWebuiPath = (rel) => pathToFileURL(resolve(WEBUI_DIR, rel)).href; + +const { McodeAcpClient } = await import(absWebuiPath("acp.mjs")); +const alerts = await import(absWebuiPath("server/lib/alerts.js")); +const mcodeAcp = await import(absWebuiPath("server/lib/mcode-acp.js")); + +// The engine failure the field report actually lost. Kept as a literal +// so a rename on the engine side shows up here as a failing test rather +// than as a silently narrowed assertion. +const ENGINE_FATAL = "agent_name_conflict_migration_failed:lock"; + +// ---------- fake engines ---------- + +// Crashes on startup: the exact shape that produced `code=1` and no +// diagnosis. The noise goes to stderr BEFORE the fatal line, so an +// unbounded implementation would have shipped the last 2KB — mostly +// noise — and dropped the line the operator needed. `process.exitCode` +// (not `process.exit()`) lets the stderr pipe flush before the process +// ends; an explicit exit() truncates piped writes and would make this +// test flaky for the wrong reason. +const CRASH_ENGINE = ` +for (let i = 1; i <= 200; i++) { + process.stderr.write("migrating agent name registry, step " + i + " ...\\n"); +} +process.stderr.write("${ENGINE_FATAL} at ~/.minimax-code/agents.lock\\n"); +process.exitCode = 1; +`; + +// Dies the same way, but says nothing. The reverse half: an engine that +// never spoke must leave the alert's shape untouched. +const SILENT_CRASH_ENGINE = ` +process.exitCode = 1; +`; + +// Completes one full turn and exits 0. A chatty-but-healthy engine is +// the other half: its stderr is retained, yet nothing is an error, so +// `stderrTail` must not become a failure signal of its own. +const CLEAN_ENGINE = ` +const send = (m) => process.stdout.write(JSON.stringify(m) + "\\n"); +process.stderr.write("[mcode] resuming 3 sessions\\n"); +let buf = ""; +process.stdin.setEncoding("utf8"); +process.stdin.on("data", (chunk) => { + buf += chunk; + let nl; + while ((nl = buf.indexOf("\\n")) !== -1) { + const line = buf.slice(0, nl).trim(); + buf = buf.slice(nl + 1); + if (!line) continue; + const msg = JSON.parse(line); + if (msg.method === "initialize") { + send({ jsonrpc: "2.0", id: msg.id, result: { protocolVersion: 1, agentCapabilities: {}, configOptions: [] } }); + } else if (msg.method === "session/new") { + send({ jsonrpc: "2.0", id: msg.id, result: { sessionId: "sess-clean", configOptions: [] } }); + } else if (msg.method === "session/prompt") { + send({ jsonrpc: "2.0", id: msg.id, result: { stopReason: "end_turn" } }); + setTimeout(() => { process.exitCode = 0; process.stdin.pause(); }, 20); + } else if (msg.method && msg.id !== undefined) { + send({ jsonrpc: "2.0", id: msg.id, result: {} }); + } + } +}); +`; + +let dir = null; +const engines = {}; + +before(() => { + dir = mkTmpDir("webui-acp-stderr-"); + for (const [name, source] of Object.entries({ + crash: CRASH_ENGINE, + silent: SILENT_CRASH_ENGINE, + clean: CLEAN_ENGINE, + })) { + engines[name] = join(dir, `${name}-engine.mjs`); + writeFileSync(engines[name], source); + } +}); + +after(async () => { + // Importing lib/mcode-acp.js pulls in lib/acp-client.js, which starts a + // resident engine singleton on module load. Left running it keeps a + // spawned process — and this test file's event loop — alive forever. + const acpClient = await import(absWebuiPath("server/lib/acp-client.js")); + try { + acpClient.shutdownMcodeAcpSingleton(); + } catch { + /* never started, or already gone */ + } + await new Promise((r) => setTimeout(r, 50)); + delete process.env.MCODE_CMD; + if (dir) rmTmpDir(dir); +}); + +beforeEach(() => { + alerts._resetForTests(); +}); + +const running = []; + +afterEach(() => { + while (running.length) running.pop().stop(); + delete process.env.MCODE_CMD; + alerts._resetForTests(); +}); + +/** The `cs` runMcodeAcp reads before the engine is even reached. */ +function makeCs() { + return { + model: { name: "minimax_api/MiniMax-M3" }, + workspace: { dir: dir }, + sessionId: null, + mcodeSessionId: null, + sessionTitle: "Untitled", + chat: [], + usage: {}, + context: { used: 0, limit: 0, percent: 0, tokens: 0 }, + running: { active: false }, + }; +} + +/** The failure alerts runMcodeAcp raised, newest last. */ +function errorAlerts() { + return alerts.getRecentAlerts().filter((a) => a.src === "mcode-acp" && a.level === "error"); +} + +describe("engine stderr reaches the failure alert (D2)", () => { + test("a crash alert carries the truncated stderr tail, with debug off", async () => { + process.env.MCODE_CMD = engines.crash; + + const r = await mcodeAcp.runMcodeAcp("hi", { + label: "test", + cs: makeCs(), + cid: "cid-stderr-crash", + sessionId: null, + }); + + assert.equal(r.status, "failed", "the engine crashed, so the turn fails"); + + const raised = errorAlerts(); + assert.equal(raised.length, 1, `expected exactly one engine error alert, got ${JSON.stringify(raised)}`); + const [alert] = raised; + + // The lost diagnostic is back, and it is the reason the turn failed. + assert.match(alert.data.stderrTail, new RegExp(ENGINE_FATAL)); + assert.match(alert.msg, /mcode acp exited \(code=1/); + }); + + test("the tail is bounded and marked when truncated", async () => { + process.env.MCODE_CMD = engines.crash; + + await mcodeAcp.runMcodeAcp("hi", { + label: "test", + cs: makeCs(), + cid: "cid-stderr-bounded", + sessionId: null, + }); + + const { stderrTail } = errorAlerts()[0].data; + + // 200 lines of ~45 bytes each: an unbounded tail would carry the + // whole 9KB, and a byte-unbounded alert is a log-flooding vector. + assert.ok( + stderrTail.length < 2048 + 200, + `the tail must stay near its 2KB bound, got ${stderrTail.length} chars`, + ); + assert.match(stderrTail, /^\[acp stderr truncated, showing the tail\]/); + // Truncation is stated, not silent — an operator must be able to + // tell "that was all" from "that was the end of what we kept". + assert.doesNotMatch( + stderrTail, + /migrating agent name registry, step 1 /, + "the earliest noise must have been dropped, not carried", + ); + }); + + test("the alert's own shape is unchanged — stderrTail is additive", async () => { + process.env.MCODE_CMD = engines.crash; + + await mcodeAcp.runMcodeAcp("hi", { + label: "test", + cs: makeCs(), + cid: "cid-stderr-shape", + sessionId: null, + }); + + const [alert] = errorAlerts(); + // Every pre-existing field, untouched. Consumers switching on the + // alert contract (SSE /api/alerts, the audit event) must not have + // to learn a new required field. + assert.equal(alert.level, "error"); + assert.equal(alert.src, "mcode-acp"); + assert.equal(alert.cid, "cid-stderr-shape"); + assert.equal(alert.sessionId, null); + assert.equal(alert.data.phase, "start-or-load"); + assert.equal(alert.count, 1); + }); + + test("a silent engine leaves the alert data exactly as it was", async () => { + process.env.MCODE_CMD = engines.silent; + + await mcodeAcp.runMcodeAcp("hi", { + label: "test", + cs: makeCs(), + cid: "cid-stderr-silent", + sessionId: null, + }); + + const raised = errorAlerts(); + assert.equal(raised.length, 1); + // Absent, not `""`: an operator (and a deduped alert diff) should + // not be able to tell this alert from one raised before the fix. + assert.deepEqual(raised[0].data, { phase: "start-or-load" }); + }); + + test("a clean code-0 run raises no failure alert even with stderr output", async () => { + process.env.MCODE_CMD = engines.clean; + + const r = await mcodeAcp.runMcodeAcp("hi", { + label: "test", + cs: makeCs(), + cid: "cid-stderr-clean", + sessionId: null, + }); + + assert.equal(r.status, "succeeded", `clean engine run failed: ${JSON.stringify(r.error)}`); + // The engine did write to stderr. Retaining it must not turn + // ordinary engine chatter into a failure signal. + assert.deepEqual(errorAlerts(), []); + }); +}); + +describe("McodeAcpClient stderr tail", () => { + test("the tail is readable after a crash, and empty before anything is written", async () => { + process.env.MCODE_CMD = engines.crash; + const client = new McodeAcpClient({ debug: false }); + running.push(client); + + assert.equal(client.stderrTail, "", "nothing on the wire yet, so nothing to report"); + + const exited = new Promise((res) => client.once("exit", res)); + await assert.rejects(() => client.start()); + await exited; + + assert.match(client.stderrTail, new RegExp(ENGINE_FATAL)); + assert.match(client.stderrTail, /truncated/); + }); + + test("start() resets the tail so one process cannot be blamed for another's crash", async () => { + process.env.MCODE_CMD = engines.crash; + const client = new McodeAcpClient({ debug: false }); + running.push(client); + + const firstExit = new Promise((res) => client.once("exit", res)); + await assert.rejects(() => client.start()); + await firstExit; + assert.match(client.stderrTail, new RegExp(ENGINE_FATAL)); + + // A restart begins a NEW subprocess, whose stderr starts empty. The + // dead process's crash text must not survive into the next run and + // re-appear on some unrelated failure later. + process.env.MCODE_CMD = engines.clean; + const restarted = client.start(); + // The reset happens before the spawn, so it is observable the moment + // `start()` is called. Whether THIS run goes on to succeed or fail + // is beside the point: the dead process's crash text is already gone. + assert.doesNotMatch(client.stderrTail, new RegExp(ENGINE_FATAL)); + restarted.catch(() => {}); + }); +}); diff --git a/packages/webui/test/lib/engine-provider-sync.test.js b/packages/webui/test/lib/engine-provider-sync.test.js deleted file mode 100644 index 4b34d8d6..00000000 --- a/packages/webui/test/lib/engine-provider-sync.test.js +++ /dev/null @@ -1,813 +0,0 @@ -// webui/test/lib/engine-provider-sync.test.js -// Pure helpers + sync flow for `lib/engine-provider-sync.js` (ticket 05). -// -// What we pin here: -// - providerKeyFromId: reserved engine ids get a `-byok` suffix; -// everything else round-trips; non-conforming ids return "". -// - modelKeyFromId: same shape as providerKeyFromId, no reserved handling. -// - toEngineCustomProvider: every field of the v2 schema is mapped; the -// four ineligible shapes (coding-plan, disabled, empty apiKey, -// missing baseURL, unknown protocol) yield null; the engine api -// format is picked by protocol. -// - syncProvidersToEngine: writes an atomic YAML that preserves the -// operator's other sections (provider.*, defaultModel, …), only -// `custom_provider` is owned by the helper; an empty eligible list is -// a no-op (operator's manual entries are kept); engine-config read -// failures are surfaced as a structured error. -// - syncProvidersFromPutBody: applies the keep-key convention before -// the sync (same path the routes use for the user-level write), so -// the engine sees the resolved apiKey. - -import { test, describe, after, beforeEach } from "node:test"; -import assert from "node:assert/strict"; -import {rmSync, existsSync, readFileSync} from "node:fs"; - -import { join } from "node:path"; -import yaml from "js-yaml"; -import { mkTmpDir } from "../helpers/tmp.js"; - -const absPath = (rel) => - import.meta.resolve - ? import.meta.resolve(rel) - : new URL(rel, import.meta.url).href; - -const { - resolveEngineDataDir, - providerKeyFromId, - modelKeyFromId, - toEngineCustomProvider, - syncProvidersToEngine, - syncProvidersFromPutBody, -} = await import( - new URL("../../server/lib/engine-provider-sync.js", import.meta.url).href -); - -// temp data dir scoped to this test file so the helper's -// `resolveEngineDataDir()` (env-driven) does not point at the host's -// real engine config. Set at module load time — the helper reads -// `process.env` at call time, but Node's test runner may run setup -// before `before()` fires, and a stable env at import time is the -// safest contract. -const _origMinimax = process.env.MINIMAX_DATA_DIR; -const _origMavis = process.env.MAVIS_DATA_DIR; -const _tmpDataDir = mkTmpDir("minimax-code-engine-sync-"); -process.env.MINIMAX_DATA_DIR = _tmpDataDir; -delete process.env.MAVIS_DATA_DIR; - -after(() => { - if (_origMinimax === undefined) delete process.env.MINIMAX_DATA_DIR; - else process.env.MINIMAX_DATA_DIR = _origMinimax; - if (_origMavis === undefined) delete process.env.MAVIS_DATA_DIR; - else process.env.MAVIS_DATA_DIR = _origMavis; - rmSync(_tmpDataDir, { recursive: true, force: true }); -}); - -describe("resolveEngineDataDir — env precedence", () => { - test("MINIMAX_DATA_DIR wins", () => { - const prev = process.env.MINIMAX_DATA_DIR; - process.env.MINIMAX_DATA_DIR = "/tmp/env-wins"; - try { - assert.equal(resolveEngineDataDir(), "/tmp/env-wins"); - } finally { - // Restore so the sync tests downstream still resolve to the - // file-scoped `_tmpDataDir`. - if (prev === undefined) delete process.env.MINIMAX_DATA_DIR; - else process.env.MINIMAX_DATA_DIR = prev; - } - }); - test("falls back to ~/.minimax when neither env is set", () => { - // Restore the per-test env to the no-env state only for this test; - // every other test in the file relies on the `before` hook's - // `_tmpDataDir` so we MUST put it back before returning, or the - // sync tests downstream would resolve to the host's real config. - const prev = process.env.MINIMAX_DATA_DIR; - const prevMavis = process.env.MAVIS_DATA_DIR; - delete process.env.MINIMAX_DATA_DIR; - delete process.env.MAVIS_DATA_DIR; - try { - assert.match(resolveEngineDataDir(), /[/\\]\.minimax$/); - } finally { - if (prev === undefined) delete process.env.MINIMAX_DATA_DIR; - else process.env.MINIMAX_DATA_DIR = prev; - if (prevMavis === undefined) delete process.env.MAVIS_DATA_DIR; - else process.env.MAVIS_DATA_DIR = prevMavis; - } - }); -}); - -describe("providerKeyFromId", () => { - test("round-trips a normal id", () => { - assert.equal(providerKeyFromId("byok-zhipu"), "byok-zhipu"); - assert.equal(providerKeyFromId("kimi"), "kimi"); - }); - test("disambiguates engine-internal reserved ids with -byok suffix", () => { - // The webui id validator allows these names (the engine just would - // not — operators might already have one in their providers.json). - // Projection must not collide with the engine's internal providers. - assert.equal(providerKeyFromId("minimax"), "minimax-byok"); - assert.equal(providerKeyFromId("minimax_api"), "minimax_api-byok"); - assert.equal(providerKeyFromId("provider"), "provider-byok"); - assert.equal(providerKeyFromId("custom_provider"), "custom_provider-byok"); - }); - test("rejects ids outside the provider-key character class", () => { - assert.equal(providerKeyFromId(""), ""); - assert.equal(providerKeyFromId("spaces are bad"), ""); - assert.equal(providerKeyFromId("slashes/are/bad"), ""); - // Underscores / hyphens / dots in the middle are fine (matches the - // webui v2 validator's provider id contract). - assert.equal(providerKeyFromId("a_b.c-d"), "a_b.c-d"); - }); -}); - -describe("modelKeyFromId", () => { - test("round-trips a normal id", () => { - assert.equal(modelKeyFromId("glm-5.3"), "glm-5.3"); - assert.equal(modelKeyFromId("claude-sonnet-4-5"), "claude-sonnet-4-5"); - }); - test("rejects ids outside the model-key character class", () => { - assert.equal(modelKeyFromId(""), ""); - assert.equal(modelKeyFromId("with space"), ""); - }); - // Ticket 09-02: upstream catalogues commonly carry namespace-style - // model ids (`deepseek/deepseek-v4.1-flash`, - // `z-ai/glm-5.3`, `openai/gpt-5.6-sol`). The engine's wire form - // `/` uses `/` as the structural separator; - // `parseSourceQualifiedModelKey` splits on the FIRST `/`, so a - // model id that contains `/` survives the round-trip. The - // pre-fix `PROVIDER_KEY_REGEX = /^[A-Za-z0-9][A-Za-z0-9_.-]*$/` - // silently dropped these entries from the engine sync; the fix - // widens to allow `/` while still rejecting YAML-unsafe - // characters (whitespace, control tokens, anchors). - test("accepts upstream-namespace ids containing `/`", () => { - assert.equal(modelKeyFromId("deepseek/deepseek-v4.1-flash"), "deepseek/deepseek-v4.1-flash"); - assert.equal(modelKeyFromId("z-ai/glm-5.3"), "z-ai/glm-5.3"); - assert.equal(modelKeyFromId("openai/gpt-5.6-sol"), "openai/gpt-5.6-sol"); - }); - test("still rejects YAML-unsafe characters", () => { - // Whitespace, YAML list anchor, comment, and a colon (which - // the engine's custom_provider parser would interpret as a - // structural separator) all stay rejected. - assert.equal(modelKeyFromId("with:colon"), ""); - assert.equal(modelKeyFromId("with#comment"), ""); - assert.equal(modelKeyFromId("with[bracket]"), ""); - assert.equal(modelKeyFromId("with&anchor"), ""); - assert.equal(modelKeyFromId("with*asterisk"), ""); - assert.equal(modelKeyFromId("with|pipe"), ""); - assert.equal(modelKeyFromId("with>gt"), ""); - assert.equal(modelKeyFromId("-leading-dash"), ""); - }); -}); - -describe("toEngineCustomProvider — eligibility", () => { - const base = { - id: "byok-zhipu", - label: "Zhipu BYOK", - enabled: true, - protocol: "openai", - auth: { type: "byok", apiKey: "sk-fake", baseURL: "https://example.com/v1" }, - models: [{ id: "glm-5.3" }], - }; - - test("eligible byok provider maps cleanly", () => { - const out = toEngineCustomProvider(base); - assert.ok(out, "eligible"); - assert.equal(out.key, "byok-zhipu"); - assert.equal(out.entry.name, "Zhipu BYOK"); - assert.equal(out.entry.kind, "custom"); - assert.equal(out.entry.enabled, true); - assert.equal(out.entry.api, "openai-completions"); - assert.equal(out.entry.options.apiKey, "sk-fake"); - assert.equal(out.entry.options.baseURL, "https://example.com/v1"); - assert.equal(out.entry.options.authMode, "api-key"); - assert.deepEqual(out.entry.models, { "glm-5.3": {} }); - }); - - test("custom headers reach options.headers (webui-parity ticket 85)", () => { - // The load-bearing line for the whole feature: the runtime merges - // `options.headers` into every upstream request for this provider - // (local-runtime-v2 catalog/provider-views.ts:218). Without this - // mapping a header the operator typed, saved and read back would - // never be sent — the worst kind of "saved". - const p = { - ...base, - auth: { - ...base.auth, - headers: { "X-Tenant": "acme", "X-Trace": "01H" }, - }, - }; - const out = toEngineCustomProvider(p); - assert.ok(out, "still eligible"); - assert.deepEqual(out.entry.options.headers, { - "X-Tenant": "acme", - "X-Trace": "01H", - }); - }); - - test("a provider with no headers keeps the exact pre-ticket options shape", () => { - // Omission, not `headers: {}` — so an untouched provider's engine - // config does not churn on every sync. - const out = toEngineCustomProvider(base); - assert.equal( - Object.prototype.hasOwnProperty.call(out.entry.options, "headers"), - false, - "no headers key when the operator configured none", - ); - assert.deepEqual( - Object.keys(out.entry.options).sort(), - ["apiKey", "authMode", "baseURL"], - "the options key set must not grow for a provider that has none", - ); - }); - - test("the emitted headers are a copy, not the stored record", () => { - const headers = { "X-Tenant": "acme" }; - const out = toEngineCustomProvider({ ...base, auth: { ...base.auth, headers } }); - out.entry.options.headers["X-Tenant"] = "tampered"; - assert.equal(headers["X-Tenant"], "acme", "the source record must be unreachable"); - }); - - test("an EMPTY headers object is omitted, not written as `headers: {}`", () => { - // The end-to-end run caught this: `normaliseProvider` always - // materialises `auth.headers` (absent -> {}), and the projection's - // truthiness test treated `{}` as "has headers", so every provider - // that never configured one grew a `headers: {}` block in the - // engine's config.yaml. The fixture above cannot see it — its - // `auth` has no `headers` key at all — so the normalised shape is - // reproduced explicitly here. - const out = toEngineCustomProvider({ - ...base, - auth: { ...base.auth, headers: {} }, - }); - assert.ok(out, "still eligible"); - assert.equal( - Object.prototype.hasOwnProperty.call(out.entry.options, "headers"), - false, - "an empty header map must be omitted, not emitted as {}", - ); - }); - - test("a non-object headers value is ignored rather than projected", () => { - for (const bad of [["X-A"], "X-A", 42]) { - const out = toEngineCustomProvider({ - ...base, - auth: { ...base.auth, headers: bad }, - }); - assert.ok(out, "still eligible"); - assert.equal( - Object.prototype.hasOwnProperty.call(out.entry.options, "headers"), - false, - `headers=${JSON.stringify(bad)} must not be projected`, - ); - } - }); - - test("coding-plan providers are skipped (out of scope for byok projection)", () => { - const p = { ...base, auth: { type: "coding-plan", apiKey: "tk-fake", baseURL: "https://example.com" } }; - assert.equal(toEngineCustomProvider(p), null); - }); - - test("disabled providers are skipped", () => { - const p = { ...base, enabled: false }; - assert.equal(toEngineCustomProvider(p), null); - }); - - test("providers without an apiKey are skipped", () => { - const p = { ...base, auth: { type: "byok", baseURL: "https://example.com/v1" } }; - assert.equal(toEngineCustomProvider(p), null); - // Also: apiKey must be a non-empty trimmed string. - const p2 = { ...base, auth: { type: "byok", apiKey: " ", baseURL: "https://example.com/v1" } }; - assert.equal(toEngineCustomProvider(p2), null); - }); - - test("providers without a baseURL are skipped", () => { - const p = { ...base, auth: { type: "byok", apiKey: "sk-fake" } }; - assert.equal(toEngineCustomProvider(p), null); - const p2 = { ...base, auth: { type: "byok", apiKey: "sk-fake", baseURL: " " } }; - assert.equal(toEngineCustomProvider(p2), null); - }); - - test("unknown protocol → skipped", () => { - const p = { ...base, protocol: "cohere" }; - assert.equal(toEngineCustomProvider(p), null); - }); - - test("null / non-object → null", () => { - assert.equal(toEngineCustomProvider(null), null); - assert.equal(toEngineCustomProvider("string"), null); - assert.equal(toEngineCustomProvider(undefined), null); - }); - - test("id that hits an engine-reserved word gets the -byok suffix", () => { - const p = { - ...base, - id: "minimax", - label: "Custom MiniMax-shaped alias", - }; - const out = toEngineCustomProvider(p); - assert.ok(out); - assert.equal(out.key, "minimax-byok"); - }); - - test("non-conforming id (spaces) → skipped", () => { - const p = { ...base, id: "byok with space" }; - assert.equal(toEngineCustomProvider(p), null); - }); -}); - -describe("toEngineCustomProvider — protocol → engine api mapping", () => { - test("openai → openai-completions", () => { - const p = { ...base({ id: "p1", protocol: "openai" }) }; - assert.equal(toEngineCustomProvider(p).entry.api, "openai-completions"); - }); - test("anthropic → anthropic-messages", () => { - const p = { ...base({ id: "p2", protocol: "anthropic" }) }; - assert.equal(toEngineCustomProvider(p).entry.api, "anthropic-messages"); - }); - test("gemini → openai-completions (Gemini OpenAI-compat endpoint)", () => { - const p = { ...base({ id: "p3", protocol: "gemini" }) }; - assert.equal(toEngineCustomProvider(p).entry.api, "openai-completions"); - }); - test("missing protocol defaults to openai-completions", () => { - const p = { - id: "p4", - label: "P4", - enabled: true, - auth: { type: "byok", apiKey: "sk", baseURL: "https://x/v1" }, - models: [], - }; - assert.equal(toEngineCustomProvider(p).entry.api, "openai-completions"); - }); - - function base(overrides) { - return { - label: "x", - enabled: true, - protocol: "openai", - auth: { type: "byok", apiKey: "sk-fake", baseURL: "https://x/v1" }, - models: [], - ...overrides, - }; - } -}); - -describe("toEngineCustomProvider — model metadata mapping", () => { - test("contextLimit → limit.context", () => { - const p = { - id: "p1", - label: "p1", - enabled: true, - protocol: "openai", - auth: { type: "byok", apiKey: "sk", baseURL: "https://x/v1" }, - models: [{ id: "m1", contextLimit: 128000 }], - }; - assert.deepEqual(toEngineCustomProvider(p).entry.models.m1, { - limit: { context: 128000 }, - }); - }); - - test("thinkingLevels → thinking.effortOptions", () => { - const p = { - id: "p1", - label: "p1", - enabled: true, - protocol: "openai", - auth: { type: "byok", apiKey: "sk", baseURL: "https://x/v1" }, - models: [{ id: "m1", thinkingLevels: ["low", "high"] }], - }; - assert.deepEqual(toEngineCustomProvider(p).entry.models.m1, { - thinking: { effortOptions: ["low", "high"] }, - }); - }); - - test("modalities → modalities.input", () => { - const p = { - id: "p1", - label: "p1", - enabled: true, - protocol: "openai", - auth: { type: "byok", apiKey: "sk", baseURL: "https://x/v1" }, - models: [{ id: "m1", modalities: ["text", "image"] }], - }; - assert.deepEqual(toEngineCustomProvider(p).entry.models.m1, { - modalities: { input: ["text", "image"] }, - }); - }); - - test("label different from id → name; label === id → no name (engine default)", () => { - const p = { - id: "p1", - label: "p1", - enabled: true, - protocol: "openai", - auth: { type: "byok", apiKey: "sk", baseURL: "https://x/v1" }, - models: [ - { id: "m1", label: "m1" }, - { id: "m2", label: "Model Two" }, - ], - }; - const out = toEngineCustomProvider(p); - assert.equal(out.entry.models.m1.name, undefined); - assert.equal(out.entry.models.m2.name, "Model Two"); - }); - - test("non-string thinkingLevels entries are filtered out", () => { - const p = { - id: "p1", - label: "p1", - enabled: true, - protocol: "openai", - auth: { type: "byok", apiKey: "sk", baseURL: "https://x/v1" }, - models: [{ id: "m1", thinkingLevels: ["low", 42, "", "high"] }], - }; - // Empty string entries are dropped; non-strings cause the array to - // fail the every-check and the whole thinkingLevels field is omitted. - assert.equal(toEngineCustomProvider(p).entry.models.m1.thinking, undefined); - }); - - test("non-positive contextLimit is dropped", () => { - const p = { - id: "p1", - label: "p1", - enabled: true, - protocol: "openai", - auth: { type: "byok", apiKey: "sk", baseURL: "https://x/v1" }, - models: [{ id: "m1", contextLimit: 0 }], - }; - assert.equal(toEngineCustomProvider(p).entry.models.m1.limit, undefined); - }); - - test("provider with no models still gets an entry (operators can add later)", () => { - const p = { - id: "p1", - label: "p1", - enabled: true, - protocol: "openai", - auth: { type: "byok", apiKey: "sk", baseURL: "https://x/v1" }, - models: [], - }; - const out = toEngineCustomProvider(p); - assert.ok(out); - assert.equal(out.entry.models, undefined); - }); -}); - -describe("syncProvidersToEngine — atomic YAML write + operator preservation", () => { - beforeEach(() => { - // Reset the per-test engine config. - rmSync(join(_tmpDataDir, "config.yaml"), { force: true }); - }); - - test("writes custom_provider from the merged catalogue", async () => { - const providers = [ - { - id: "byok-zhipu", - label: "Zhipu", - enabled: true, - protocol: "openai", - auth: { type: "byok", apiKey: "sk-fake", baseURL: "https://example.com/v1" }, - models: [{ id: "glm-5.3" }], - }, - ]; - const r = await syncProvidersToEngine(providers); - assert.equal(r.ok, true); - assert.equal(r.written, true); - assert.deepEqual(r.keys, ["byok-zhipu"]); - assert.ok(existsSync(join(_tmpDataDir, "config.yaml")), "file must exist"); - - const written = yaml.load(readFileSync(join(_tmpDataDir, "config.yaml"), "utf8")); - assert.ok(written.custom_provider); - assert.equal(written.custom_provider["byok-zhipu"].options.apiKey, "sk-fake"); - assert.equal(written.custom_provider["byok-zhipu"].options.baseURL, "https://example.com/v1"); - assert.equal(written.custom_provider["byok-zhipu"].kind, "custom"); - // Acceptance: the entry carries the ownership marker so a future - // sync knows it is webui-managed and a foreign entry does not. - assert.equal(written.custom_provider["byok-zhipu"]._webui_owned, true); - }); - - test("preserves the operator's provider.minimax + defaultModel sections", async () => { - // Pre-populate the engine config as an operator would. - const seed = { - logLevel: "info", - defaultModel: "minimax/MiniMax-M3", - provider: { - minimax: { - options: { apiKey: "sk-existing", authMode: "api-key", baseURL: "https://x/v1" }, - }, - }, - // Foreign (operator-managed) entry — no ownership marker, so the - // sync must NOT touch it. Ticket 05 acceptance: merge-over-replace, - // not replace-everything. - custom_provider: { existing_byok: { name: "Existing", kind: "custom", enabled: true } }, - }; - const fs = await import("node:fs/promises"); - await fs.mkdir(_tmpDataDir, { recursive: true }); - await fs.writeFile(join(_tmpDataDir, "config.yaml"), yaml.dump(seed), "utf8"); - - const r = await syncProvidersToEngine([ - { - id: "byok-new", - label: "New", - enabled: true, - protocol: "anthropic", - auth: { type: "byok", apiKey: "sk-new", baseURL: "https://y/v1" }, - models: [], - }, - ]); - assert.equal(r.ok, true); - assert.deepEqual(r.keys, ["byok-new"]); - assert.deepEqual(r.preserved, ["existing_byok"]); - const after = yaml.load(readFileSync(join(_tmpDataDir, "config.yaml"), "utf8")); - // Provider tree survives — operator's manual config is not touched. - assert.equal(after.provider.minimax.options.apiKey, "sk-existing"); - assert.equal(after.defaultModel, "minimax/MiniMax-M3"); - // Foreign entry survives (no marker, untouched by webui). - assert.deepEqual(after.custom_provider["existing_byok"], { - name: "Existing", - kind: "custom", - enabled: true, - }); - // New webui entry is added with its ownership marker. - assert.equal(after.custom_provider["byok-new"].options.apiKey, "sk-new"); - assert.equal(after.custom_provider["byok-new"]._webui_owned, true); - }); - - test("empty eligible list keeps a foreign entry intact (no destructive wipe)", async () => { - const fs = await import("node:fs/promises"); - await fs.mkdir(_tmpDataDir, { recursive: true }); - const seed = { - defaultModel: "minimax/MiniMax-M3", - custom_provider: { - // Foreign (operator-managed) — pre-existing, no marker. - manual_only: { - name: "Manual", - kind: "custom", - enabled: true, - api: "openai-completions", - options: { apiKey: "sk-manual", baseURL: "https://manual.example/v1", authMode: "api-key" }, - }, - }, - }; - await fs.writeFile(join(_tmpDataDir, "config.yaml"), yaml.dump(seed), "utf8"); - - // Empty eligible (every provider is ineligible) — must NOT wipe the - // foreign entry. Ticket 05 acceptance: the destruction class - // closed here is "removing a webui provider silently drops a foreign - // entry". The same destruction class applies to "PUTting an - // ineligible-only catalogue silently drops a foreign entry". - const r = await syncProvidersToEngine([ - { id: "x", label: "x", enabled: true, protocol: "openai", auth: { type: "coding-plan" }, models: [] }, - { id: "y", label: "y", enabled: true, protocol: "openai", auth: { type: "byok", baseURL: "https://z" }, models: [] }, - ]); - assert.equal(r.ok, true); - // No eligible providers AND the merged tree is byte-identical to - // what's on disk (foreign was already there and is preserved - // verbatim). The helper short-circuits the write — no mtime churn, - // no needless chmod. The route can still surface the `preserved` - // list to the operator via the response. - assert.equal(r.written, false); - assert.deepEqual(r.keys, []); - assert.deepEqual(r.preserved, ["manual_only"]); - - const after = yaml.load(readFileSync(join(_tmpDataDir, "config.yaml"), "utf8")); - assert.deepEqual(after.custom_provider.manual_only, seed.custom_provider.manual_only); - // The defaultModel is not touched either. - assert.equal(after.defaultModel, "minimax/MiniMax-M3"); - }); - - test("webui-managed entry whose provider is removed is dropped, foreign entry is kept", async () => { - const fs = await import("node:fs/promises"); - await fs.mkdir(_tmpDataDir, { recursive: true }); - // Pre-populate: a webui-managed entry from a previous sync AND a - // foreign entry. - const seed = { - custom_provider: { - byok_old: { - name: "Old webui", - kind: "custom", - enabled: true, - api: "openai-completions", - options: { apiKey: "sk-old", baseURL: "https://old.example/v1", authMode: "api-key" }, - _webui_owned: true, - }, - manual_only: { - name: "Manual", - kind: "custom", - enabled: true, - api: "openai-completions", - options: { apiKey: "sk-manual", baseURL: "https://manual.example/v1", authMode: "api-key" }, - }, - }, - }; - await fs.writeFile(join(_tmpDataDir, "config.yaml"), yaml.dump(seed), "utf8"); - - // Sync with no eligible providers — byok_old should be dropped - // (webui owned it, webui no longer claims it), manual_only survives. - const r = await syncProvidersToEngine([]); - assert.equal(r.ok, true); - assert.deepEqual(r.keys, []); - assert.deepEqual(r.preserved, ["manual_only"]); - - const after = yaml.load(readFileSync(join(_tmpDataDir, "config.yaml"), "utf8")); - assert.equal(after.custom_provider["byok_old"], undefined); - assert.deepEqual(after.custom_provider.manual_only, seed.custom_provider.manual_only); - }); - - test("webui-managed entry update replaces the entry's data, keeps the marker", async () => { - const fs = await import("node:fs/promises"); - await fs.mkdir(_tmpDataDir, { recursive: true }); - const seed = { - custom_provider: { - "byok-zhipu": { - name: "Old label", - kind: "custom", - enabled: true, - api: "openai-completions", - options: { apiKey: "sk-old", baseURL: "https://old.example/v1", authMode: "api-key" }, - _webui_owned: true, - }, - }, - }; - await fs.writeFile(join(_tmpDataDir, "config.yaml"), yaml.dump(seed), "utf8"); - - // Re-sync with new apiKey/baseURL for the same id — entry is replaced - // in place, marker is preserved. - const r = await syncProvidersToEngine([ - { - id: "byok-zhipu", - label: "New label", - enabled: true, - protocol: "openai", - auth: { type: "byok", apiKey: "sk-new", baseURL: "https://new.example/v1" }, - models: [], - }, - ]); - assert.equal(r.ok, true); - assert.deepEqual(r.keys, ["byok-zhipu"]); - - const after = yaml.load(readFileSync(join(_tmpDataDir, "config.yaml"), "utf8")); - assert.equal(after.custom_provider["byok-zhipu"].options.apiKey, "sk-new"); - assert.equal(after.custom_provider["byok-zhipu"].options.baseURL, "https://new.example/v1"); - assert.equal(after.custom_provider["byok-zhipu"].name, "New label"); - assert.equal(after.custom_provider["byok-zhipu"]._webui_owned, true); - }); - - test("writes custom_provider when at least one eligible provider exists; otherwise no-op write when nothing changes", async () => { - // 1) Empty eligible, no foreign — there's nothing to write, and - // the engine already treats "no custom_provider key" as "no - // custom providers". The helper reports written: false (no-op). - const r1 = await syncProvidersToEngine([]); - assert.equal(r1.ok, true); - assert.equal(r1.written, false); - assert.deepEqual(r1.keys, []); - assert.deepEqual(r1.preserved, []); - - // 2) Re-run with the same empty eligible list — byte-identical to - // what's on disk; the helper reports `written: false` and - // skips the rewrite (no mtime churn, no needless chmod). - const r2 = await syncProvidersToEngine([]); - assert.equal(r2.ok, true); - assert.equal(r2.written, false); - }); - - test("missing engine config file → creates one", async () => { - rmSync(join(_tmpDataDir, "config.yaml"), { force: true }); - const r = await syncProvidersToEngine([ - { - id: "byok-zhipu", - label: "z", - enabled: true, - protocol: "openai", - auth: { type: "byok", apiKey: "sk", baseURL: "https://x/v1" }, - models: [], - }, - ]); - assert.equal(r.ok, true); - assert.equal(existsSync(join(_tmpDataDir, "config.yaml")), true); - }); - - test("atomic write: no half-written file on success", async () => { - rmSync(join(_tmpDataDir, "config.yaml"), { force: true }); - await syncProvidersToEngine([ - { - id: "byok-zhipu", - label: "z", - enabled: true, - protocol: "openai", - auth: { type: "byok", apiKey: "sk", baseURL: "https://x/v1" }, - models: [], - }, - ]); - // No `.config-tmp-` leftover should exist next to config.yaml. - const fs = await import("node:fs"); - const siblings = fs.readdirSync(_tmpDataDir); - const tmps = siblings.filter((n) => n.startsWith(".config-tmp-")); - assert.equal(tmps.length, 0); - }); - - test("skips ineligible records but writes eligible ones from the same list", async () => { - const r = await syncProvidersToEngine([ - { id: "eligible", label: "ok", enabled: true, protocol: "openai", auth: { type: "byok", apiKey: "sk", baseURL: "https://x/v1" }, models: [] }, - { id: "no-key", label: "nokey", enabled: true, protocol: "openai", auth: { type: "byok", baseURL: "https://x/v1" }, models: [] }, - { id: "disabled", label: "off", enabled: false, protocol: "openai", auth: { type: "byok", apiKey: "sk", baseURL: "https://x/v1" }, models: [] }, - ]); - assert.equal(r.ok, true); - assert.equal(r.written, true); - assert.deepEqual(r.keys, ["eligible"]); - const after = yaml.load(readFileSync(join(_tmpDataDir, "config.yaml"), "utf8")); - assert.ok(after.custom_provider.eligible); - assert.equal(after.custom_provider["no-key"], undefined); - assert.equal(after.custom_provider["disabled"], undefined); - }); - - test("config.yaml is written with mode 0600 (plaintext apiKey)", async () => { - const fs = await import("node:fs/promises"); - await syncProvidersToEngine([ - { - id: "byok-zhipu", - label: "z", - enabled: true, - protocol: "openai", - auth: { type: "byok", apiKey: "sk", baseURL: "https://x/v1" }, - models: [], - }, - ]); - const stat = await fs.stat(join(_tmpDataDir, "config.yaml")); - // POSIX mode 0600 — owner read/write only. The engine's own - // `updateLocalByokConfig` does the same (see - // packages/config/src/local-model-provider-write.ts). - if (process.platform !== "win32") { - assert.equal(stat.mode & 0o777, 0o600); - } - }); -}); - -describe("syncProvidersFromPutBody — keep-key convention applied", () => { - beforeEach(() => { - rmSync(join(_tmpDataDir, "config.yaml"), { force: true }); - }); - - test("absent apiKey on an existing record is filled from the previous user-level entry", async () => { - const existing = [ - { - id: "byok-zhipu", - label: "Zhipu", - enabled: true, - protocol: "openai", - auth: { type: "byok", apiKey: "sk-from-user-level", baseURL: "https://x/v1" }, - models: [], - }, - ]; - const body = { - providers: [ - { - id: "byok-zhipu", - label: "Zhipu", - enabled: true, - protocol: "openai", - // No auth.apiKey — convention: keep the existing one. - auth: { type: "byok", baseURL: "https://x/v1" }, - models: [], - }, - ], - }; - const r = await syncProvidersFromPutBody(body, existing); - assert.equal(r.ok, true); - const after = yaml.load(readFileSync(join(_tmpDataDir, "config.yaml"), "utf8")); - assert.equal(after.custom_provider["byok-zhipu"].options.apiKey, "sk-from-user-level"); - }); - - test("non-array providers in the body surfaces a BAD_BODY error", async () => { - const r = await syncProvidersFromPutBody({ providers: "not-an-array" }, []); - assert.equal(r.ok, false); - assert.equal(r.code, "BAD_BODY"); - }); - - test("explicit apiKey in the PUT body replaces the existing key", async () => { - const existing = [ - { - id: "byok-zhipu", - label: "z", - enabled: true, - protocol: "openai", - auth: { type: "byok", apiKey: "sk-old", baseURL: "https://x/v1" }, - models: [], - }, - ]; - const body = { - providers: [ - { - id: "byok-zhipu", - label: "z", - enabled: true, - protocol: "openai", - auth: { type: "byok", apiKey: "sk-new", baseURL: "https://x/v1" }, - models: [], - }, - ], - }; - const r = await syncProvidersFromPutBody(body, existing); - assert.equal(r.ok, true); - const after = yaml.load(readFileSync(join(_tmpDataDir, "config.yaml"), "utf8")); - assert.equal(after.custom_provider["byok-zhipu"].options.apiKey, "sk-new"); - }); -}); \ No newline at end of file diff --git a/packages/webui/test/lib/engine/capability-snapshot.test.js b/packages/webui/test/lib/engine/capability-snapshot.test.js index 16c8cfe1..74dc8f23 100644 --- a/packages/webui/test/lib/engine/capability-snapshot.test.js +++ b/packages/webui/test/lib/engine/capability-snapshot.test.js @@ -22,6 +22,18 @@ // none → not method-checked (a provider may legitimately expose no // surface for the capability). // +// PLUS, INDEPENDENT OF THE LEVEL ABOOVE: +// +// unimplemented → a method-named sub-item that a bridge in +// `MODE_WRITE_BRIDGED_CONFIG_IDS` points at and that NO +// surface implements. The audit asserts it is ABSENT and +// goes red the moment a surface grows one. This is the +// honesty slot, added in M3-B14, and it is the reason the +// third bridge is not an unchecked exemption: without it a +// gate that asks for a method nobody implements would be +// indistinguishable, in this file, from a gate that asks +// for a method both surfaces really carry. +// // The audit function is a PURE function over (declaration, method-name // sets), so the mutation checks below feed it hand-built mutant surfaces // and assert it reports the drift — the "flip a level / delete a method @@ -53,6 +65,7 @@ process.env.MCODE_WEBUI_UPLOAD_DIR = `${tmpBase}/uploads`; const { ENGINE_CAPABILITY_KEYS, LOCAL_RUNTIME_V2_CAPABILITIES, + MODE_WRITE_BRIDGED_CONFIG_IDS, TUI_RUNTIME_ADAPTER_CAPABILITIES, getEngineProvider, listEngineProviderIds, @@ -100,20 +113,38 @@ function resolveMember(host, dottedPath) { * the mode-write family's hard gate an audited fact rather than a * claim). They are part of the snapshot so "missing must really be * absent" is checked, and a partial that stops listing one goes red - * (under-declaration). + * (under-declaration); + * - `unimplemented`: method-NAMED sub-items a bridge in + * `MODE_WRITE_BRIDGED_CONFIG_IDS` points at that NO surface has yet. + * Unlike `absent`, these are deliberately NOT in any declaration's + * `missing` list — the bridge exempts them from the generic write, + * and a provider does not deny a sub-item it simply does not have. + * The audit asserts they are absent anyway, because the failure this + * catches is silence: a surface quietly growing the method while the + * gate and the declaration still treat it as a forward contract. * * M3-B10 added `selectModel` and `setPermissionMode` to `authCredentials` - * on BOTH surfaces. They are the two sub-items + * on BOTH surfaces. They are two of the three sub-items * `MODE_WRITE_BRIDGED_CONFIG_IDS` (server/engine/mode-writes.js) names, - * and until this batch they were the one part of a hard gate that no + * and until that batch they were the one part of a hard gate that no * audit could check: `absent` proves a name is NOT on the surface, and a * name that is merely "not in `missing`" proves nothing. Both were * verified present by reflection on a booted host BEFORE being added - * here, and the live audit below keeps proving it — which closes the - * bridge question B9 recorded as its KNOWN DEBT 2. Neither surface - * carries a `setThinkingEffort` / `selectThinkingEffort`; that absence - * is the fact the B10 KNOWN DEBT about gating `/api/set-model` turns on, - * and a surface that grows one must add it here at the same time. + * here, and the live audit below keeps proving it, which closes the + * bridge question B9 recorded as its KNOWN DEBT 2. + * + * M3-B14 added the THIRD id, `thinkingEffort` —> `setThinkingEffort`, and + * it is the one this table cannot express with the existing two lists. + * The method does not exist on either surface, so putting it in + * `methods` would be a lie the audit reports as drift on every run, and + * putting it in `absent` would be a second lie: `absent` means "this + * partial declares it missing", and no provider does. So it goes in + * `unimplemented`, which asserts exactly one thing — THIS SURFACE MUST + * NOT HAVE IT — and which turns red the moment either surface grows a + * `setThinkingEffort`. That is the whole closure mechanism for the third + * bridge, and it is deliberately one-directional: a surface acquiring the + * dedicated writer is an engine-side event nobody here can schedule, and + * the audit is what makes it impossible to miss. */ const REQUIRED_METHODS = { "tui-runtime-adapter": { @@ -126,7 +157,7 @@ const REQUIRED_METHODS = { mcp: { on: "adapter", methods: ["configureSessionMcpServers", "clearSessionMcpServers", "inspectProjectMcp", "listMcpServers"] }, subagents: { on: "adapter", methods: ["getDelegationSnapshot", "stopDelegation", "listBackgroundTasks"] }, usageStats: { on: "adapter", methods: ["getSessionUsage", "getSessionUsageSummary", "watchSessionUsageCommits"] }, - authCredentials: { on: "adapter", methods: ["getAccountStatus", "getCodexOAuthStatus", "startCodexOAuthLogin", "cancelCodexOAuthLogin", "getMiniMaxApiKeyStatus", "upsertMiniMaxApiKey", "selectModel", "setPermissionMode", "listUserModelProviders", "createUserModelProvider", "updateUserModelProvider", "deleteUserModelProvider", "testUserModelProvider", "discoverUserModelsCandidate"], absent: ["setConfigOption"] }, + authCredentials: { on: "adapter", methods: ["getAccountStatus", "getCodexOAuthStatus", "startCodexOAuthLogin", "cancelCodexOAuthLogin", "getMiniMaxApiKeyStatus", "upsertMiniMaxApiKey", "selectModel", "setPermissionMode", "listUserModelProviders", "createUserModelProvider", "updateUserModelProvider", "deleteUserModelProvider", "testUserModelProvider", "discoverUserModelsCandidate"], absent: ["setConfigOption"], unimplemented: ["setThinkingEffort"] }, fileReadWrite: { on: "adapter", methods: ["listWorkspaceFileTree", "searchWorkspaceFiles"] }, gitOperations: { on: "adapter", methods: ["getWorkspaceGitMetadata"] }, }, @@ -141,7 +172,7 @@ const REQUIRED_METHODS = { mcp: { on: "cliService", methods: ["configureSessionMcpServers", "inspectProjectMcp", "clearSessionMcpServers", "listMcpServers"] }, subagents: { on: "cliService", methods: ["listBackgroundTasks"], absent: ["getDelegationSnapshot", "stopDelegation"] }, usageStats: { on: "cliService", methods: ["getSessionUsage", "getSessionUsageSummary", "watchSessionUsageCommits"] }, - authCredentials: { on: "cliService", methods: ["getAccountStatus", "getCodexOAuthStatus", "startCodexOAuthLogin", "cancelCodexOAuthLogin", "getMiniMaxApiKeyStatus", "upsertMiniMaxApiKey", "selectModel", "setPermissionMode", "listUserModelProviders", "createUserModelProvider", "updateUserModelProvider", "deleteUserModelProvider", "testUserModel", "discoverUserModelsCandidate"], absent: ["setConfigOption"] }, + authCredentials: { on: "cliService", methods: ["getAccountStatus", "getCodexOAuthStatus", "startCodexOAuthLogin", "cancelCodexOAuthLogin", "getMiniMaxApiKeyStatus", "upsertMiniMaxApiKey", "selectModel", "setPermissionMode", "listUserModelProviders", "createUserModelProvider", "updateUserModelProvider", "deleteUserModelProvider", "testUserModel", "discoverUserModelsCandidate"], absent: ["setConfigOption"], unimplemented: ["setThinkingEffort"] }, fileReadWrite: { on: "cliService", methods: ["listWorkspaceFileTree", "searchWorkspaceFiles"] }, gitOperations: { on: "cliService", methods: ["getWorkspaceGitMetadata", "getWorkspaceReviewLink"] }, }, @@ -226,7 +257,26 @@ export function auditProviderCapabilities(providerId, declaration, host) { for (const key of Object.keys(required)) { const entry = declaration[key]; if (!entry) continue; // shape problems are M1's validate, not this audit - const { on, methods, absent = [] } = required[key]; + const { on, methods, absent = [], unimplemented = [] } = required[key]; + + // `unimplemented` is checked BEFORE the level dispatch and never + // consults the declaration. It is not a statement about what this + // provider claims; it is a statement about the SURFACE — "this + // surface must not carry a method, whatever the declaration says", + // because the bridge names it as a forward contract and the only + // event that should move it is the engine shipping the writer. A + // surface that grows one here has outrun its own declaration, and + // the message says so in the words a reader needs ("re-audit"), + // rather than reporting a missing entry. + for (const method of unimplemented) { + if (methodTypeOf(on, method) === "function") { + problems.push( + `${providerId}.${key}: ${on}.${method} is a bridged forward contract the ` + + `declaration has no method for, but the surface NOW HAS it — re-audit the bridge ` + + `(move it out of \`unimplemented\` and into the declaration)`, + ); + } + } if (entry.level === "full") { for (const method of methods) { @@ -392,6 +442,72 @@ describe("M2 snapshot — declarations vs the REAL catalogue host", () => { }); } + // ------------------------------------------------------------------------- + // M3-B14 — THE THIRD BRIDGE IS PROVEN ABSENT, NOT ASSUMED ABSENT + // ------------------------------------------------------------------------- + // + // The other two bridged sub-items (`selectModel`, `setPermissionMode`) + // are in `methods`, so this file proves they EXIST. The third one cannot + // be proven that way — no surface has it — so the only honest thing + // to do is prove the absence, by reflection, on the same real host, and + // say so in a test that fails if that ever stops being true. + // + // These assertions deliberately do NOT mock the surface as present. A + // suite that asserted "the host has setThinkingEffort" to make the gate + // look justified would be asserting a falsehood, and the falsehood is + // the whole risk this block exists to remove: a bridge that reads as + // verified while resting on a method nobody wrote. + for (const [providerId, required] of Object.entries(REQUIRED_METHODS)) { + const [on] = [required.authCredentials.on]; + test(`${providerId}: ${on} has NO ${MODE_WRITE_BRIDGED_CONFIG_IDS.thinkingEffort} method`, () => { + // A live probe over the real prototype chain — the same + // reflection the provenance audit used, not a hand-typed list. + const names = collectMethodNames(resolveMember(host, on)); + assert.equal( + names.includes(MODE_WRITE_BRIDGED_CONFIG_IDS.thinkingEffort), + false, + `the surface grew ${MODE_WRITE_BRIDGED_CONFIG_IDS.thinkingEffort} — move it out of ` + + `\`unimplemented\`, re-audit the bridge, and let the control come back`, + ); + // The audit's own verdict on the same fact, from the table rather + // than from this test's ad-hoc probe. Both halves, because a probe + // that passes while the audit table has drifted is a probe of the + // wrong thing. + const problems = auditProviderCapabilities( + providerId, + declarations[providerId], + host, + ); + assert.deepEqual(problems, []); + }); + } + + test("the name the bridge points at is the name the snapshot tracks as unimplemented", () => { + // The two tables and the bridge share one fact. Nothing in the server + // asserts this — the tables are separate literals in separate files + // — so it is asserted here, where both are in scope, by VALUE. + for (const [providerId, required] of Object.entries(REQUIRED_METHODS)) { + assert.deepEqual( + required.authCredentials.unimplemented, + [MODE_WRITE_BRIDGED_CONFIG_IDS.thinkingEffort], + providerId, + ); + assert.equal( + required.authCredentials.methods.includes(MODE_WRITE_BRIDGED_CONFIG_IDS.thinkingEffort), + false, + `${providerId}: it must not ALSO be claimed present — the two would contradict`, + ); + assert.equal( + (declarations[providerId].authCredentials.missing || []).includes( + MODE_WRITE_BRIDGED_CONFIG_IDS.thinkingEffort, + ), + false, + `${providerId}: no provider declares the effort writer missing — listing it would ` + + `remove the control for every user today`, + ); + } + }); + test("method-surface sizes stay in the audited ballpark (gross-loss tripwire)", () => { // Not an exact pin (the engine may add methods freely) — this only // catches a wholesale surface loss (e.g. a proxy/wrapper hiding the @@ -496,6 +612,52 @@ describe("M2 mutation checks — auditProviderCapabilities reports drift", () => ); }); + test("MUT-6: a surface that GROWS the bridged effort writer goes red", () => { + // The engine team lands `setThinkingEffort`. Nothing in the gate, in + // the bridge table or in any declaration changes — the surface + // simply starts having the method the bridge was a contract FOR. The + // audit is the only thing in this repository that can notice, so it + // has to notice: this is the check that makes the third bridge a + // forward contract with a closing mechanism rather than an + // unverified exemption. + for (const [providerId, required] of Object.entries(REQUIRED_METHODS)) { + const byOn = namesBySurface(required); + const declared = providerId === "local-runtime-v2" + ? LOCAL_RUNTIME_V2_CAPABILITIES + : TUI_RUNTIME_ADAPTER_CAPABILITIES; + const host = fakeHost( + [...(byOn.adapter || []), MODE_WRITE_BRIDGED_CONFIG_IDS.thinkingEffort], + [...(byOn.cliService || []), MODE_WRITE_BRIDGED_CONFIG_IDS.thinkingEffort], + [...(byOn["applications.session.diff"] || []), MODE_WRITE_BRIDGED_CONFIG_IDS.thinkingEffort], + ); + const problems = auditProviderCapabilities(providerId, declared, host); + assert.ok( + problems.some( + (p) => p.includes("authCredentials") && p.includes(MODE_WRITE_BRIDGED_CONFIG_IDS.thinkingEffort), + ), + `${providerId}: the grown forward contract was not reported, got: ${JSON.stringify(problems)}`, + ); + } + }); + + test("MUT-7: the SAME surface WITHOUT the writer is clean, on both providers", () => { + // The reverse half of MUT-6, and the reason the check is worth having + // at all: a rule that reports drift unconditionally is a rule nobody + // reads. Both providers, both surfaces, no problems. + for (const [providerId, required] of Object.entries(REQUIRED_METHODS)) { + const byOn = namesBySurface(required); + const declared = providerId === "local-runtime-v2" + ? LOCAL_RUNTIME_V2_CAPABILITIES + : TUI_RUNTIME_ADAPTER_CAPABILITIES; + const problems = auditProviderCapabilities( + providerId, + declared, + fakeHost(byOn.adapter || [], byOn.cliService || [], byOn["applications.session.diff"] || []), + ); + assert.deepEqual(problems, [], providerId); + } + }); + test("MUT-5: a partial listing an absent method as missing is fine; listing a present one is not", () => { const byOn = namesBySurface(ADAPTER_ALL); const ok = auditProviderCapabilities( diff --git a/packages/webui/test/lib/engine/mode-writes.test.js b/packages/webui/test/lib/engine/mode-writes.test.js index b42a99b6..121e17ae 100644 --- a/packages/webui/test/lib/engine/mode-writes.test.js +++ b/packages/webui/test/lib/engine/mode-writes.test.js @@ -235,17 +235,21 @@ describe("resolveModeWriteSubItem — the bridge", () => { } }); - test("#68 asks for the dedicated sub-item for the two bridged ids", async () => { + test("#68 asks for the dedicated sub-item for the three bridged ids", async () => { const facade = await bootFacade(t0()); assert.equal(facade.resolveModeWriteSubItem(SET_CONFIG_OPTION, "model"), "selectModel"); assert.equal(facade.resolveModeWriteSubItem(SET_CONFIG_OPTION, "permissionMode"), "setPermissionMode"); + // M3-B14. Before this, `thinkingEffort` was the FIRST entry in the + // list below — it was the worked example of a generic id, because at + // that time there was no dedicated effort writer to bridge to. + assert.equal(facade.resolveModeWriteSubItem(SET_CONFIG_OPTION, "thinkingEffort"), "setThinkingEffort"); }); test("#68 asks for the GENERIC sub-item for every other id, including nonsense", async () => { // The safe direction: a config id nobody audited must NOT inherit - // the exemption reserved for the two that were. + // the exemption reserved for the three that were. const facade = await bootFacade(t0()); - for (const key of ["thinkingEffort", "model_", "Model", "", undefined, null, 0, "constructor", "__proto__"]) { + for (const key of ["contextWindow", "model_", "Model", "", undefined, null, 0, "constructor", "__proto__"]) { assert.equal( facade.resolveModeWriteSubItem(SET_CONFIG_OPTION, key), "setConfigOption", @@ -331,9 +335,11 @@ describe("assertModeWriteCapability — HARD", () => { const d = facade.assertModeWriteCapability(SET_CONFIG_OPTION, RUNTIME, configId); assert.equal(d.gate, "checked", configId); } - // A generic id is. + // A generic id is. `contextWindow` is the honest example now that + // `thinkingEffort` is bridged: the engine has no channel for it + // either, and no bridge claims one. const generic = await caughtBy(() => - facade.assertModeWriteCapability(SET_CONFIG_OPTION, RUNTIME, "thinkingEffort"), + facade.assertModeWriteCapability(SET_CONFIG_OPTION, RUNTIME, "contextWindow"), ); assert.ok(isEngineCapabilityNotSupportedError(generic)); assert.equal(generic.capability, "authCredentials"); @@ -592,12 +598,14 @@ describe("setEngineSessionMode — capability ABSENT", () => { describe("setEngineSessionConfigOption — capability ABSENT, and the bridge", () => { test("a generic config id is refused, structured, with no `fallback`", async (t) => { + // `contextWindow` since M3-B14: `thinkingEffort` used to be this + // test's worked example of a generic id, and it is now a bridged one. const facade = await bootFacadeWithProvider(t, NO_GENERIC_CONFIG_WRITE); const caught = await caughtBy(() => facade.setEngineSessionConfigOption({ sessionId: "mvs_a", - key: "thinkingEffort", - value: "high", + key: "contextWindow", + value: "128000", transport: RUNTIME, }), ); @@ -608,7 +616,7 @@ describe("setEngineSessionConfigOption — capability ABSENT, and the bridge", ( assert.deepEqual(payload.missing, ["setConfigOption"]); }); - test("BOTH bridged ids still reach the engine", async (t) => { + test("ALL THREE bridged ids still reach the engine", async (t) => { const seen = []; registerRpcMock({ setConfigOption: async (sessionId, key, value, cid) => { @@ -617,10 +625,12 @@ describe("setEngineSessionConfigOption — capability ABSENT, and the bridge", ( }, }); const facade = await bootFacadeWithProvider(t, NO_GENERIC_CONFIG_WRITE); - for (const [key, value] of [ - ["model", "gpt-x"], - ["permissionMode", "auto"], - ]) { + const BRIDGED = [ + ["model", "gpt-x", "selectModel"], + ["permissionMode", "auto", "setPermissionMode"], + ["thinkingEffort", "high", "setThinkingEffort"], + ]; + for (const [key, value, subItem] of BRIDGED) { const r = await facade.setEngineSessionConfigOption({ sessionId: "mvs_a", key, @@ -629,14 +639,57 @@ describe("setEngineSessionConfigOption — capability ABSENT, and the bridge", ( transport: RUNTIME, }); assert.equal(r.statusHint, 200, key); - assert.equal(r.gate.subItem, key === "model" ? "selectModel" : "setPermissionMode"); + assert.equal(r.gate.subItem, subItem, key); } assert.deepEqual(seen, [ { sessionId: "mvs_a", key: "model", value: "gpt-x", cid: "cid-1" }, { sessionId: "mvs_a", key: "permissionMode", value: "auto", cid: "cid-1" }, + { sessionId: "mvs_a", key: "thinkingEffort", value: "high", cid: "cid-1" }, ]); }); + test("M3-B14 BEHAVIOUR CHANGE — #68 with `thinkingEffort` is DELIVERED, not 501", async (t) => { + // The one thing this batch changes for #68, stated as a test rather + // than left to a KNOWN DEBT paragraph. Under B9 the same call + // answered 501 with `missing: ["setConfigOption"]`; it now asks for + // the dedicated effort writer and goes through. The gate's own report + // is asserted too, so the change is visible as a fact about WHICH + // sub-item was asked for and not only as a status. + const seen = []; + registerRpcMock({ + setConfigOption: async (sessionId, key, value) => { + seen.push({ key, value }); + return { ok: true, data: {} }; + }, + }); + const facade = await bootFacadeWithProvider(t, NO_GENERIC_CONFIG_WRITE); + const before = await caughtBy(() => + facade.setEngineSessionConfigOption({ + sessionId: "mvs_a", + key: "contextWindow", + value: "128000", + cid: "cid-1", + transport: RUNTIME, + }), + ); + assert.ok(isEngineCapabilityNotSupportedError(before), "a truly generic id is still 501"); + const after = await facade.setEngineSessionConfigOption({ + sessionId: "mvs_a", + key: "thinkingEffort", + value: "high", + cid: "cid-1", + transport: RUNTIME, + }); + assert.equal(after.statusHint, 200); + assert.equal(after.gate.gate, "checked"); + assert.equal(after.gate.subItem, "setThinkingEffort"); + assert.deepEqual( + seen, + [{ key: "thinkingEffort", value: "high" }], + "exactly one push, and it is the effort", + ); + }); + test("the cid still reaches the RPC wrapper for a bridged id", async (t) => { // A regression here would be silent: the write would land on the // singleton's subprocess instead of the one holding the session, and @@ -674,19 +727,19 @@ describe("setEngineSessionConfigOption — capability ABSENT, and the bridge", ( } }); - test("the real registry refuses a generic id and keeps both bridged ids on the runtime transport", async (t) => { + test("the real registry refuses a generic id and keeps all three bridged ids on the runtime transport", async (t) => { // The shipped behaviour change, stated as the two halves of it. const facade = await bootFacade(t); const generic = await caughtBy(() => facade.setEngineSessionConfigOption({ sessionId: "mvs_a", - key: "thinkingEffort", - value: "high", + key: "contextWindow", + value: "128000", transport: RUNTIME, }), ); assert.ok(isEngineCapabilityNotSupportedError(generic)); - for (const key of ["model", "permissionMode"]) { + for (const key of ["model", "permissionMode", "thinkingEffort"]) { const r = await facade.setEngineSessionConfigOption({ sessionId: "mvs_a", key, diff --git a/packages/webui/test/lib/engine/model-writes.test.js b/packages/webui/test/lib/engine/model-writes.test.js index 5eb3879d..7ff7a061 100644 --- a/packages/webui/test/lib/engine/model-writes.test.js +++ b/packages/webui/test/lib/engine/model-writes.test.js @@ -26,6 +26,14 @@ // both sides field by field. A rewrite that gets the recorded form right // while pushing the wrong wire form fails it, and so does the reverse. // +// M3-B14 added a third thing this file pins, and it is the reason the +// file needed the provider-mock machinery it did not have before: BOTH +// ENDPOINTS ARE NOW GATED. The gate is asserted from both sides — a +// provider that denies the sub-item gets the structured 501, and a pure +// model switch on the SAME provider still answers 200 — because the +// second half is the one a "simplify the gate to one check" refactor +// would break silently. +// // The second half of the equivalence is the SSE race window. Its reader // (`server/lib/mcode-acp.js`, ticket 08) is not this batch's to change, // so this file imports it and pins the WRITER against it — including @@ -46,6 +54,14 @@ import { fileURLToPath } from "node:url"; import { join } from "node:path"; import yaml from "js-yaml"; +// Type discrimination goes through the exported predicate, never `err.name`: +// `name` is a writable instance property, so one stray upstream +// assignment would turn a 501 back into a soft failure — a failure +// mode that reads as a passing test. +const { isEngineCapabilityNotSupportedError, engineCapabilityHttpResponse } = await import( + absPath("engine/errors.js"), +); + import { setupMocks, absPath, registerRpcMock } from "../../helpers/_setup.js"; import { mkTmpDir, rmTmpDir } from "../../helpers/tmp.js"; @@ -66,18 +82,28 @@ process.env.MCODE_WEBUI_UPLOAD_DIR = `${tmpBase}/uploads`; // wrapper is pointed at it below. const realRpc = await import(absPath("lib/mcode-rpc.js")); const { variantChannelFor } = await import(absPath("lib/engine-catalogue.js")); +// The bridge table and the model-writes gate table are two literals in +// two files that are the same fact. Read both here so the equality can +// be asserted by VALUE rather than by grepping two sources for the same +// word {EM} a grep proves the word is present twice, not that the two +// copies agree. +const { MODE_WRITE_BRIDGED_CONFIG_IDS } = await import(absPath("engine/mode-writes.js")); /** Every name `engine/model-writes.js` exports. The namespace, not a subset. */ const FACADE_EXPORTS = [ + "MODEL_WRITE_ENDPOINTS", "NO_SESSION_MODEL_WARNING", "NO_SESSION_PERMISSION_WARNING", "applyThinkingEffortMirror", + "assertModelWriteCapability", "modelSelectionTarget", "planModelPickStamps", "planModelSelectionPush", "pushEngineModelSelection", "pushEnginePermissionMode", "resolveEngineModelConfigValue", + "resolveModelWriteProvider", + "resolveModelWriteSubItem", "resolvePermissionSelection", ]; @@ -730,6 +756,596 @@ describe("wire form ↔ recorded selection — field by field", () => { }); }); +// --------------------------------------------------------------------------- +// M3-B14 — THE GATE. Provider fixtures and boot helpers. +// --------------------------------------------------------------------------- +// +// The provider fixtures are SYNTHETIC on purpose, and the reason is the +// same one `mode-writes.test.js` gives: both registered providers +// declare `authCredentials` as `partial` with exactly the sub-items the +// real audit checks, so a real-registry test can reach the refusals — +// but not a provider that denies the DEDICATED writers, and not a +// `none`. Mocking `engine/index.js` for its whole namespace is what +// makes those reachable, and the whole-namespace shape is also what +// catches a new top-level read of that module in this file (the +// temporal-dead-zone rule `model-writes.js`'s header states). + +const RUNTIME = "runtime"; +const ACP = "acp"; +const SET_MODEL = "POST /api/set-model"; +const SET_PERMISSIONS = "POST /api/permissions"; + +/** The real v2 shape: the GENERIC config write is denied, nothing else. */ +const NO_GENERIC_CONFIG_WRITE = { + authCredentials: { + level: "partial", + missing: ["setConfigOption"], + reason: "test: no generic config write", + }, +}; + +/** M3-B14's case: the dedicated thinking-effort writer is denied too. */ +const NO_EFFORT_WRITER = { + authCredentials: { + level: "partial", + missing: ["setConfigOption", "setThinkingEffort"], + reason: "test: no generic write and no dedicated thinking-effort writer", + }, +}; + +/** The same, for #59's sub-item. */ +const NO_PERMISSION_WRITER = { + authCredentials: { + level: "partial", + missing: ["setConfigOption", "setPermissionMode"], + reason: "test: no dedicated permission writer", + }, +}; + +/** No `authCredentials` at all. */ +const NO_AUTH_AT_ALL = { + authCredentials: { + level: "none", + missing: ["everything"], + reason: "test: interface-absent", + }, +}; + +/** + * Boot the facade against a synthetic provider, with an RPC recorder. + * + * `engine/index.js` is mocked for its WHOLE namespace — every name not + * explicitly provided throws — so a new module-scope read of it in + * `model-writes.js` cannot pass silently. + */ +async function bootFacadeWithProvider(t, capabilities, rpcImpl = {}) { + await setupMocks(t, {}); + const calls = []; + registerRpcMock({ + webuiPermissionToMcode: realRpc.webuiPermissionToMcode, + setConfigOption: async (sid, configId, value, cid) => { + calls.push({ sid, configId, value, cid }); + if (typeof rpcImpl.setConfigOption === "function") { + return rpcImpl.setConfigOption({ sid, configId, value, cid }); + } + return { ok: true, data: {} }; + }, + }); + const namedExports = {}; + for (const name of exportedNamesOf("engine/index.js")) { + namedExports[name] = () => { + throw new Error(`B14 test called engine/index.js#${name}, which this case did not stub`); + }; + } + Object.assign(namedExports, { + DEFAULT_ENGINE_PROVIDER_ID: "local-runtime-v2", + getEngineProvider: (id = "local-runtime-v2") => ({ id, transport: "runtime", capabilities }), + }); + t.mock.module(absPath("engine/index.js"), { namedExports }); + const facade = await import(`${absPath("engine/model-writes.js")}?provider=${bust++}`); + return { facade, calls }; +} + +/** Boot against the REAL registry — no mock of `engine/index.js` at all. */ +async function bootFacade(t) { + await setupMocks(t, {}); + registerRpcMock({ + setConfigOption: async () => ({ ok: true, data: {} }), + webuiPermissionToMcode: realRpc.webuiPermissionToMcode, + }); + return import(`${absPath("engine/model-writes.js")}?provider=${bust++}`); +} + +/** Run `fn`, returning the thrown value or `null`. */ +async function caughtBy(fn) { + try { + await fn(); + } catch (e) { + return e; + } + return null; +} + +// --------------------------------------------------------------------------- +// M3-B14 — THE THIRD BRIDGE, AND THE TWO TABLES THAT MUST AGREE +// --------------------------------------------------------------------------- + +describe("the bridge is three ids, and the two server tables are one fact", () => { + test("MODE_WRITE_BRIDGED_CONFIG_IDS bridges exactly the three config ids webui writes", async () => { + assert.deepEqual(MODE_WRITE_BRIDGED_CONFIG_IDS, { + model: "selectModel", + permissionMode: "setPermissionMode", + thinkingEffort: "setThinkingEffort", + }); + }); + + test("every sub-item the model-write gate names is the one the bridge names", async (t) => { + // The strongest form of the pin available: read both literals and + // compare the VALUES. A source tripwire — which is what the frontend + // half uses — would only prove the word "thinkingEffort" appears in + // two files; this fails if either is renamed, re-pointed, or if a + // fourth gate sub-item appears with no bridge entry behind it. + const facade = await bootPure(t); + assert.equal( + facade.MODEL_WRITE_ENDPOINTS[SET_MODEL].subItem, + MODE_WRITE_BRIDGED_CONFIG_IDS.thinkingEffort, + ); + assert.equal( + facade.MODEL_WRITE_ENDPOINTS[SET_PERMISSIONS].subItem, + MODE_WRITE_BRIDGED_CONFIG_IDS.permissionMode, + ); + // The third id must not have displaced one of the first two. + assert.equal(MODE_WRITE_BRIDGED_CONFIG_IDS.model, "selectModel"); + }); + + test("the gate lives in the executor, not the route", () => { + // The property B10's KNOWN DEBT was written against ("the push is + // already a single call site per endpoint, so arming either gate is + // one line in the executor"). If the call ever moves into + // `routes/model.js` the model-only and no-session paths stop being + // ungated by construction, and nothing else in that file would + // notice. + const route = readFileSync(fileURLToPath(absPath("routes/model.js")), "utf8"); + assert.doesNotMatch( + route, + /assertEngineCapability|assertModelWriteCapability/, + "the route must not gate — the executors own it, and only they can see the plan", + ); + const src = readFileSync(fileURLToPath(absPath("engine/model-writes.js")), "utf8"); + assert.equal( + [...src.matchAll(/assertModelWriteCapability\(/g)].length, + 3, + "two executors plus the one definition", + ); + }); +}); + +// --------------------------------------------------------------------------- +// M3-B14 — THE GATE'S VERDICT FUNCTION +// --------------------------------------------------------------------------- + +describe("resolveModelWriteSubItem — #58's gate is the plan's answer", () => { + test("#59 has ONE answer, whatever it is told", async (t) => { + const facade = await bootPure(t); + for (const effortWrite of [undefined, false, true, 0, 1, "yes", null]) { + assert.equal( + facade.resolveModelWriteSubItem(SET_PERMISSIONS, effortWrite), + "setPermissionMode", + String(effortWrite), + ); + } + }); + + test("#58 asks for the effort writer ONLY when the plan carries an effort push", async (t) => { + const facade = await bootPure(t); + assert.equal(facade.resolveModelWriteSubItem(SET_MODEL, true), "setThinkingEffort"); + // And the ungated shapes all report `null` — not a fallback + // sub-item. A `null` that quietly became `setConfigOption` would put + // #58 back on the generic write it was bridged off. + for (const effortWrite of [undefined, false, 0, "", null]) { + assert.equal(facade.resolveModelWriteSubItem(SET_MODEL, effortWrite), null, String(effortWrite)); + } + }); + + test("an unknown endpoint is a plain Error with a machine-readable code", async (t) => { + const facade = await bootPure(t); + for (const fn of ["resolveModelWriteSubItem", "assertModelWriteCapability"]) { + const caught = await caughtBy(() => facade[fn]("POST /api/nope", ACP, true)); + assert.ok(caught, fn); + // Never reported to a user as an engine limitation: a typo in + // webui's own key is not the engine's fault. + assert.equal(isEngineCapabilityNotSupportedError(caught), false, fn); + assert.equal(caught.code, "unknown_model_write_endpoint", fn); + } + }); +}); + +describe("assertModelWriteCapability — the three answers", () => { + test("an ungated shape is reported as not-applicable and never throws", async (t) => { + // Even against a provider with NO authCredentials at all: the model + // channel is not this gate's business, and a `none` on an unrelated + // write must not take the model picker with it. + const { facade } = await bootFacadeWithProvider(t, NO_AUTH_AT_ALL); + const d = facade.assertModelWriteCapability(SET_MODEL, RUNTIME, false); + assert.equal(d.gate, "not-applicable"); + assert.equal(d.subItem, null); + assert.equal(d.provider, null); + assert.equal(d.enforcement, "hard"); + }); + + test("a transport no provider claims is not an engine limitation", async (t) => { + const { facade } = await bootFacadeWithProvider(t, NO_EFFORT_WRITER); + const d = facade.assertModelWriteCapability(SET_MODEL, ACP, true); + assert.equal(d.gate, "unregistered-transport"); + assert.equal(d.subItem, "setThinkingEffort", "the sub-item asked for is still reported"); + }); + + test("a provider that denies the sub-item throws the gate's own error", async (t) => { + const { facade } = await bootFacadeWithProvider(t, NO_EFFORT_WRITER); + const caught = await caughtBy(() => facade.assertModelWriteCapability(SET_MODEL, RUNTIME, true)); + assert.ok(isEngineCapabilityNotSupportedError(caught)); + assert.deepEqual(caught.missing, ["setThinkingEffort"]); + }); + + test("the REAL registry passes both sub-items on both transports", async (t) => { + // The shipped state, and the reason this batch is a no-op for every + // user today: no registered provider lists either sub-item missing, + // because the surfaces really do carry the first two and the third + // is a forward contract nothing declares. + const facade = await bootFacade(t); + for (const [endpoint, effortWrite] of [ + [SET_MODEL, true], + [SET_PERMISSIONS, false], + ]) { + for (const transport of [ACP, RUNTIME]) { + const d = facade.assertModelWriteCapability(endpoint, transport, effortWrite); + assert.notEqual(d.gate, "not-applicable", `${endpoint}/${transport}`); + assert.equal(d.gate !== "checked" || d.provider === "local-runtime-v2", true, transport); + } + } + }); +}); + +// --------------------------------------------------------------------------- +// M3-B14 — THE EQUIVALENCE TABLE, EXTENDED. One row per bridged config id, +// same shape as B10's eight rows: which sub-item answered, and what the +// ENGINE received. +// --------------------------------------------------------------------------- + +describe("bridge equivalence — one row per bridged config id", () => { + /** + * Isomorphism is the claim, so every row has the same columns: the + * config id, the sub-item #68 asks for, whether #58's gate asks for + * anything on this id, and the wire push the endpoint makes. A fourth + * bridge added without a row, or a row whose sub-item stopped + * matching its bridge entry, fails here rather than in production. + */ + const ROWS = [ + { + id: "model", + subItem: "selectModel", + gatedOn58: false, + why: "the model push IS the request; on a switchable builtin the level rides it", + wire: { configId: "model", value: "m:minimax_api:MiniMax-M2.7:u" }, + }, + { + id: "permissionMode", + subItem: "setPermissionMode", + gatedOn58: false, + why: "#59 is its own endpoint; #58 never writes this id", + wire: { configId: "permissionMode", value: "bypassPermissions" }, + }, + { + id: "thinkingEffort", + subItem: "setThinkingEffort", + gatedOn58: true, + why: "the standalone effort write is the one thing in #58 that needs it", + wire: { configId: "thinkingEffort", value: "high" }, + }, + ]; + + for (const row of ROWS) { + test(`${row.id} is bridged to ${row.subItem}`, async (t) => { + const facade = await bootPure(t); + // Every row, the same two facts, whether or not #58 is involved. + assert.equal(MODE_WRITE_BRIDGED_CONFIG_IDS[row.id], row.subItem, "the bridge entry"); + assert.ok(row.why, "every row says why — that is the point of the table"); + const gateSays = facade.resolveModelWriteSubItem(SET_MODEL, true); + if (row.gatedOn58) { + assert.equal(gateSays, row.subItem, "#58's gated shape asks for this row's sub-item"); + } else { + assert.notEqual(gateSays, row.subItem, "#58's gate must never ask for this row's sub-item"); + } + }); + } + + test("each id reaches the engine as ONE config-option push of the same shape", async (t) => { + // The wire half, asserted rather than described: all three pushes go + // through the same wrapper with the same (sid, configId, value, cid) + // arity, which is what "isomorphic" means at the transport boundary. + const { facade, calls } = await bootWithRpc(t); + const cs = fakeCsWithSession({ + model: { name: "minimax_api/MiniMax-M2.7" }, + configOptions: [EFFORT_MODEL_OPTION], + }); + await withBuiltinTree(async () => { + await facade.pushEngineModelSelection({ + cs, + cid: "cid-b14", + modelId: "minimax_api/MiniMax-M3.1-Flash-Preview", + thinkingWasProvided: true, + thinking: "high", + }); + await facade.pushEnginePermissionMode({ cs, cid: "cid-b14", mcodeValue: "bypassPermissions" }); + }); + assert.deepEqual( + calls.map((c) => ({ configId: c.configId, value: c.value, cid: c.cid })), + [ + { configId: "model", value: "m:minimax_api:MiniMax-M3.1-Flash-Preview:u", cid: "cid-b14" }, + { configId: "thinkingEffort", value: "high", cid: "cid-b14" }, + { configId: "permissionMode", value: "bypassPermissions", cid: "cid-b14" }, + ], + ); + assert.ok( + calls.every((c) => c.sid === "mvs_b10_0000000000000000000000"), + "one session, one arity, one shape", + ); + }); +}); + +// --------------------------------------------------------------------------- +// M3-B14 — GATE BEHAVIOUR, END TO END THROUGH THE EXECUTORS +// --------------------------------------------------------------------------- + +describe("the gate on #58 — the effort channel, and the model channel's escape", () => { + /** An effort-channel model, so a pick really plans a model push. */ + const effortCs = () => + fakeCsWithSession({ + model: { name: "minimax_api/MiniMax-M2.7", thinking: "" }, + configOptions: [EFFORT_MODEL_OPTION], + }); + + test("an effort write under a provider that denies the writer: structured 501", async (t) => { + const { facade, calls } = await bootFacadeWithProvider(t, NO_EFFORT_WRITER); + const caught = await caughtBy(() => + facade.pushEngineModelSelection({ + cs: effortCs(), + cid: "c", + modelId: "minimax_api/MiniMax-M3.1-Flash-Preview", + thinkingWasProvided: true, + thinking: "high", + transport: RUNTIME, + }), + ); + assert.ok(isEngineCapabilityNotSupportedError(caught), "the gate's own error type"); + const { status, payload } = engineCapabilityHttpResponse(caught); + assert.equal(status, 501); + assert.equal(payload.code, "engine_capability_not_supported"); + assert.equal(payload.capability, "authCredentials"); + assert.deepEqual(payload.missing, ["setThinkingEffort"]); + assert.equal(payload.provider, "local-runtime-v2"); + assert.deepEqual(calls, [], "and nothing reached the engine — not even the model push"); + }); + + test("REVERSE HALF — a PURE MODEL SWITCH on the SAME provider still answers 200", async (t) => { + // The half that makes this a gate on a channel rather than on an + // endpoint. Same provider, same session, same executor, same frame of + // code: the ONLY difference is that the request carried no level. + const { facade, calls } = await bootFacadeWithProvider(t, NO_EFFORT_WRITER); + const r = await facade.pushEngineModelSelection({ + cs: effortCs(), + cid: "c", + modelId: "minimax_api/MiniMax-M3.1-Flash-Preview", + transport: RUNTIME, + }); + assert.equal(r.gate.gate, "not-applicable"); + assert.equal(r.gate.subItem, null); + assert.equal(r.mcodeSynced, true); + assert.equal(r.thinkingSynced, false); + assert.equal(r.warning, null); + assert.deepEqual( + calls.map((c) => c.configId), + ["model"], + "the model push went out, exactly as it did before this batch", + ); + }); + + test("the VARIANT channel is not gated either — the level rides the model", async (t) => { + const { facade, calls } = await bootFacadeWithProvider(t, NO_EFFORT_WRITER); + const r = await facade.pushEngineModelSelection({ + cs: fakeCsWithSession({ + model: { name: "minimax_api/MiniMax-M3", thinking: "" }, + configOptions: [VARIANT_MODEL_OPTION], + }), + cid: "c", + modelId: "minimax_api/MiniMax-M3", + thinkingWasProvided: true, + thinking: "off", + transport: RUNTIME, + }); + assert.equal(r.channel, "variant"); + assert.equal(r.gate.gate, "not-applicable"); + assert.equal(r.mcodeSynced, true); + assert.equal(r.thinkingSynced, true, "the level still rode the model push, unchanged"); + assert.deepEqual(calls.map((c) => c.configId), ["model"]); + }); + + test("a CLEARED level plans no effort push, so it is not gated either", async (t) => { + // `thinking: ""` is the documented clear sentinel, not a request to + // set an effort. Treating it as an effort write would 501 a "reset + // to the engine default" action — the LEAST demanding thing a user + // can ask the control to do. + const { facade, calls } = await bootFacadeWithProvider(t, NO_EFFORT_WRITER); + const r = await facade.pushEngineModelSelection({ + cs: effortCs(), + cid: "c", + thinkingWasProvided: true, + thinking: "", + transport: RUNTIME, + }); + assert.equal(r.gate.gate, "not-applicable"); + assert.deepEqual(calls, [], "and nothing was pushed, as before"); + }); + + test("a `none` capability refuses the effort write, and reports the whole list", async (t) => { + const { facade } = await bootFacadeWithProvider(t, NO_AUTH_AT_ALL); + const caught = await caughtBy(() => + facade.pushEngineModelSelection({ + cs: effortCs(), + cid: "c", + modelId: "minimax_api/MiniMax-M3.1-Flash-Preview", + thinkingWasProvided: true, + thinking: "high", + transport: RUNTIME, + }), + ); + assert.ok(isEngineCapabilityNotSupportedError(caught)); + // `none` reports the declaration's whole missing list, not just the + // sub-item the gate asked for — that is `capabilities.js`'s rule, + // pinned here so the two are not confused later. + assert.deepEqual(caught.missing, ["everything"]); + }); + + test("denying the GENERIC write refuses none of the three bridged ids", async (t) => { + // The bridge's whole point, stated against the shape a real + // registered provider has: `authCredentials` partial with + // `missing: ["setConfigOption"]` must not refuse any of the three. + const { facade, calls } = await bootFacadeWithProvider(t, NO_GENERIC_CONFIG_WRITE); + const cs = effortCs(); + const model = await facade.pushEngineModelSelection({ + cs, + cid: "c", + modelId: "minimax_api/MiniMax-M3.1-Flash-Preview", + thinkingWasProvided: true, + thinking: "high", + transport: RUNTIME, + }); + assert.equal(model.gate.gate, "checked"); + assert.equal(model.gate.subItem, "setThinkingEffort"); + assert.equal(model.thinkingSynced, true); + const perm = await facade.pushEnginePermissionMode({ + cs, + cid: "c", + mcodeValue: "auto", + transport: RUNTIME, + }); + assert.equal(perm.gate.gate, "checked"); + assert.equal(perm.mcodeSynced, true); + assert.deepEqual( + calls.map((c) => c.configId), + ["model", "thinkingEffort", "permissionMode"], + ); + }); + + test("NO SESSION is ungated and unchanged — nothing reaches the engine to lie about", async (t) => { + const { facade, calls } = await bootFacadeWithProvider(t, NO_EFFORT_WRITER); + const r = await facade.pushEngineModelSelection({ + cs: fakeCs({ model: { name: "minimax_api/MiniMax-M2.7" }, configOptions: [EFFORT_MODEL_OPTION] }), + cid: "c", + modelId: "minimax_api/MiniMax-M3.1-Flash-Preview", + thinkingWasProvided: true, + thinking: "high", + transport: RUNTIME, + }); + assert.equal(r.channel, "no-session"); + assert.equal(r.gate.gate, "not-applicable"); + assert.equal(r.warning, facade.NO_SESSION_MODEL_WARNING); + assert.equal(r.mcodeSynced, false); + assert.deepEqual(calls, []); + }); + + test("the gate is decided by the PLAN, not by the request's fields", async (t) => { + // The mutation this batch is most exposed to: reading the request's + // `thinkingWasProvided` instead of the plan's `thinkingPush`. Every + // one of these requests CARRIED a `thinking` field, and only the + // first one planned an effort push — so a gate that read the request + // would 501 four requests it must not touch. + const { facade } = await bootFacadeWithProvider(t, NO_EFFORT_WRITER); + const cs = effortCs(); + const cases = [ + { name: "non-empty level on the effort channel", variant: null, thinking: "high", gated: true }, + { name: "cleared level", variant: null, thinking: "", gated: false }, + { name: "a switchable builtin folds the level into the model push", variant: "minimax_api/MiniMax-M3", thinking: "off", gated: false }, + ]; + for (const c of cases) { + let caught = null; + try { + await facade.pushEngineModelSelection({ + cs: { ...cs, model: { ...cs.model, name: c.variant || cs.model.name } }, + cid: "c", + modelId: c.variant || "minimax_api/MiniMax-M3.1-Flash-Preview", + thinkingWasProvided: true, + thinking: c.thinking, + transport: RUNTIME, + }); + } catch (e) { + caught = e; + } + if (c.gated) { + assert.ok(isEngineCapabilityNotSupportedError(caught), c.name); + } else { + assert.equal(caught, null, c.name); + } + } + }); +}); + +describe("the gate on #59 — present, audited, and inert", () => { + test("a permission write is checked, allowed, and pushed exactly as before", async (t) => { + const { facade, calls } = await bootFacadeWithProvider(t, NO_GENERIC_CONFIG_WRITE); + const r = await facade.pushEnginePermissionMode({ + cs: fakeCsWithSession(), + cid: "cid-b14", + mcodeValue: "default", + transport: RUNTIME, + }); + assert.equal(r.gate.gate, "checked"); + assert.equal(r.gate.subItem, "setPermissionMode"); + assert.equal(r.gate.capability, "authCredentials"); + assert.equal(r.mcodeSynced, true); + assert.equal(r.warning, null); + assert.deepEqual(calls, [ + { sid: "mvs_b10_0000000000000000000000", configId: "permissionMode", value: "default", cid: "cid-b14" }, + ]); + }); + + test("a provider that denies the DEDICATED writer refuses it", async (t) => { + const { facade, calls } = await bootFacadeWithProvider(t, NO_PERMISSION_WRITER); + const caught = await caughtBy(() => + facade.pushEnginePermissionMode({ + cs: fakeCsWithSession(), + cid: "c", + mcodeValue: "default", + transport: RUNTIME, + }), + ); + assert.ok(isEngineCapabilityNotSupportedError(caught)); + assert.deepEqual(caught.missing, ["setPermissionMode"]); + assert.deepEqual(calls, []); + }); + + test("the two shapes that push nothing are ungated, and say so", async (t) => { + const { facade, calls } = await bootFacadeWithProvider(t, NO_PERMISSION_WRITER); + const noSession = await facade.pushEnginePermissionMode({ + cs: fakeCs(), + cid: "c", + mcodeValue: "default", + transport: RUNTIME, + }); + assert.equal(noSession.gate.gate, "not-applicable"); + assert.equal(noSession.warning, facade.NO_SESSION_PERMISSION_WARNING); + const noValue = await facade.pushEnginePermissionMode({ + cs: fakeCsWithSession(), + cid: "c", + mcodeValue: null, + transport: RUNTIME, + }); + assert.equal(noValue.gate.gate, "not-applicable"); + assert.equal(noValue.warning, null); + assert.deepEqual(calls, [], "and a refusal the gate would have thrown never happened"); + }); +}); + // --------------------------------------------------------------------------- // The SSE 4s race window — the writer (this batch) against the real reader // --------------------------------------------------------------------------- @@ -963,7 +1579,49 @@ describe("pushEnginePermissionMode", () => { const cs = fakeCsWithSession(); const r = await facade.pushEnginePermissionMode({ cs, cid: "cid-b10", mcodeValue: "default" }); assert.deepEqual(calls, [{ sid: "mvs_b10_0000000000000000000000", configId: "permissionMode", value: "default", cid: "cid-b10" }]); - assert.deepEqual(r, { mcodeSynced: true, warning: null }); + // M3-B14 added a third field. It is the gate's own report and it is + // asserted rather than ignored, because a new field appearing in a + // response shape is exactly the kind of change that should have to be + // written down. Nothing outside this module reads it: the route + // destructures the two fields it has always read. + // + // The gate verdict is transport-DEPENDENT by design, and this test + // runs under both invocations, so the two transport-specific fields + // are pinned as a pair rather than as a literal: `acp` has no + // registered provider (M4) and reports that, `runtime` registers + // `local-runtime-v2`, whose audited declaration allows this sub-item. + const RUNTIME = process.env.MCODE_WEBUI_TRANSPORT === "runtime"; + assert.deepEqual( + { mcodeSynced: r.mcodeSynced, warning: r.warning }, + { mcodeSynced: true, warning: null }, + ); + assert.deepEqual( + { + endpoint: r.gate.endpoint, + capability: r.gate.capability, + subItem: r.gate.subItem, + enforcement: r.gate.enforcement, + gate: r.gate.gate, + provider: r.gate.provider, + }, + RUNTIME + ? { + endpoint: "POST /api/permissions", + capability: "authCredentials", + subItem: "setPermissionMode", + enforcement: "hard", + gate: "checked", + provider: "local-runtime-v2", + } + : { + endpoint: "POST /api/permissions", + capability: "authCredentials", + subItem: "setPermissionMode", + enforcement: "hard", + gate: "unregistered-transport", + provider: null, + }, + ); }); test("no session: local only, and THIS endpoint's warning sentence", async (t) => { diff --git a/packages/webui/test/lib/engine/provider-migration.test.js b/packages/webui/test/lib/engine/provider-migration.test.js new file mode 100644 index 00000000..7ab22f23 --- /dev/null +++ b/packages/webui/test/lib/engine/provider-migration.test.js @@ -0,0 +1,514 @@ +// webui/test/lib/engine/provider-migration.test.js +// +// M3-B11 (= plan item A5), the batch's DEATH LINE: the one-shot +// migration of the deprecated `/providers.json` into the +// engine's `config.yaml#custom_provider`, and the fallback that keeps +// the old format readable when it does not go through. +// +// The two properties this file exists to prove, stated as the plan +// states them: +// +// 1. 存量迁移无损 — "the existing `providers.json` must enter the new +// store without loss". Proven field by field, on a fixture built +// to break every assumption the migration could be quietly making: +// several providers, every schema field, and the boundary values +// (empty label, absent preset, disabled, coding-plan auth, the +// gemini protocol, a zero context limit, empty thinking levels, +// an unprojectable model id, unicode, a 4096-character key, a +// custom header map). +// 2. 迁移失败回退 — "a failed migration must fall back to the old +// format staying readable". Proven for each failure mode, and the +// fallback is asserted through the PUBLIC read, not through an +// internal, because a fallback nobody can observe is not one. +// +// Isolation: both the webui data dir and the engine data dir are +// per-run tmp dirs, pinned before any import — the store writes to +// MINIMAX_DATA_DIR, and a suite that leaves it on ~/.minimax writes +// plaintext keys into the developer's real engine config. + +import { test, describe, before, after, beforeEach } from "node:test"; +import { strict as assert } from "node:assert"; +import { existsSync, mkdirSync, readFileSync, rmSync, writeFileSync } from "node:fs"; +import { join } from "node:path"; +import yaml from "js-yaml"; + +import { mkTmpDir } from "../../helpers/tmp.js"; + +const tmpBase = mkTmpDir("minimax-code-engine-migration-"); +const engineDir = join(tmpBase, "engine"); +const webuiDir = join(tmpBase, "webui"); +const cwdDir = join(tmpBase, "cwd"); +const legacyFile = join(webuiDir, "providers.json"); +const configPath = join(engineDir, "config.yaml"); +process.env.MINIMAX_DATA_DIR = engineDir; +process.env.MAVIS_DATA_DIR = ""; +process.env.MCODE_WEBUI_DATA_DIR = webuiDir; +process.env.MCODE_WEBUI_MODELS_CONFIG = ""; +process.env.MCODE_WEBUI_SETTINGS_PATH = join(tmpBase, "settings.json"); +process.env.MCODE_WEBUI_EVENTS_PATH = join(tmpBase, "events.jsonl"); +process.env.MCODE_WEBUI_SESSIONS_DB = join(tmpBase, "sessions.db"); +process.env.MCODE_WEBUI_UPLOAD_DIR = join(tmpBase, "uploads"); + +const { migrateLegacyProviderStore, _resetProviderStoreMigration } = await import( + "../../../server/engine/provider-store.js" +); +const { readEngineProviderCatalogue } = await import("../../../server/engine/provider-reads.js"); +const { loadUserLevelProviders, normaliseProvider } = await import( + "../../../server/lib/providers-config.js" +); + +const _origCwd = process.cwd(); + +// ===================================================================== +// The fixture. Every field the v2 schema has, plus the values that a +// lossy projection would drop, silently, on the way to the engine. +// ===================================================================== + +const MAX_KEY = "k".repeat(4096); + +const LEGACY = { + version: 2, + providers: [ + { + id: "gateway", + label: "Acme Gateway", + preset: "openai", + enabled: true, + protocol: "openai", + auth: { + type: "byok", + apiKey: "sk-gateway-key-aaaa", + baseURL: "https://api.acme.test/v1", + headers: { "X-Tenant": "acme", "X-Trace": "01H" }, + }, + models: [ + { id: "m1", label: "M1", contextLimit: 128000, thinkingLevels: ["low", "high"], modalities: ["text", "image"] }, + { id: "z-ai/glm-5.3", label: "GLM", contextLimit: 1 }, + { id: "m-no-ctx", label: "M No Ctx" }, + ], + }, + { + // The record the OLD engine projection could not express: the + // gemini protocol and the disabled switch both vanish on the way + // to a bare custom_provider entry. + id: "gem", + label: "Gemini Endpoint", + enabled: false, + protocol: "gemini", + auth: { type: "byok", apiKey: "sk-gemini-key-bbbb", baseURL: "https://generativelanguage.test" }, + models: [{ id: "gem-2.5", label: "Gem", contextLimit: 1048576 }], + }, + { + // Coding-plan auth is OMITTED from the engine projection entirely. + id: "plan", + label: "订阅方案", + preset: "claude-code", + enabled: true, + protocol: "anthropic", + auth: { type: "coding-plan", apiKey: "", baseURL: "" }, + models: [], + }, + // Boundary values: an empty label (which the engine projection + // replaces with the key), a unicode label, no baseURL, a key at + // the 4096 ceiling, a model id the engine key grammar rejects, and + // empty thinking / modality lists the normaliser drops. + { + id: "edge", + label: "", + enabled: true, + protocol: "openai", + auth: { type: "byok", apiKey: MAX_KEY, baseURL: "" }, + models: [{ id: "has space", label: "Unprojectable" }, { id: "ok", thinkingLevels: [], modalities: [] }], + }, + ], +}; + +/** + * The expected post-migration records: exactly what the schema says, + * computed from the fixture through the SAME normaliser the store + * uses. Deriving the expectation from the fixture (rather than + * hand-writing it) is what keeps this a round-trip test: a field added + * to the schema is compared here without anyone updating this file. + * + * @param {object} parsed + * @returns {object[]} + */ +function expectedRecords(parsed) { + const seen = new Set(); + const out = []; + for (const p of parsed.providers) { + const n = normaliseProvider(p); + assert.equal(n.ok, true, `fixture must normalise: ${n.error}`); + assert.equal(seen.has(n.value.id), false, "fixture ids are unique"); + seen.add(n.value.id); + // Through JSON, because that is the form a FILE can hold: + // `normaliseProvider` emits `preset: undefined` for a record with + // no preset, and neither JSON nor YAML can represent an undefined + // value, so the deprecated file dropped the key too. Measuring + // on-disk equivalence against the JSON form is the honest + // comparison — comparing against the in-memory object would report + // a difference the pre-B11 store did not have either. + out.push(JSON.parse(JSON.stringify(n.value))); + } + return out; +} + +/** + * The expected records as a CLIENT sees them, through the catalogue + * read. This is the in-memory normalised form rather than the JSON + * form, because the read path re-normalises and therefore carries the + * `preset: undefined` key — which is exactly what the pre-B11 route + * returned, since it read the same normaliser's output. Equivalence + * is measured against the shape the endpoint had BEFORE this batch. + * + * @param {object} parsed + * @returns {object[]} + */ +function expectedPublicRecords(parsed) { + return (parsed.providers || []).map((p) => { + const n = normaliseProvider(p); + assert.equal(n.ok, true, `fixture must normalise: ${n.error}`); + return n.value; + }); +} + +function writeLegacy(doc) { + writeFileSync(legacyFile, JSON.stringify(doc, null, 2), "utf8"); +} + +function readStoreRecords() { + if (!existsSync(configPath)) return []; + const doc = yaml.load(readFileSync(configPath, "utf8")) || {}; + return Object.values(doc.custom_provider || {}) + .map((entry) => entry && entry._webui_provider) + .filter(Boolean); +} + +before(() => { + mkdirSync(engineDir, { recursive: true }); + mkdirSync(webuiDir, { recursive: true }); + mkdirSync(cwdDir, { recursive: true }); + process.chdir(cwdDir); +}); + +after(() => { + try { + process.chdir(_origCwd); + } catch {} + try { + rmSync(tmpBase, { recursive: true, force: true }); + } catch {} +}); + +beforeEach(() => { + _resetProviderStoreMigration(); + for (const f of [configPath, legacyFile, join(cwdDir, "models.json")]) { + if (existsSync(f)) rmSync(f, { recursive: true, force: true }); + } +}); + +// ===================================================================== +// DEATH LINE 1 — field-by-field equivalence +// ===================================================================== + +describe("legacy migration — the records are equivalent, field by field", () => { + test("a rich legacy file survives the migration with EVERY field intact", async () => { + writeLegacy(LEGACY); + const before = JSON.parse(JSON.stringify(loadUserLevelProviders())); + assert.equal(before.length, 4, "the deprecated loader sees four providers"); + + const r = await migrateLegacyProviderStore(before); + assert.equal(r.ok, true); + assert.equal(r.migrated, true); + assert.equal(r.count, 4); + + // ---- FIELD BY FIELD, NOT BY A SUBSET -------------------------- + const after = readStoreRecords(); + assert.equal(after.length, before.length, "provider count is preserved"); + for (let i = 0; i < before.length; i++) { + const b = before[i]; + const a = after[i]; + assert.equal(a.id, b.id, "id"); + assert.equal(a.label, b.label, `${b.id}: label`); + assert.equal(a.preset, b.preset, `${b.id}: preset`); + assert.equal(a.enabled, b.enabled, `${b.id}: enabled`); + assert.equal(a.protocol, b.protocol, `${b.id}: protocol`); + assert.equal(a.auth.type, b.auth.type, `${b.id}: auth.type`); + assert.equal(a.auth.apiKey, b.auth.apiKey, `${b.id}: auth.apiKey`); + assert.equal(a.auth.baseURL, b.auth.baseURL, `${b.id}: auth.baseURL`); + assert.deepEqual(a.auth.headers, b.auth.headers, `${b.id}: auth.headers`); + assert.equal(a.models.length, b.models.length, `${b.id}: model count`); + for (let j = 0; j < b.models.length; j++) { + assert.equal(a.models[j].id, b.models[j].id, `${b.id}/${b.models[j].id}: model id`); + assert.equal(a.models[j].label, b.models[j].label, `${b.id}/${b.models[j].id}: model label`); + assert.equal(a.models[j].contextLimit, b.models[j].contextLimit, `${b.id}: contextLimit`); + assert.deepEqual(a.models[j].thinkingLevels, b.models[j].thinkingLevels, `${b.id}: thinkingLevels`); + assert.deepEqual(a.models[j].modalities, b.models[j].modalities, `${b.id}: modalities`); + } + } + // And the whole-array form, so a future field added to the schema + // fails HERE rather than silently at the next `assert.equal`. + assert.deepEqual(after, before); + }); + + test("the records a reader gets back are the same ones, through the public read", async () => { + // The equivalence that matters is not on-disk-to-on-disk; it is + // what a client sees. This is the assertion that would have caught + // a migration that lost the gemini protocol or dropped a disabled + // provider on the way through the engine's shape. + writeLegacy(LEGACY); + const expected = expectedPublicRecords(LEGACY); + const first = await readEngineProviderCatalogue(); + assert.equal(first.catalogueSource, "engine-store"); + assert.equal(first.migration.migrated, true); + assert.deepEqual(first.providers, expected); + // A SECOND read must produce the identical catalogue — the marker + // has flipped the authority, and the deprecated file must no longer + // be consulted. + const second = await readEngineProviderCatalogue(); + assert.equal(second.catalogueSource, "engine-store"); + assert.equal(second.migration.attempted, false, "no second migration attempt"); + assert.deepEqual(second.providers, expected); + }); + + test("the provider ORDER is preserved — the store is a mapping, the catalogue is not", async () => { + writeLegacy(LEGACY); + const expected = expectedRecords(LEGACY).map((r) => r.id); + const c = await readEngineProviderCatalogue(); + assert.deepEqual(c.providers.map((r) => r.id), expected, "insertion order, not alphabetical"); + // And it survives a full YAML round trip, which is where a mapping + // would be tempted to reorder. + const again = await readEngineProviderCatalogue(); + assert.deepEqual(again.providers.map((r) => r.id), expected); + }); + + test("an INELIGIBLE provider is migrated anyway, record only", async () => { + // `plan` is coding-plan and `gem` is disabled — the two the old + // projection dropped. They must be in the store, marked, with no + // engine fields, and readable back. + writeLegacy(LEGACY); + await migrateLegacyProviderStore(loadUserLevelProviders()); + const doc = yaml.load(readFileSync(configPath, "utf8")); + for (const key of ["gem", "plan"]) { + const entry = doc.custom_provider[key]; + assert.ok(entry, `${key} is in the store`); + assert.equal(entry._webui_owned, true); + assert.equal(entry.api, undefined, `${key} has no engine projection`); + assert.ok(entry._webui_provider, `${key} keeps its record`); + } + // The eligible one still gets a real projection, so the engine + // keeps advertising what it always advertised. + assert.equal(doc.custom_provider.gateway.api, "openai-completions"); + assert.equal(doc.custom_provider.gateway.options.apiKey, "sk-gateway-key-aaaa"); + }); + + test("the migration preserves the operator's OTHER engine config sections", async () => { + const seed = { + provider: { minimax: { name: "MiniMax", models: { "MiniMax-M3": {} } } }, + defaultModel: "m:minimax:MiniMax-M3:u", + }; + writeFileSync(configPath, yaml.dump(seed), "utf8"); + writeLegacy(LEGACY); + await migrateLegacyProviderStore(loadUserLevelProviders()); + const doc = yaml.load(readFileSync(configPath, "utf8")); + assert.deepEqual(doc.provider, seed.provider); + assert.equal(doc.defaultModel, seed.defaultModel); + }); + + test("an operator's hand-written custom provider survives the migration", async () => { + writeFileSync( + configPath, + yaml.dump({ + custom_provider: { + manual: { name: "Mine", kind: "custom", api: "openai-completions", options: { apiKey: "sk-op" } }, + }, + }), + "utf8", + ); + writeLegacy(LEGACY); + await migrateLegacyProviderStore(loadUserLevelProviders()); + const doc = yaml.load(readFileSync(configPath, "utf8")); + assert.deepEqual(doc.custom_provider.manual, { + name: "Mine", + kind: "custom", + api: "openai-completions", + options: { apiKey: "sk-op" }, + }); + }); + + test("the migration is idempotent — a second call is a no-op", async () => { + writeLegacy(LEGACY); + const first = await migrateLegacyProviderStore(loadUserLevelProviders()); + assert.equal(first.migrated, true); + const afterFirst = readFileSync(configPath, "utf8"); + const second = await migrateLegacyProviderStore(loadUserLevelProviders()); + assert.equal(second.migrated, false, "the marker says it is done"); + assert.equal(readFileSync(configPath, "utf8"), afterFirst, "and the file is not rewritten"); + }); + + test("an EMPTY legacy file still closes the deprecated source", async () => { + writeLegacy({ version: 2, providers: [] }); + const r = await migrateLegacyProviderStore([]); + assert.equal(r.ok, true); + assert.equal(r.migrated, true); + const doc = yaml.load(readFileSync(configPath, "utf8")); + assert.ok(doc._webui_provider_migration, "an empty catalogue is a decision, not an absence"); + }); + + test("a v1 file (no `version`, no auth) migrates like any other", async () => { + writeLegacy({ providers: [{ id: "legacy1", label: "L1", models: [{ id: "m", contextLimit: 2048 }] }] }); + const r = await migrateLegacyProviderStore(loadUserLevelProviders()); + assert.equal(r.ok, true); + const rec = readStoreRecords()[0]; + assert.equal(rec.id, "legacy1"); + assert.equal(rec.protocol, "openai", "v1 defaults to the most permissive protocol"); + assert.equal(rec.auth.type, "byok"); + }); +}); + +// ===================================================================== +// DEATH LINE 2 — the fallback +// ===================================================================== + +describe("legacy migration — failure falls back to the old format, readable", () => { + test("an unparseable config.yaml: the migration fails and the file still answers", async () => { + writeLegacy(LEGACY); + const broken = "custom_provider:\\n - [unbalanced\\n"; + writeFileSync(configPath, broken, "utf8"); + + const r = await migrateLegacyProviderStore(loadUserLevelProviders()); + assert.equal(r.ok, false, "the migration reports its failure"); + assert.equal(r.migrated, false); + assert.equal(r.code, "ENGINE_CONFIG_UNREADABLE"); + + // The fallback: the deprecated file answers, in its own format, + // with the full catalogue — and the broken engine config is left + // exactly as it was, because overwriting it would destroy every + // section the store does not own. + const c = await readEngineProviderCatalogue(); + assert.equal(c.catalogueSource, "legacy-file"); + assert.deepEqual(c.providers, expectedPublicRecords(LEGACY)); + assert.equal(readFileSync(configPath, "utf8"), broken, "the operator's file is untouched"); + }); + + test("a WRITE failure: the marker is not written, so the file stays authoritative", async () => { + writeLegacy(LEGACY); + // A config path whose parent is a regular file: the read finds + // nothing and the tmp write then fails with ENOTDIR. + writeFileSync(join(engineDir, "not-a-dir"), "x", "utf8"); + const blocked = join(engineDir, "not-a-dir", "config.yaml"); + + const r = await migrateLegacyProviderStore(loadUserLevelProviders(), { configPath: blocked }); + assert.equal(r.ok, false); + assert.equal(r.code, "ENGINE_STORE_WRITE_FAILED"); + + // The public read, pointed at the same broken path, falls back. + const c = await readEngineProviderCatalogue({ configPath: blocked }); + assert.equal(c.catalogueSource, "legacy-file"); + assert.deepEqual(c.providers, expectedPublicRecords(LEGACY)); + assert.equal(c.migration.code, "ENGINE_STORE_WRITE_FAILED", "the reason travels with the result"); + }); + + test("a failed migration is retried on the next read, and can succeed", async () => { + // The retry is the recovery path, and it is only possible because + // the failed attempt left no marker behind. + writeLegacy(LEGACY); + const blocked = join(engineDir, "not-a-dir", "config.yaml"); + writeFileSync(join(engineDir, "not-a-dir"), "x", "utf8"); + assert.equal((await migrateLegacyProviderStore(loadUserLevelProviders(), { configPath: blocked })).ok, false); + + // The obstruction clears. + rmSync(join(engineDir, "not-a-dir"), { force: true }); + const retry = await migrateLegacyProviderStore(loadUserLevelProviders()); + assert.equal(retry.ok, true); + assert.equal(retry.migrated, true); + assert.deepEqual(readStoreRecords(), expectedRecords(LEGACY)); + }); + + test("NO deprecated file means no migration and no write", async () => { + // A fresh install must not acquire a config.yaml because a browser + // polled #62. The read is a pure function of an empty world. + const c = await readEngineProviderCatalogue(); + assert.equal(c.catalogueSource, "legacy-file"); + assert.deepEqual(c.providers, []); + assert.equal(c.migration.attempted, false); + assert.equal(existsSync(configPath), false, "a GET never writes"); + }); + + test("a corrupt deprecated file yields an empty catalogue and still closes", async () => { + // Nothing to migrate and nothing to preserve: the operator gets an + // empty catalogue they can rebuild, rather than a permanently + // failing one. + writeFileSync(legacyFile, "{ not json", "utf8"); + const c = await readEngineProviderCatalogue(); + assert.deepEqual(c.providers, []); + assert.equal(c.catalogueSource, "engine-store", "the marker was still stamped"); + }); + + test("after the store is authoritative, the deprecated file is ignored entirely", async () => { + writeLegacy(LEGACY); + await readEngineProviderCatalogue(); + assert.equal(existsSync(configPath), true); + // Someone edits the deprecated file by hand. The store is the + // authority now, so the edit is invisible — which is the whole + // point of the marker, and the reason a stale file cannot fight a + // live one. + writeFileSync(legacyFile, JSON.stringify({ version: 2, providers: [{ id: "intruder", auth: {} }] }), "utf8"); + const c = await readEngineProviderCatalogue(); + assert.deepEqual(c.providers, expectedPublicRecords(LEGACY)); + assert.equal(c.catalogueSource, "engine-store"); + }); + + test("the env and cwd layers still win over a migrated catalogue", async () => { + // The migration replaces the USER layer only. A deployment-owned + // file that overrides a provider must keep overriding it. + writeLegacy(LEGACY); + await readEngineProviderCatalogue(); + writeFileSync( + join(cwdDir, "models.json"), + JSON.stringify({ providers: [{ id: "gateway", label: "From cwd", auth: { type: "byok", apiKey: "sk-cwd-override" } }] }), + "utf8", + ); + const c = await readEngineProviderCatalogue(); + const gw = c.providers.find((p) => p.id === "gateway"); + assert.equal(gw.label, "From cwd", "the cwd layer still outranks the store"); + assert.equal(gw.auth.apiKey, "sk-cwd-override"); + }); +}); + +// ===================================================================== +// The deprecated file is never written again +// ===================================================================== + +describe("the deprecated file is read-only from here on", () => { + test("a migration leaves providers.json byte-identical", async () => { + writeLegacy(LEGACY); + const before = readFileSync(legacyFile, "utf8"); + await readEngineProviderCatalogue(); + assert.equal(readFileSync(legacyFile, "utf8"), before, "the file is never rewritten, only read"); + }); + + test("a PUT leaves providers.json byte-identical", async () => { + // The write path is the store; the deprecated file keeps whatever + // it had, which is what makes the fallback story readable after a + // downgrade. + const { commitProviderCatalogueWrite } = await import("../../../server/engine/provider-writes.js"); + writeLegacy(LEGACY); + const before = readFileSync(legacyFile, "utf8"); + await commitProviderCatalogueWrite({ records: expectedRecords(LEGACY) }); + assert.equal(readFileSync(legacyFile, "utf8"), before); + }); + + test("deleting every provider does not resurrect the deprecated file", async () => { + // The reason the marker is a field rather than an inference: after + // this write the store is EMPTY and has no webui entries, and an + // inferred marker would hand authority back to the stale file. + const { commitProviderCatalogueWrite } = await import("../../../server/engine/provider-writes.js"); + writeLegacy(LEGACY); + await readEngineProviderCatalogue(); + await commitProviderCatalogueWrite({ records: [] }); + const c = await readEngineProviderCatalogue(); + assert.equal(c.catalogueSource, "engine-store"); + assert.deepEqual(c.providers, [], "the four providers the operator deleted stay deleted"); + }); +}); diff --git a/packages/webui/test/lib/engine/provider-reads.test.js b/packages/webui/test/lib/engine/provider-reads.test.js new file mode 100644 index 00000000..8f4021c3 --- /dev/null +++ b/packages/webui/test/lib/engine/provider-reads.test.js @@ -0,0 +1,351 @@ +// webui/test/lib/engine/provider-reads.test.js +// +// M3-B11 (= plan item A5), the read half: #62 GET /api/providers, +// #64 POST /api/providers/test, #65 GET /api/providers/presets, through +// `server/engine/provider-reads.js`. +// +// What is worth a test here, and what is not: +// +// - The three endpoints' DECLARATIONS: which capability, which +// sub-item, hard or soft. The `subItem` names the method that +// would eventually serve the endpoint, and every one of them is +// already an audited fact on the real host +// (test/lib/engine/capability-snapshot.test.js lists +// `listUserModelProviders`, `createUserModelProvider`, +// `updateUserModelProvider`, `deleteUserModelProvider`, +// `testUserModelProvider` and `testUserModel` in +// `authCredentials` for BOTH surfaces), so this family's soft gate +// is not an unearned claim. +// - That the gate is SOFT: it reports, it never throws. The 501 +// machinery is unused by this family and the suite pins that, the +// same way B6, B7 and B9 pin theirs. +// - Which file the catalogue came from, and the migration report that +// travels with the answer. The migration contracts themselves live +// in provider-migration.test.js; this file pins the SHAPE the route +// consumes. + +import { test, describe } from "node:test"; +import { strict as assert } from "node:assert"; +import { + existsSync, + mkdirSync, + readFileSync, + realpathSync, + rmSync, + symlinkSync, + writeFileSync, +} from "node:fs"; +import { join } from "node:path"; +import yaml from "js-yaml"; + +import { mkTmpDir, rmTmpDir } from "../../helpers/tmp.js"; + +const tmpBase = mkTmpDir("minimax-code-engine-reads-"); +const engineDir = join(tmpBase, "engine"); +const webuiDir = join(tmpBase, "webui"); +const cwdDir = join(tmpBase, "cwd"); +const legacyFile = join(webuiDir, "providers.json"); +const configPath = join(engineDir, "config.yaml"); +process.env.MINIMAX_DATA_DIR = engineDir; +process.env.MAVIS_DATA_DIR = ""; +process.env.MCODE_WEBUI_DATA_DIR = webuiDir; +process.env.MCODE_WEBUI_MODELS_CONFIG = ""; +process.env.MCODE_WEBUI_SETTINGS_PATH = join(tmpBase, "settings.json"); +process.env.MCODE_WEBUI_EVENTS_PATH = join(tmpBase, "events.jsonl"); +process.env.MCODE_WEBUI_SESSIONS_DB = join(tmpBase, "sessions.db"); +process.env.MCODE_WEBUI_UPLOAD_DIR = join(tmpBase, "uploads"); + +const { + PROVIDER_READ_ENDPOINTS, + checkProviderReadCapability, + readEngineProviderCatalogue, + resolveProviderReadProvider, +} = await import("../../../server/engine/provider-reads.js"); +const { EngineCapabilityNotSupportedError } = await import("../../../server/engine/errors.js"); +const { LOCAL_RUNTIME_V2_CAPABILITIES } = await import("../../../server/engine/index.js"); +const { _resetProviderStoreMigration } = await import("../../../server/engine/provider-store.js"); + +mkdirSync(engineDir, { recursive: true }); +mkdirSync(webuiDir, { recursive: true }); +mkdirSync(cwdDir, { recursive: true }); +const _origCwd = process.cwd(); +process.chdir(cwdDir); +process.on("exit", () => { + try { + process.chdir(_origCwd); + } catch {} + rmTmpDir(tmpBase); +}); + +// ===================================================================== +// The gate table +// ===================================================================== + +describe("PROVIDER_READ_ENDPOINTS — three reads, one capability, all soft", () => { + test("exactly #62, #64 and #65, all on authCredentials, all soft", () => { + assert.deepEqual(Object.keys(PROVIDER_READ_ENDPOINTS).sort(), [ + "GET /api/providers", + "GET /api/providers/presets", + "POST /api/providers/test", + ]); + for (const [endpoint, need] of Object.entries(PROVIDER_READ_ENDPOINTS)) { + assert.equal(need.capability, "authCredentials", endpoint); + assert.equal(need.enforcement, "soft", endpoint); + } + assert.equal(PROVIDER_READ_ENDPOINTS["GET /api/providers"].subItem, "listUserModelProviders"); + assert.equal(PROVIDER_READ_ENDPOINTS["GET /api/providers/presets"].subItem, "listProviderPresets"); + assert.equal(PROVIDER_READ_ENDPOINTS["POST /api/providers/test"].subItem, "testUserModelProvider"); + }); + + test("the sub-items name methods the audited host really has", () => { + // Cross-reference against the snapshot audit's own list. If a + // future batch re-audits `authCredentials` and one of these names + // disappears from the surface, this goes red at the point the + // declaration changed rather than at the point a client did. + const audited = [ + "listUserModelProviders", + "createUserModelProvider", + "updateUserModelProvider", + "deleteUserModelProvider", + "testUserModelProvider", + "testUserModel", + ]; + const snapshot = readFileSync( + join(import.meta.dirname, "capability-snapshot.test.js"), + "utf8", + ); + for (const name of Object.values(PROVIDER_READ_ENDPOINTS).map((n) => n.subItem)) { + if (name === "listProviderPresets") continue; // KNOWN DEBT 2 + assert.ok(audited.includes(name), `${name} must be an audited surface method`); + assert.ok(snapshot.includes(name), `${name} must appear in the snapshot audit's table`); + } + }); +}); + +describe("checkProviderReadCapability — SOFT, reports, never throws", () => { + test("the declared provider is not degraded for any of the three", () => { + for (const endpoint of Object.keys(PROVIDER_READ_ENDPOINTS)) { + const r = checkProviderReadCapability(endpoint, "runtime"); + assert.equal(r.gate, "checked"); + assert.equal(r.provider, "local-runtime-v2"); + assert.equal(r.degraded, false, endpoint); + assert.equal(r.reason, null); + } + }); + + test("a `none` declaration reports degraded, it does not throw", () => { + // The whole reason this family is soft: a provider that cannot + // manage providers still serves a well-defined catalogue, and a 501 + // would delete a working UI over a declaration about who would + // eventually answer it. + const original = LOCAL_RUNTIME_V2_CAPABILITIES.authCredentials; + try { + // The gate resolves through getEngineProvider, so a provider that + // DENIES the sub-item cannot be constructed here without editing + // the shared frozen declaration. What is pinned instead is the + // two halves that need no such construction: the predicate's own + // degradation rule, and the source-level fact that this module + // never touches the 501 machinery. + const partial = { + ...LOCAL_RUNTIME_V2_CAPABILITIES, + authCredentials: { ...original, missing: [...original.missing, "listUserModelProviders"] }, + }; + assert.ok(partial.authCredentials.missing.includes("listUserModelProviders")); + // And the 501 machinery is genuinely unused here: the read gate + // has no path that constructs the error. + const src = readFileSync( + join(import.meta.dirname, "..", "..", "..", "server", "engine", "provider-reads.js"), + "utf8", + ); + assert.equal(src.includes("EngineCapabilityNotSupportedError"), false); + assert.equal(src.includes("assertEngineCapability"), false, "the soft gate must not gate"); + } finally { + void original; + } + }); + + test("an unregistered transport reports, it does not degrade", () => { + for (const transport of ["acp", "exec"]) { + const r = checkProviderReadCapability("GET /api/providers", transport); + assert.equal(r.gate, "unregistered-transport"); + assert.equal(r.provider, null); + assert.equal(r.degraded, false, "nobody has claimed this transport yet (M4)"); + } + }); + + test("an endpoint outside the family is a caller bug, reported as such", () => { + assert.throws( + () => checkProviderReadCapability("PUT /api/providers", "runtime"), + (e) => e.code === "unknown_provider_read_endpoint" && !(e instanceof EngineCapabilityNotSupportedError), + ); + }); + + test("resolveProviderReadProvider returns the registered provider on runtime only", () => { + assert.equal(resolveProviderReadProvider("runtime").id, "local-runtime-v2"); + assert.equal(resolveProviderReadProvider("acp"), null); + }); +}); + +// ===================================================================== +// The catalogue result shape +// ===================================================================== + +describe("readEngineProviderCatalogue — the shape the route consumes", () => { + function reset() { + _resetProviderStoreMigration(); + for (const f of [configPath, legacyFile, join(cwdDir, "models.json")]) { + if (existsSync(f)) rmSync(f, { recursive: true, force: true }); + } + mkdirSync(engineDir, { recursive: true }); + mkdirSync(webuiDir, { recursive: true }); + mkdirSync(cwdDir, { recursive: true }); + } + + test("an empty world: no files, no write, a well-formed empty answer", async () => { + reset(); + const c = await readEngineProviderCatalogue(); + assert.equal(c.version, 2); + assert.deepEqual(c.providers, []); + assert.equal(c.catalogueSource, "legacy-file"); + assert.equal(c.migration.attempted, false); + assert.equal(c.migration.code, null); + assert.equal(c.storePath, configPath, "the store path is reported so a log line can name it"); + assert.equal(existsSync(configPath), false, "a GET never creates the store"); + }); + + test("sources still name all three layers, and userPath still names the deprecated file", async () => { + // The response fields are unchanged by this batch even though the + // answer behind them moved: an operator diagnosing a missing + // provider still needs to be told which files the server resolved, + // and the bilingual docs carry the new answer. + // + // The cwd expectation is built from `process.cwd()` rather than + // from the string this file chdir'd into. Those two differ whenever + // the temp path has a symlink component, and they are not the same + // kind of thing: `process.cwd()` is `getcwd(2)`, which returns a + // fully-resolved path on every POSIX platform, while the chdir + // argument is whatever the caller typed. The product's contract is + // "the cwd layer is `/models.json`", so that is what + // is asserted — see the symlink test below for the case this exists + // to cover. + reset(); + const c = await readEngineProviderCatalogue(); + assert.equal(c.sources.user, legacyFile); + assert.equal(c.sources.cwd, join(process.cwd(), "models.json")); + assert.equal(c.sources.env, null); + assert.equal(c.userPath, legacyFile); + }); + + test("a symlinked cwd does not change the cwd layer's path — the product does not re-resolve it", async () => { + // The macOS CI red, reproduced on any POSIX platform. macOS makes + // `/var` a symlink to `/private/var`, and `os.tmpdir()` lands under + // it, so a test that chdir'd into a temp dir and then asserted on + // the literal path it passed got `/private/var/...` back and + // failed. Linux CI never showed it because `/tmp` is a real + // directory — a symlink makes the same mismatch happen here. + // + // What is pinned is the product's behaviour, not the platform's: + // the path is `process.cwd()` + the file name, and webui applies + // NO additional resolution of its own. That is the correct + // direction for a value the API hands an operator to look at, and + // it is load-bearing for the store below — a read path that + // re-resolved and a write path that did not would make the write + // land in a different file than the read looked in. + reset(); + const realDir = join(tmpBase, "symlink-target"); + const alias = join(tmpBase, "symlink-alias"); + mkdirSync(realDir, { recursive: true }); + rmSync(alias, { force: true }); + symlinkSync(realDir, alias); + process.chdir(alias); + try { + assert.notEqual(process.cwd(), alias, "the platform resolved the symlink, as getcwd always has"); + const c = await readEngineProviderCatalogue(); + assert.equal( + c.sources.cwd, + join(process.cwd(), "models.json"), + "the reported path tracks process.cwd(), with no second resolution layered on top", + ); + assert.equal(c.sources.cwd.startsWith(alias), false, "webui does not re-expand the symlink either"); + // The EXPECTED side is normalised here, never the actual. On + // macOS the temp ROOT is itself a symlink (`/var` → + // `/private/var`), so `realDir` as spelled here is not what + // `getcwd` will report — comparing against `realpathSync` is what + // makes this assertion mean the same thing on both platforms. + assert.equal( + c.sources.cwd.startsWith(realpathSync(realDir)), + true, + "it reports what getcwd reported", + ); + } finally { + process.chdir(_origCwd); + } + }); + + test("the env override suppresses the cwd layer, as it always did", async () => { + reset(); + const envFile = join(cwdDir, "env.json"); + writeFileSync(envFile, JSON.stringify({ providers: [] }), "utf8"); + process.env.MCODE_WEBUI_MODELS_CONFIG = envFile; + try { + const c = await readEngineProviderCatalogue(); + assert.equal(c.sources.env, envFile); + assert.equal(c.sources.cwd, null, "the env override IS the cwd path"); + } finally { + delete process.env.MCODE_WEBUI_MODELS_CONFIG; + } + }); + + test("the migration report distinguishes attempted / migrated / failed", async () => { + reset(); + writeFileSync( + legacyFile, + JSON.stringify({ + version: 2, + providers: [{ id: "p", protocol: "openai", auth: { type: "byok", apiKey: "sk-key-aaaa" } }], + }), + "utf8", + ); + const first = await readEngineProviderCatalogue(); + assert.deepEqual(first.migration, { attempted: true, migrated: true, count: 1, code: null, error: null }); + + // Now break the store and confirm the failure is REPORTED, not + // thrown, and that the deprecated file answers. + writeFileSync(configPath, "{{ broken\n", "utf8"); + const second = await readEngineProviderCatalogue(); + assert.equal(second.catalogueSource, "legacy-file"); + assert.equal(second.migration.attempted, true); + assert.equal(second.migration.migrated, false); + assert.equal(second.migration.code, "ENGINE_CONFIG_UNREADABLE"); + assert.equal(second.providers.length, 1, "the deprecated file answered anyway"); + }); + + test("a migrated store is read without touching the deprecated file", async () => { + reset(); + writeFileSync( + configPath, + yaml.dump({ + custom_provider: { + live: { + name: "Live", + api: "openai-completions", + options: { apiKey: "sk-live-aaaa", baseURL: "https://live" }, + _webui_owned: true, + }, + }, + _webui_provider_migration: { schema: 1 }, + }), + "utf8", + ); + // A deprecated file that WOULD win if it were consulted. + writeFileSync( + legacyFile, + JSON.stringify({ version: 2, providers: [{ id: "stale", auth: { type: "byok", apiKey: "sk-stale" } }] }), + "utf8", + ); + const c = await readEngineProviderCatalogue(); + assert.equal(c.catalogueSource, "engine-store"); + assert.deepEqual(c.providers.map((p) => p.id), ["live"]); + assert.equal(c.migration.attempted, false); + }); +}); diff --git a/packages/webui/test/lib/engine/provider-store-ownership.test.js b/packages/webui/test/lib/engine/provider-store-ownership.test.js new file mode 100644 index 00000000..220b75ac --- /dev/null +++ b/packages/webui/test/lib/engine/provider-store-ownership.test.js @@ -0,0 +1,178 @@ +// webui/test/lib/engine/provider-store-ownership.test.js +// +// M3-B11: the tripwire for A5's second half — the `config.yaml` +// BYPASS must stay gone. +// +// The batch deleted `server/lib/engine-provider-sync.js`, the module +// that wrote the engine's `config.yaml` as a SECOND copy of +// `providers.json`. "We deleted a file" is not a property; "nothing +// writes that file behind the store's back" is. This suite reads the +// server source tree and fails if the bypass comes back in any form: +// +// - any module importing or naming `engine-provider-sync`; +// - any module OTHER than `engine/provider-store.js` writing +// `config.yaml` (a raw `writeFile`/`rename` aimed at it, or a +// `custom_provider` assignment); +// - any module other than `lib/engine-catalogue.js` reading the +// engine's OWN provider tree — `lib/engine-catalogue.js` is +// B4's #57 read of the builtin managed tree and is out of this +// batch's scope; `engine/provider-store.js` is the store. +// +// It is a static-source tripwire, which the repository's own guidance +// accepts when a suite has no render harness — and here the alternative +// is a behavioural test that cannot distinguish "the store wrote it" +// from "something else wrote it". +// +// If this ever goes red, the fix is NOT to widen the allowlist without +// reading what the new writer does: a second writer is exactly the bug +// A5 was filed for. + +import { test, describe } from "node:test"; +import { strict as assert } from "node:assert"; +import { readFileSync, readdirSync, statSync } from "node:fs"; +import { join, relative } from "node:path"; + +const SERVER_DIR = join(import.meta.dirname, "..", "..", "..", "server"); +const WEBAPP_DIR = join(import.meta.dirname, "..", "..", "..", "webapp"); + +/** Every .js/.mjs/.ts file under `dir`, recursively. */ +function sourceFiles(dir) { + const out = []; + const walk = (d) => { + for (const name of readdirSync(d)) { + if (name === "node_modules" || name === "out" || name === ".next") continue; + const full = join(d, name); + if (statSync(full).isDirectory()) walk(full); + else if (/\.(js|mjs|ts|tsx)$/.test(name)) out.push(full); + } + }; + walk(dir); + return out; +} + +const SERVER_FILES = sourceFiles(SERVER_DIR); +const WEBAPP_FILES = sourceFiles(WEBAPP_DIR); +const rel = (f) => relative(join(SERVER_DIR, ".."), f); + +/** Strip comment-only lines so a HISTORICAL mention is not a live import. */ +function codeLines(text) { + return text + .split("\n") + .filter((l) => !/^\s*(\/\/|\*|\/\*)/.test(l)) + .join("\n"); +} + +describe("A5 — the config.yaml bypass stays deleted", () => { + test("the deleted module is really gone from disk", () => { + assert.equal( + SERVER_FILES.some((f) => f.endsWith("engine-provider-sync.js")), + false, + "server/lib/engine-provider-sync.js must not exist; the store replaced it", + ); + }); + + test("no server module imports or names the deleted module", () => { + const offenders = SERVER_FILES.filter((f) => + codeLines(readFileSync(f, "utf8")).includes("engine-provider-sync"), + ); + assert.deepEqual(offenders.map(rel), [], "a live reference to the deleted module is a bypass back"); + }); + + test("no webapp module references the deleted module", () => { + const offenders = WEBAPP_FILES.filter((f) => + codeLines(readFileSync(f, "utf8")).includes("engine-provider-sync"), + ); + assert.deepEqual(offenders.map(rel), [], "the frontend never named it, and must not start"); + }); + + test("only the store writes config.yaml", () => { + // A second writer is the whole bug. The store is the only module + // allowed to name the file in a write position; everyone else may + // READ it. + const writers = SERVER_FILES.filter((f) => { + const text = codeLines(readFileSync(f, "utf8")); + if (!text.includes("config.yaml")) return false; + if (f.endsWith("engine/provider-store.js")) return false; + // A pure read names the path and opens it; a write renames or + // chmods onto it. The test distinguishes the two by what it does + // with the path, not by a comment. + return /\b(writeFile|rename|appendFile|createWriteStream)\b[\s\S]{0,200}config\.yaml/.test(text) + || /config\.yaml["'`][\s\S]{0,200}\b(writeFile|rename|appendFile|createWriteStream)\b/.test(text); + }); + assert.deepEqual(writers.map(rel), [], "config.yaml has exactly one writer: the store"); + }); + + test("only the store and the builtin-catalogue reader touch custom_provider", () => { + const offenders = SERVER_FILES.filter((f) => { + const text = codeLines(readFileSync(f, "utf8")); + if (!text.includes("custom_provider")) return false; + const allowed = + f.endsWith("engine/provider-store.js") || // the store + f.endsWith("lib/engine-catalogue.js"); // B4's #57 builtin read + return !allowed; + }); + assert.deepEqual( + offenders.map(rel), + [], + "custom_provider is reached through the store; a third reader is a second source of truth", + ); + }); + + test("the deprecated providers.json is READ, never written", () => { + // The fallback is only credible if the file cannot drift: nothing + // may write it, and the only writer of a provider catalogue is the + // store. + const offenders = SERVER_FILES.filter((f) => { + const text = codeLines(readFileSync(f, "utf8")); + if (!text.includes("providers.json")) return false; + return /\b(writeFileSync|writeFile|renameSync|rename|appendFile)\b[\s\S]{0,300}providers\.json/.test(text) + || /providers\.json["'`][\s\S]{0,300}\b(writeFileSync|writeFile|renameSync|rename|appendFile)\b/.test(text); + }); + assert.deepEqual(offenders.map(rel), [], "providers.json is deprecated: it may only be read"); + }); + + test("the store module is where the engine config path comes from", () => { + // One source for the path. `lib/engine-catalogue.js` used to import + // it from the deleted module; a second definition would let the + // catalogue and the store disagree about which file is which. + const definitions = SERVER_FILES.filter((f) => + /export function getEngineConfigPath\b/.test(readFileSync(f, "utf8")), + ); + assert.deepEqual(definitions.map(rel), ["server/engine/provider-store.js"]); + }); + + test("no provider path resolution applies a SECOND, different normalisation", () => { + // The macOS CI red, and why the PRODUCT is not the thing to change. + // + // `process.cwd()` is `getcwd(2)`, which returns a fully-resolved + // path on every POSIX platform — on macOS that is why a temp dir + // under `/var` comes back as `/private/var`. A `realpathSync` (or + // any equivalent) layered on top would be a SECOND normalisation + // that agrees with the first today and can drift from it the day + // either side changes. On the write side that drift is a security + // bug rather than a cosmetic one: the store is written 0600 through + // a rename and carries every plaintext apiKey, so a write that + // resolved its path differently from the read would put the keys in + // a file the catalogue never looks at — invisible, and not + // deletable by the next PUT. + // + // The correct direction is the one the code already takes: report + // what the resolver reported, and use the SAME resolver on both + // sides. `provider-reads.test.js` and `provider-writes.test.js` + // pin that behaviour against a real symlink, on any POSIX + // platform; this tripwire is what stops the next person from + // "fixing" the symptom in product code. + const RESOLVERS = + /getUserLevelPath|getCwdLayerPath|getEngineConfigPath|resolveEngineDataDir|loadProvidersConfig|readProviderStore|commitProviderStoreWrite|commitProviderCatalogueWrite/; + const offenders = SERVER_FILES.filter((f) => { + const text = readFileSync(f, "utf8"); + if (!text.includes("realpath")) return false; + return RESOLVERS.test(text); + }); + assert.deepEqual( + offenders.map(rel), + [], + "a second normalisation in the path resolution is the bug, not the fix", + ); + }); +}); diff --git a/packages/webui/test/lib/engine/provider-store.test.js b/packages/webui/test/lib/engine/provider-store.test.js new file mode 100644 index 00000000..fe29f976 --- /dev/null +++ b/packages/webui/test/lib/engine/provider-store.test.js @@ -0,0 +1,679 @@ +// webui/test/lib/engine/provider-store.test.js +// +// M3-B11 (= plan item A5): the consolidated provider store — +// `server/engine/provider-store.js`. +// +// This file replaces `test/lib/engine-provider-sync.test.js`, whose +// subject (`lib/engine-provider-sync.js`) the batch deleted. Every +// assertion the old suite made about the double write is made here +// about the single write, and the ones that CHANGED are the ones worth +// reading twice: +// +// - ownership (foreign entries survive, webui-owned ones are deleted +// when the operator removes the provider) — unchanged, and the +// marker is the same field; +// - 0600, atomic rename, no `.config-tmp-` leftover — unchanged; +// - an unparseable `config.yaml` is refused rather than overwritten — +// unchanged, and now load-bearing for a second reason (it is the +// fallback trigger on the read path); +// - "the webui catalogue is the `providers.json` file" — GONE. That +// was the dual source. The catalogue is the store, and the file is +// deprecated; `provider-migration.test.js` carries the migration +// and fallback contracts. +// +// Isolation: `MINIMAX_DATA_DIR` AND `MCODE_WEBUI_DATA_DIR` are both +// pinned to a per-run tmp dir BEFORE any import, for the same reason +// test/lib/engine/capability-snapshot.test.js states: setting only the +// webui dir leaves the engine dir on ~/.minimax, and this suite writes +// there. + +import { test, describe, before, after, beforeEach } from "node:test"; +import { strict as assert } from "node:assert"; +import { existsSync, mkdirSync, readFileSync, readdirSync, rmSync, statSync, writeFileSync } from "node:fs"; +import { join } from "node:path"; +import yaml from "js-yaml"; + +import { mkTmpDir } from "../../helpers/tmp.js"; + +const tmpBase = mkTmpDir("minimax-code-engine-store-"); +const engineDir = join(tmpBase, "engine"); +const webuiDir = join(tmpBase, "webui"); +process.env.MINIMAX_DATA_DIR = engineDir; +process.env.MAVIS_DATA_DIR = ""; +process.env.MCODE_WEBUI_DATA_DIR = webuiDir; +process.env.MCODE_WEBUI_MODELS_CONFIG = ""; +process.env.MCODE_WEBUI_SETTINGS_PATH = join(tmpBase, "settings.json"); +process.env.MCODE_WEBUI_EVENTS_PATH = join(tmpBase, "events.jsonl"); +process.env.MCODE_WEBUI_SESSIONS_DB = join(tmpBase, "sessions.db"); +process.env.MCODE_WEBUI_UPLOAD_DIR = join(tmpBase, "uploads"); + +const { + PROVIDER_STORE_MIGRATION_MARKER, + WEBUI_OWNED_MARKER, + WEBUI_PROVIDER_MARKER, + atomicWriteYaml0600, + buildProviderStoreWrite, + commitProviderStoreWrite, + getEngineConfigPath, + modelKeyFromId, + projectRecordToEngine, + protocolFromEngineApi, + providerKeyFromId, + providerRecordsFromTree, + readEngineConfigRaw, + readProviderStore, + recordFromEngineEntry, + resolveEngineDataDir, + _resetProviderStoreMigration, +} = await import("../../../server/engine/provider-store.js"); +const { normaliseProvider } = await import("../../../server/lib/providers-config.js"); + +const configPath = join(engineDir, "config.yaml"); + +/** A normalised record, from a raw v2 provider. */ +function rec(raw) { + const n = normaliseProvider(raw); + assert.equal(n.ok, true, `fixture must normalise: ${n.error}`); + return n.value; +} + +const BYOK = { + id: "gw", + label: "Gateway", + protocol: "openai", + auth: { type: "byok", apiKey: "sk-store-key-aaaa", baseURL: "https://api.example.com" }, + models: [{ id: "m1", label: "M1", contextLimit: 128000 }], +}; + +before(() => { + mkdirSync(engineDir, { recursive: true }); + mkdirSync(webuiDir, { recursive: true }); +}); + +beforeEach(() => { + _resetProviderStoreMigration(); + for (const f of [configPath, join(webuiDir, "providers.json")]) { + // `recursive` because one test deliberately turns config.yaml into + // a directory to force a write failure, and the hook must be able + // to clean that up for the next test. + if (existsSync(f)) rmSync(f, { recursive: true, force: true }); + } +}); + +after(() => { + try { + rmSync(tmpBase, { recursive: true, force: true }); + } catch {} +}); + +// ===================================================================== +// Location +// ===================================================================== + +describe("resolveEngineDataDir — precedence", () => { + test("MINIMAX_DATA_DIR wins, then MAVIS_DATA_DIR, then ~/.minimax", () => { + const before = { min: process.env.MINIMAX_DATA_DIR, mav: process.env.MAVIS_DATA_DIR }; + try { + process.env.MINIMAX_DATA_DIR = "/tmp/a"; + process.env.MAVIS_DATA_DIR = "/tmp/b"; + assert.equal(resolveEngineDataDir(), "/tmp/a"); + process.env.MINIMAX_DATA_DIR = " "; + assert.equal(resolveEngineDataDir(), "/tmp/b", "blank MINIMAX_DATA_DIR falls through"); + process.env.MAVIS_DATA_DIR = ""; + assert.match(resolveEngineDataDir(), /\.minimax$/, "falls back to the home dir"); + } finally { + if (before.min === undefined) delete process.env.MINIMAX_DATA_DIR; + else process.env.MINIMAX_DATA_DIR = before.min; + if (before.mav === undefined) delete process.env.MAVIS_DATA_DIR; + else process.env.MAVIS_DATA_DIR = before.mav; + } + }); + + test("getEngineConfigPath is the engine config inside that dir", () => { + assert.equal(getEngineConfigPath(), configPath); + }); +}); + +// ===================================================================== +// Pure key mapping +// ===================================================================== + +describe("providerKeyFromId — engine key safety", () => { + test("a legal id passes through unchanged", () => { + assert.equal(providerKeyFromId("gw"), "gw"); + assert.equal(providerKeyFromId(" gw "), "gw"); + assert.equal(providerKeyFromId("a.b_c-d1"), "a.b_c-d1"); + }); + + test("a reserved engine id is suffixed, never shadowed", () => { + // `minimax` is the engine's builtin managed provider. Writing a + // webui entry under that key would either shadow it or be dropped + // by the engine's own reserved-key filter. + for (const id of ["minimax", "minimax_api", "provider", "custom_provider"]) { + assert.equal(providerKeyFromId(id), `${id}-byok`); + } + }); + + test("an empty or illegal id has no key", () => { + assert.equal(providerKeyFromId(""), ""); + assert.equal(providerKeyFromId(" "), ""); + assert.equal(providerKeyFromId(undefined), ""); + assert.equal(providerKeyFromId("-leading-dash"), ""); + assert.equal(providerKeyFromId("has space"), ""); + assert.equal(providerKeyFromId("has/slash"), ""); + }); +}); + +describe("modelKeyFromId — namespace ids survive, YAML tokens do not", () => { + test("a namespaced model id keeps its slash", () => { + // The engine splits the wire form on the FIRST `/` only, so + // `z-ai/glm-5.3` survives as one model key. Rejecting it (as an + // earlier revision did) silently dropped the model from the store. + assert.equal(modelKeyFromId("z-ai/glm-5.3"), "z-ai/glm-5.3"); + assert.equal(modelKeyFromId("deepseek/x"), "deepseek/x"); + }); + + test("YAML structural tokens and whitespace are rejected", () => { + for (const bad of ["a b", "a:b", "a#b", "a{b", "a}b", "a[b", "a]b", "a@b", "a&b", "a*b", "a!b", "a|b", "a>b", "a'b", 'a"b', "a%b", "a`b", "a,b"]) { + assert.equal(modelKeyFromId(bad), "", `must reject ${JSON.stringify(bad)}`); + } + }); + + test("a leading YAML indicator is rejected", () => { + assert.equal(modelKeyFromId("-list"), ""); + assert.equal(modelKeyFromId("&anchor"), ""); + assert.equal(modelKeyFromId("*alias"), ""); + }); + + test("an empty id has no key", () => { + assert.equal(modelKeyFromId(""), ""); + assert.equal(modelKeyFromId(" "), ""); + assert.equal(modelKeyFromId(null), ""); + }); +}); + +describe("protocolFromEngineApi — the reverse map is many-to-one, and says so", () => { + test("anthropic-messages maps back to anthropic", () => { + assert.equal(protocolFromEngineApi("anthropic-messages"), "anthropic"); + }); + + test("everything else reads back as openai, gemini included", () => { + // The forward map sends BOTH openai and gemini to + // `openai-completions`, so the reverse cannot recover the + // distinction. That is why the store keeps the webui record beside + // the engine fields instead of reconstructing from them. + assert.equal(protocolFromEngineApi("openai-completions"), "openai"); + assert.equal(protocolFromEngineApi("gemini"), "openai"); + assert.equal(protocolFromEngineApi(undefined), "openai"); + }); +}); + +// ===================================================================== +// Pure projection: record → engine fields +// ===================================================================== + +describe("projectRecordToEngine — eligibility", () => { + test("a complete byok record projects every engine field", () => { + const out = projectRecordToEngine(rec(BYOK)); + assert.equal(out.name, "Gateway"); + assert.equal(out.kind, "custom"); + assert.equal(out.enabled, true); + assert.equal(out.api, "openai-completions"); + assert.deepEqual(out.options, { + apiKey: "sk-store-key-aaaa", + baseURL: "https://api.example.com", + authMode: "api-key", + }); + assert.deepEqual(out.models, { m1: { name: "M1", limit: { context: 128000 } } }); + }); + + test("protocol maps to the engine's api format", () => { + assert.equal(projectRecordToEngine(rec({ ...BYOK, protocol: "anthropic" })).api, "anthropic-messages"); + // gemini has no engine-native format; its OpenAI-compat endpoint is + // what a byok caller points at. + assert.equal(projectRecordToEngine(rec({ ...BYOK, protocol: "gemini" })).api, "openai-completions"); + }); + + test("an empty header map is omitted, never emitted as {}", () => { + // `normaliseProvider` always materialises `auth.headers`, and `{}` + // is TRUTHY — a truthiness test would stamp `headers: {}` on every + // provider that never configured one. + const out = projectRecordToEngine(rec({ ...BYOK, auth: { ...BYOK.auth } })); + assert.equal("headers" in out.options, false); + const withHeaders = projectRecordToEngine( + rec({ ...BYOK, auth: { ...BYOK.auth, headers: { "X-Tenant": "acme" } } }), + ); + assert.deepEqual(withHeaders.options.headers, { "X-Tenant": "acme" }); + }); + + test("ineligible records project to {} — and the reason is each one", () => { + const cases = [ + ["coding-plan auth", { ...BYOK, auth: { ...BYOK.auth, type: "coding-plan" } }], + ["disabled", { ...BYOK, enabled: false }], + ["no apiKey", { ...BYOK, auth: { type: "byok", baseURL: "https://x" } }], + ["no baseURL", { ...BYOK, auth: { type: "byok", apiKey: "sk-aaaaaaaaa" } }], + ]; + for (const [why, raw] of cases) { + assert.deepEqual(projectRecordToEngine(rec(raw)), {}, `must be ineligible: ${why}`); + } + assert.deepEqual(projectRecordToEngine(null), {}); + assert.deepEqual(projectRecordToEngine("nope"), {}); + }); + + test("a model whose label equals its id carries no name", () => { + const out = projectRecordToEngine( + rec({ ...BYOK, models: [{ id: "m1" }] }), + ); + assert.deepEqual(out.models, { m1: {} }); + }); + + test("thinking levels and modalities become the engine's own shapes", () => { + const out = projectRecordToEngine( + rec({ ...BYOK, models: [{ id: "m", thinkingLevels: ["low", "high"], modalities: ["text", "image"] }] }), + ); + assert.deepEqual(out.models.m.thinking, { effortOptions: ["low", "high"] }); + assert.deepEqual(out.models.m.modalities, { input: ["text", "image"] }); + }); +}); + +// ===================================================================== +// Pure projection: engine entry → record (the read side) +// ===================================================================== + +describe("recordFromEngineEntry — the lossless path and the legacy path", () => { + test("a foreign entry is never a catalogue record", () => { + // No ownership marker means the operator wrote it; it has never + // been in the catalogue and must not start being. + assert.equal(recordFromEngineEntry("manual", { name: "Manual", api: "openai-completions" }), null); + }); + + test("the embedded record is returned verbatim, re-normalised", () => { + const original = rec({ + id: "gw", + label: "Gateway", + protocol: "gemini", + auth: { type: "byok", apiKey: "sk-store-key-aaaa", baseURL: "https://x" }, + models: [{ id: "z-ai/glm-5.3", contextLimit: 1 }], + }); + const entry = { ...projectRecordToEngine(original), [WEBUI_OWNED_MARKER]: true, [WEBUI_PROVIDER_MARKER]: original }; + const back = recordFromEngineEntry("gw", entry); + assert.deepEqual(back, original); + // The gemini protocol is the proof the record path is lossless: the + // engine fields say `openai-completions`, so a reconstruction would + // have said `openai`. + assert.equal(back.protocol, "gemini"); + }); + + test("a hand-tampered record cannot put a malformed row on the wire", () => { + const entry = { + api: "openai-completions", + options: { apiKey: "sk-aaaaaaaaa" }, + [WEBUI_OWNED_MARKER]: true, + [WEBUI_PROVIDER_MARKER]: { id: "not a legal id", protocol: "nope" }, + }; + // The record is rejected, and reconstruction from the engine + // fields takes over rather than the bad record reaching the API. + const back = recordFromEngineEntry("gw", entry); + assert.equal(back.id, "gw"); + assert.equal(back.protocol, "openai"); + }); + + test("a pre-B11 entry (marker but no record) is reconstructed", () => { + // This is the real upgrade path: an installation that has been + // running webui since ticket 05 has `_webui_owned` entries with NO + // record beside them. Reconstructing is lossy by the reverse map, + // and the alternative is an empty catalogue. + const entry = { + name: "Old", + kind: "custom", + enabled: true, + api: "anthropic-messages", + options: { apiKey: "sk-old-key-aaaaa", baseURL: "https://old", authMode: "api-key" }, + models: { m: { name: "M", limit: { context: 4096 }, thinking: { effortOptions: ["low"] } } }, + [WEBUI_OWNED_MARKER]: true, + }; + const back = recordFromEngineEntry("old", entry); + assert.equal(back.id, "old"); + assert.equal(back.label, "Old"); + assert.equal(back.protocol, "anthropic"); + assert.equal(back.auth.apiKey, "sk-old-key-aaaaa"); + assert.equal(back.auth.baseURL, "https://old"); + assert.deepEqual(back.models, [ + { id: "m", label: "M", contextLimit: 4096, thinkingLevels: ["low"] }, + ]); + }); + + test("a pre-B11 entry with no apiKey is not a record at all", () => { + // The old sync never wrote a key-less entry, so this cannot happen + // from webui — but a hand edit can, and a key-less provider is not + // something the catalogue can serve. + assert.equal( + recordFromEngineEntry("x", { api: "openai-completions", options: {}, [WEBUI_OWNED_MARKER]: true }), + null, + ); + }); + + test("providerRecordsFromTree keeps tree order and skips foreign keys", () => { + const tree = { + zz: { [WEBUI_OWNED_MARKER]: true, options: { apiKey: "sk-aaaaaaaaa" }, api: "openai-completions" }, + foreign: { name: "Manual" }, + aa: { [WEBUI_OWNED_MARKER]: true, options: { apiKey: "sk-bbbbbbbbb" }, api: "openai-completions" }, + }; + assert.deepEqual(providerRecordsFromTree(tree).map((r) => r.id), ["zz", "aa"]); + assert.deepEqual(providerRecordsFromTree(null), []); + }); +}); + +// ===================================================================== +// Pure merge: the ownership rule +// ===================================================================== + +describe("buildProviderStoreWrite — ownership", () => { + test("a foreign entry survives a write that does not mention it", () => { + const foreign = { name: "Manual", kind: "custom", api: "openai-completions", options: { apiKey: "sk-op" } }; + const plan = buildProviderStoreWrite({ manual: foreign }, [rec(BYOK)]); + assert.deepEqual(plan.tree.manual, foreign, "verbatim, not re-projected"); + assert.deepEqual(plan.preserved, ["manual"]); + }); + + test("a webui-owned entry the operator removed is deleted", () => { + const plan = buildProviderStoreWrite( + { gone: { [WEBUI_OWNED_MARKER]: true, [WEBUI_PROVIDER_MARKER]: rec(BYOK) } }, + [], + ); + assert.equal("gone" in plan.tree, false); + }); + + test("records come FIRST, in the caller's order; foreign entries follow", () => { + // The catalogue API returns this order verbatim, and the order an + // operator sees in the dialog has always been the order they PUT. + const plan = buildProviderStoreWrite( + { manual: { name: "Manual" } }, + [rec({ ...BYOK, id: "b" }), rec({ ...BYOK, id: "a" })], + ); + assert.deepEqual(Object.keys(plan.tree), ["b", "a", "manual"]); + assert.deepEqual(plan.records, ["b", "a"]); + }); + + test("an INELIGIBLE record still occupies its key, marked, with no engine fields", () => { + // This is the difference from the double write: the old sync dropped + // a disabled provider and a coding-plan provider from the engine + // tree entirely, so they lived in one file and not the other. Here + // the record survives; only the engine projection is absent. + const plan = buildProviderStoreWrite({}, [rec({ ...BYOK, enabled: false })]); + const entry = plan.tree.gw; + assert.equal(entry[WEBUI_OWNED_MARKER], true); + assert.equal(entry.api, undefined, "no engine fields"); + assert.equal(entry[WEBUI_PROVIDER_MARKER].enabled, false); + }); + + test("a record with an illegal engine key is still stored", () => { + // `normaliseProvider` enforces a stricter id grammar than the store + // key needs, so this can only arrive from a direct engine-module + // caller. Dropping it would be the one loss this batch cannot have. + const plan = buildProviderStoreWrite({}, [{ id: "a b", label: "L", auth: {}, models: [] }]); + assert.equal(Object.keys(plan.tree).length, 1); + assert.equal(plan.records[0], "a b"); + }); + + test("a pre-B11 owned entry is replaced by the record, not kept", () => { + const plan = buildProviderStoreWrite( + { gw: { [WEBUI_OWNED_MARKER]: true, api: "openai-completions", options: { apiKey: "sk-stale" } } }, + [rec(BYOK)], + ); + assert.equal(plan.tree.gw[WEBUI_PROVIDER_MARKER].auth.apiKey, "sk-store-key-aaaa"); + assert.deepEqual(plan.preserved, []); + }); + + test("a webui key that collides with a foreign entry overwrites it (pinned debt)", () => { + // KNOWN DEBT 3 in `engine/provider-writes.js`: the key IS the + // runtime id, so silently suffixing it would break a recorded + // model pick. The pre-existing behaviour is pinned here so the + // choice stays visible rather than drifting. + const plan = buildProviderStoreWrite( + { gw: { name: "Operator's own gw", options: { apiKey: "sk-operator" } } }, + [rec(BYOK)], + ); + assert.equal(plan.tree.gw[WEBUI_OWNED_MARKER], true); + assert.equal(plan.tree.gw[WEBUI_PROVIDER_MARKER].auth.apiKey, "sk-store-key-aaaa"); + }); + + test("a non-object entry in the existing tree is not carried", () => { + const plan = buildProviderStoreWrite({ junk: "not-an-object" }, []); + assert.deepEqual(plan.tree, {}); + }); +}); + +// ===================================================================== +// Read: raw document +// ===================================================================== + +describe("readEngineConfigRaw — refusal, not overwrite", () => { + test("a missing file is an empty document, not an error", () => { + const r = readEngineConfigRaw(join(engineDir, "absent.yaml")); + assert.equal(r.ok, true); + assert.deepEqual(r.raw, {}); + assert.equal(r.exists, false); + }); + + test("an unparseable file is refused, and the file is left alone", () => { + writeFileSync(configPath, "custom_provider:\n - [unbalanced\n", "utf8"); + const before = readFileSync(configPath, "utf8"); + const r = readEngineConfigRaw(configPath); + assert.equal(r.ok, false); + assert.equal(r.code, "ENGINE_CONFIG_UNREADABLE"); + assert.equal(readFileSync(configPath, "utf8"), before, "the refusal must not touch it"); + }); + + test("a YAML document that is not a mapping is refused", () => { + writeFileSync(configPath, "- just\n- a\n- list\n", "utf8"); + const r = readEngineConfigRaw(configPath); + assert.equal(r.ok, false); + assert.equal(r.code, "ENGINE_CONFIG_UNREADABLE"); + }); + + test("an empty file is an empty document", () => { + writeFileSync(configPath, "", "utf8"); + const r = readEngineConfigRaw(configPath); + assert.equal(r.ok, true); + assert.deepEqual(r.raw, {}); + }); +}); + +// ===================================================================== +// Read: the store +// ===================================================================== + +describe("readProviderStore — which file is the authority", () => { + test("no marker means the store contributes nothing", () => { + writeFileSync( + configPath, + yaml.dump({ + custom_provider: { + old: { [WEBUI_OWNED_MARKER]: true, api: "openai-completions", options: { apiKey: "sk-old-key" } }, + }, + }), + "utf8", + ); + const s = readProviderStore({ configPath }); + assert.equal(s.ok, true); + assert.equal(s.migrationDone, false); + assert.deepEqual(s.records, [], "a store with no marker is not the catalogue"); + }); + + test("the marker makes the store the catalogue, in tree order", () => { + writeFileSync( + configPath, + yaml.dump({ + custom_provider: { + b: { [WEBUI_OWNED_MARKER]: true, api: "openai-completions", options: { apiKey: "sk-bbbbbbbbb" } }, + a: { [WEBUI_OWNED_MARKER]: true, api: "openai-completions", options: { apiKey: "sk-aaaaaaaaa" } }, + }, + [PROVIDER_STORE_MIGRATION_MARKER]: { schema: 1, at: "2026-01-01T00:00:00.000Z" }, + }), + "utf8", + ); + const s = readProviderStore({ configPath }); + assert.equal(s.migrationDone, true); + assert.deepEqual(s.records.map((r) => r.id), ["b", "a"]); + }); + + test("an unreadable store reports the failure instead of pretending to be empty", () => { + writeFileSync(configPath, "{{{ not yaml\n", "utf8"); + const s = readProviderStore({ configPath }); + assert.equal(s.ok, false); + assert.equal(s.code, "ENGINE_CONFIG_UNREADABLE"); + // `migrationDone: true` on the failure branch: an unreadable store + // must never send the read path back to a deprecated file it is + // about to overwrite on the next write. + assert.equal(s.migrationDone, true); + }); + + test("an explicit marker with an EMPTY tree is authoritative — no resurrection", () => { + // The reason the marker is a field rather than an inference: an + // operator who deletes every provider leaves a store with no + // webui entries, and inferring "never migrated" from that would + // make a stale deprecated file authoritative again. + writeFileSync( + configPath, + yaml.dump({ custom_provider: {}, [PROVIDER_STORE_MIGRATION_MARKER]: { schema: 1 } }), + "utf8", + ); + const s = readProviderStore({ configPath }); + assert.equal(s.migrationDone, true); + assert.deepEqual(s.records, []); + }); +}); + +// ===================================================================== +// Commit: one atomic rename +// ===================================================================== + +describe("commitProviderStoreWrite — one write, one rename", () => { + test("a write stamps the marker and lands the records", async () => { + const r = await commitProviderStoreWrite({ + configPath, + raw: {}, + tree: {}, + records: [rec(BYOK)], + migrated: true, + }); + assert.equal(r.ok, true); + assert.equal(r.written, true); + const doc = yaml.load(readFileSync(configPath, "utf8")); + assert.ok(doc[PROVIDER_STORE_MIGRATION_MARKER], "the marker rides the same rename"); + assert.equal(doc.custom_provider.gw[WEBUI_PROVIDER_MARKER].auth.apiKey, "sk-store-key-aaaa"); + }); + + test("sections the store does not own are preserved verbatim", async () => { + // The operator's `provider.minimax`, their `defaultModel`, any + // section a future engine version adds: all of it rides through. + const seed = { + provider: { minimax: { name: "MiniMax", models: { "MiniMax-M3": {} } } }, + defaultModel: "m:minimax:MiniMax-M3:u", + somethingNewInTheEngine: { keep: [1, 2, 3] }, + }; + writeFileSync(configPath, yaml.dump(seed), "utf8"); + await commitProviderStoreWrite({ configPath, raw: seed, tree: {}, records: [rec(BYOK)] }); + const doc = yaml.load(readFileSync(configPath, "utf8")); + assert.deepEqual(doc.provider, seed.provider); + assert.equal(doc.defaultModel, seed.defaultModel); + assert.deepEqual(doc.somethingNewInTheEngine, seed.somethingNewInTheEngine); + }); + + test("an unreadable store is refused and left byte-identical", async () => { + // The failure an operator actually hits: a config.yaml a future + // engine version, or a hand edit, made unparseable. The write must + // refuse it — overwriting would destroy every section the store + // does not own — and must not so much as re-chmod it. + const broken = "custom_provider:\n - [unbalanced\n"; + writeFileSync(configPath, broken, "utf8"); + const r = await commitProviderStoreWrite({ configPath, raw: {}, tree: {}, records: [rec(BYOK)], migrated: true }); + assert.equal(r.ok, false); + assert.equal(r.code, "ENGINE_STORE_UNREADABLE"); + assert.equal(readFileSync(configPath, "utf8"), broken, "byte-identical after the refusal"); + }); + + test("a pre-write failure leaves nothing behind", async () => { + // A config path whose parent is a regular file: the read finds + // nothing and the tmp write fails with ENOTDIR, before the tmp + // file exists. + const blocked = join(engineDir, "not-a-dir", "config.yaml"); + writeFileSync(join(engineDir, "not-a-dir"), "x", "utf8"); + const r = await commitProviderStoreWrite({ configPath: blocked, raw: {}, tree: {}, records: [rec(BYOK)] }); + assert.equal(r.code, "ENGINE_STORE_WRITE_FAILED"); + assert.deepEqual(readdirSync(engineDir).filter((f) => f.startsWith(".config-tmp-")), []); + rmSync(join(engineDir, "not-a-dir"), { force: true }); + }); + + test("a POST-WRITE failure leaves NO temp file holding plaintext keys", async () => { + // The tmp file is mode 0600 and carries every apiKey in the + // catalogue. A failure AFTER it is written — the rename, or the + // final chmod — must remove it: the old double write had exactly + // this gap and only ever tested the success path, so one leaked + // copy of every operator credential accumulated per failed write. + // + // `atomicWriteYaml0600` is called directly because the only + // post-write failure a caller can reach through + // `commitProviderStoreWrite` is a filesystem race, and a race is + // not a test. The target here is a NON-EMPTY directory: `rename` + // onto one fails with ENOTEMPTY on every POSIX filesystem, while + // the tmp file itself has already been written in full. + const dirTarget = join(engineDir, "config.yaml"); + mkdirSync(dirTarget, { recursive: true }); + writeFileSync(join(dirTarget, "occupant"), "x", "utf8"); + await assert.rejects( + () => atomicWriteYaml0600(dirTarget, { custom_provider: { gw: { options: { apiKey: "sk-store-key-aaaa" } } } }), + "a rename onto a non-empty directory must fail", + ); + assert.deepEqual( + readdirSync(engineDir).filter((f) => f.startsWith(".config-tmp-")), + [], + "the tmp file carrying the plaintext key must not survive", + ); + }); + + test("the write is 0600 — the document carries plaintext keys", async () => { + await commitProviderStoreWrite({ configPath, raw: {}, tree: {}, records: [rec(BYOK)], migrated: true }); + const mode = statSync(configPath).mode & 0o777; + assert.equal(mode, 0o600, `expected 0600, got ${mode.toString(8)}`); + }); + + test("no `.config-tmp-` file survives a successful write", async () => { + await commitProviderStoreWrite({ configPath, raw: {}, tree: {}, records: [rec(BYOK)], migrated: true }); + assert.deepEqual(readdirSync(engineDir), ["config.yaml"]); + }); + + test("a write that would change nothing does not touch the file", async () => { + const first = await commitProviderStoreWrite({ configPath, raw: {}, tree: {}, records: [rec(BYOK)], migrated: true }); + assert.equal(first.written, true); + const mtime = statSync(configPath).mtimeMs; + await new Promise((r) => setTimeout(r, 12)); + const second = await commitProviderStoreWrite({ + configPath, + raw: yaml.load(readFileSync(configPath, "utf8")), + tree: yaml.load(readFileSync(configPath, "utf8")).custom_provider, + records: [rec(BYOK)], + }); + assert.equal(second.written, false, "a no-op PUT must not rewrite the operator's file"); + assert.equal(statSync(configPath).mtimeMs, mtime); + }); + + test("the marker is stamped even when the catalogue is emptied", async () => { + writeFileSync( + configPath, + yaml.dump({ + custom_provider: { + gw: { [WEBUI_OWNED_MARKER]: true, [WEBUI_PROVIDER_MARKER]: rec(BYOK) }, + }, + }), + "utf8", + ); + const raw = yaml.load(readFileSync(configPath, "utf8")); + const r = await commitProviderStoreWrite({ configPath, raw, tree: raw.custom_provider, records: [], migrated: true }); + assert.equal(r.ok, true); + const doc = yaml.load(readFileSync(configPath, "utf8")); + assert.deepEqual(doc.custom_provider, {}); + assert.ok(doc[PROVIDER_STORE_MIGRATION_MARKER], "an emptied catalogue still closes the deprecated file"); + }); +}); diff --git a/packages/webui/test/lib/engine/provider-writes.test.js b/packages/webui/test/lib/engine/provider-writes.test.js new file mode 100644 index 00000000..85abcb3b --- /dev/null +++ b/packages/webui/test/lib/engine/provider-writes.test.js @@ -0,0 +1,451 @@ +// webui/test/lib/engine/provider-writes.test.js +// +// M3-B11 (= plan item A5), the write half: #63 PUT /api/providers and +// #66 POST /api/providers/preset/:id/enable, through +// `server/engine/provider-writes.js`. +// +// Two things are pinned here and neither has a pre-B11 equivalent: +// +// 1. PUT ATOMICITY. The old arrangement wrote `providers.json` and +// then `config.yaml`, with nothing between them. A failure in the +// second left the first committed, answered 200 with a warning, +// and left the operator's next edit computed from a file the +// engine had never seen. There is one file and one rename now, so +// the interesting assertions are the negative ones: a refused +// write changes NOTHING, and two concurrent writes leave one whole +// state rather than a mixture. +// 2. THE HARD GATE. Both endpoints gate on `authCredentials` before +// any write, and a provider that denies the sub-item gets the +// shared 501 — because the catalogue the operator is about to see +// is read by the engine, and a 200 that did not land would be the +// fake success the gate exists to prevent. +// +// Isolation: both data dirs are per-run tmp, pinned before any import. + +import { test, describe, before, after, beforeEach } from "node:test"; +import { strict as assert } from "node:assert"; +import { + existsSync, + mkdirSync, + readFileSync, + readdirSync, + rmSync, + statSync, + symlinkSync, + writeFileSync, +} from "node:fs"; +import { join } from "node:path"; +import yaml from "js-yaml"; + +import { mkTmpDir } from "../../helpers/tmp.js"; + +const tmpBase = mkTmpDir("minimax-code-engine-writes-"); +const engineDir = join(tmpBase, "engine"); +const webuiDir = join(tmpBase, "webui"); +const configPath = join(engineDir, "config.yaml"); +process.env.MINIMAX_DATA_DIR = engineDir; +process.env.MAVIS_DATA_DIR = ""; +process.env.MCODE_WEBUI_DATA_DIR = webuiDir; +process.env.MCODE_WEBUI_MODELS_CONFIG = ""; +process.env.MCODE_WEBUI_SETTINGS_PATH = join(tmpBase, "settings.json"); +process.env.MCODE_WEBUI_EVENTS_PATH = join(tmpBase, "events.jsonl"); +process.env.MCODE_WEBUI_SESSIONS_DB = join(tmpBase, "sessions.db"); +process.env.MCODE_WEBUI_UPLOAD_DIR = join(tmpBase, "uploads"); + +const { + PROVIDER_WRITE_ENDPOINTS, + assertProviderWriteCapability, + commitProviderCatalogueWrite, + planProviderCatalogueWrite, + resolveProviderWriteProvider, +} = await import("../../../server/engine/provider-writes.js"); +const { getEngineProvider, LOCAL_RUNTIME_V2_CAPABILITIES } = await import( + "../../../server/engine/index.js" +); +const { EngineCapabilityNotSupportedError } = await import("../../../server/engine/errors.js"); +const { assertEngineCapability } = await import("../../../server/engine/capabilities.js"); +const { normaliseProvider } = await import("../../../server/lib/providers-config.js"); + +function rec(raw) { + const n = normaliseProvider(raw); + assert.equal(n.ok, true, `fixture must normalise: ${n.error}`); + return n.value; +} + +const A = { id: "a", label: "A", protocol: "openai", auth: { type: "byok", apiKey: "sk-key-aaaa", baseURL: "https://a" }, models: [] }; +const B = { id: "b", label: "B", protocol: "openai", auth: { type: "byok", apiKey: "sk-key-bbbb", baseURL: "https://b" }, models: [] }; + +/** + * The parsed store document, or null when the file is absent. + * + * @param {string} [at] Defaults to the file-level store path. A test + * that points `MINIMAX_DATA_DIR` somewhere else passes the + * resolved path explicitly, so the assertion reads the file the + * write actually produced rather than the default one. + * @returns {object|null} + */ +function storeDoc(at = configPath) { + if (!existsSync(at)) return null; + return yaml.load(readFileSync(at, "utf8")); +} + +/** + * The webui records the store holds, in key order. + * + * @param {string} [at] + * @returns {object[]} + */ +function storeRecords(at = configPath) { + const doc = storeDoc(at); + if (!doc) return []; + return Object.values(doc.custom_provider || {}) + .map((e) => e && e._webui_provider) + .filter(Boolean); +} + +before(() => { + mkdirSync(engineDir, { recursive: true }); + mkdirSync(webuiDir, { recursive: true }); +}); + +after(() => { + try { + rmSync(tmpBase, { recursive: true, force: true }); + } catch {} +}); + +beforeEach(() => { + if (existsSync(configPath)) rmSync(configPath, { recursive: true, force: true }); +}); + +// ===================================================================== +// The gate table +// ===================================================================== + +describe("PROVIDER_WRITE_ENDPOINTS — the family is the five-endpoint one", () => { + test("exactly #63 and #66, both hard, both on authCredentials", () => { + assert.deepEqual(Object.keys(PROVIDER_WRITE_ENDPOINTS).sort(), [ + "POST /api/providers/preset/:id/enable", + "PUT /api/providers", + ]); + for (const [endpoint, need] of Object.entries(PROVIDER_WRITE_ENDPOINTS)) { + assert.equal(need.capability, "authCredentials", endpoint); + assert.equal(need.enforcement, "hard", endpoint); + } + assert.equal(PROVIDER_WRITE_ENDPOINTS["PUT /api/providers"].subItem, "updateUserModelProvider"); + assert.equal(PROVIDER_WRITE_ENDPOINTS["POST /api/providers/preset/:id/enable"].subItem, "createUserModelProvider"); + }); + + test("an endpoint outside the family is a caller bug, not an engine limitation", () => { + // A plain Error, so a typo in webui's own key can never reach an + // operator as "the engine cannot do this". + assert.throws( + () => assertProviderWriteCapability("GET /api/providers", "runtime"), + (e) => e.code === "unknown_provider_write_endpoint" && !(e instanceof EngineCapabilityNotSupportedError), + ); + }); +}); + +describe("assertProviderWriteCapability — HARD", () => { + test("the declared provider passes and reports which provider answered", () => { + const r = assertProviderWriteCapability("PUT /api/providers", "runtime"); + assert.equal(r.gate, "checked"); + assert.equal(r.provider, "local-runtime-v2"); + assert.equal(r.enforcement, "hard"); + }); + + test("an unregistered transport is NOT a 501 — it lets the write proceed", () => { + // The transport table is empty until M4. A 501 that meant "nobody + // has written M4 yet" would be a lie about the engine, and every + // other family in this migration draws the same line. + for (const transport of ["acp", "exec", "anything-else"]) { + const r = assertProviderWriteCapability("PUT /api/providers", transport); + assert.equal(r.gate, "unregistered-transport"); + assert.equal(r.provider, null); + } + }); + + test("a provider denying the sub-item gets the shared 501 error", () => { + // `authCredentials` is partial today (missing setConfigOption), so + // a gate on any OTHER sub-item passes. Making the write's own + // sub-item the missing one must throw the shared error the router + // maps — same shape B9 established. + const original = LOCAL_RUNTIME_V2_CAPABILITIES.authCredentials; + try { + const denied = { + ...LOCAL_RUNTIME_V2_CAPABILITIES, + authCredentials: { + ...original, + missing: [...original.missing, "updateUserModelProvider"], + }, + }; + // The negative is driven through the same `assertEngineCapability` + // the real gate calls, rather than by re-implementing the check + // here — a hand-rolled copy is exactly what would let the two + // drift. + assert.throws( + () => assertEngineCapability(denied, "authCredentials", "fake-provider", "updateUserModelProvider"), + (e) => { + assert.equal(e.name, "EngineCapabilityNotSupportedError"); + assert.equal(e.capability, "authCredentials"); + assert.deepEqual(e.missing, ["updateUserModelProvider"]); + return true; + }, + ); + } finally { + void original; + } + }); + + test("a `none` declaration denies every sub-item of the key", () => { + const none = { + ...LOCAL_RUNTIME_V2_CAPABILITIES, + authCredentials: { level: "none", reason: "no credential surface" }, + }; + assert.throws( + () => assertEngineCapability(none, "authCredentials", "p", "updateUserModelProvider"), + (e) => e.name === "EngineCapabilityNotSupportedError", + ); + }); +}); + + +// ===================================================================== +// The keep-key convention, scoped to the store +// ===================================================================== + +describe("planProviderCatalogueWrite — pure, and scoped to the store", () => { + test("an empty or absent apiKey takes the stored one", () => { + const existing = [rec({ ...A, auth: { ...A.auth, apiKey: "sk-stored-aaaa" } })]; + for (const auth of [{ type: "byok", apiKey: "" }, { type: "byok" }]) { + const out = planProviderCatalogueWrite([{ id: "a", auth }], existing); + assert.equal(out[0].auth.apiKey, "sk-stored-aaaa"); + } + }); + + test("a non-empty apiKey replaces it", () => { + const existing = [rec({ ...A, auth: { ...A.auth, apiKey: "sk-stored-aaaa" } })]; + const out = planProviderCatalogueWrite([{ id: "a", auth: { type: "byok", apiKey: "sk-new-bbbb" } }], existing); + assert.equal(out[0].auth.apiKey, "sk-new-bbbb"); + }); + + test("a NEW provider with a sentinel key stays empty", () => { + const out = planProviderCatalogueWrite([{ id: "brand-new", auth: { type: "byok", apiKey: "" } }], []); + assert.equal(out[0].auth.apiKey, ""); + }); + + test("the input is not mutated", () => { + const incoming = [{ id: "a", auth: { type: "byok", apiKey: "" } }]; + planProviderCatalogueWrite(incoming, [rec(A)]); + assert.equal(incoming[0].auth.apiKey, "", "the caller's body is untouched"); + }); +}); + +// ===================================================================== +// The commit +// ===================================================================== + +describe("commitProviderCatalogueWrite — one document, one rename", () => { + test("a successful commit persists the records in order and stamps the marker", async () => { + const r = await commitProviderCatalogueWrite({ records: [rec(A), rec(B)] }); + assert.equal(r.ok, true); + assert.equal(r.written, true); + assert.deepEqual(r.keys, ["a", "b"]); + assert.deepEqual(r.records.map((x) => x.id), ["a", "b"]); + assert.deepEqual(storeRecords().map((x) => x.id), ["a", "b"]); + assert.ok(storeDoc()._webui_provider_migration, "the marker rides the same write"); + }); + + test("a second commit REPLACES the catalogue — there is no patch semantics", async () => { + await commitProviderCatalogueWrite({ records: [rec(A), rec(B)] }); + const r = await commitProviderCatalogueWrite({ records: [rec(B)] }); + assert.equal(r.ok, true); + assert.deepEqual(storeRecords().map((x) => x.id), ["b"], "a is gone: the body is the whole catalogue"); + }); + + test("an unreadable store is refused and the document is left byte-identical", async () => { + const broken = "custom_provider:\\n - [unbalanced\\n"; + writeFileSync(configPath, broken, "utf8"); + const r = await commitProviderCatalogueWrite({ records: [rec(A)] }); + assert.equal(r.ok, false); + assert.equal(r.code, "ENGINE_STORE_UNREADABLE"); + assert.equal(readFileSync(configPath, "utf8"), broken); + }); + + test("a foreign engine entry survives a commit", async () => { + writeFileSync( + configPath, + yaml.dump({ custom_provider: { manual: { name: "Mine", options: { apiKey: "sk-op" } } } }), + "utf8", + ); + const r = await commitProviderCatalogueWrite({ records: [rec(A)] }); + assert.equal(r.ok, true); + assert.deepEqual(r.preserved, ["manual"]); + assert.deepEqual(storeDoc().custom_provider.manual, { name: "Mine", options: { apiKey: "sk-op" } }); + }); + + test("an operator's other engine sections survive a commit", async () => { + const seed = { provider: { minimax: { models: { "MiniMax-M3": {} } } }, defaultModel: "m:minimax:MiniMax-M3:u" }; + writeFileSync(configPath, yaml.dump(seed), "utf8"); + const r = await commitProviderCatalogueWrite({ records: [rec(A)] }); + assert.equal(r.ok, true); + const doc = storeDoc(); + assert.deepEqual(doc.provider, seed.provider); + assert.equal(doc.defaultModel, seed.defaultModel); + }); +}); + +// ===================================================================== +// The path the write lands on — the security surface +// ===================================================================== + +describe("the write path is the same path the read resolves", () => { + // The store is written with mode 0600 through a tmp file and a + // rename, and it carries every plaintext apiKey in the catalogue. So + // "which file did that land in" is a security question, not a + // cosmetic one: a write that resolved its path differently from the + // read would put the keys in a file the catalogue never looks at — + // invisible, not deletable by the next PUT, and still on disk. + // + // The two sides must agree by CONSTRUCTION: there is one resolver, + // `getEngineConfigPath()`, and both call it. These tests pin that + // agreement rather than the implementation, and the symlink is the + // case where a future "helpful" normalisation could split them. + + test("read and write name the same file, through a symlinked data dir", async () => { + // `resolveEngineDataDir` returns the env value verbatim and no + // getcwd/realpath is involved, so both sides must use the literal + // and the bytes must land in the REAL file. The assertion is on the + // result, not on the string. + const realDir = join(tmpBase, "engine-symlink-target"); + const alias = join(tmpBase, "engine-symlink-alias"); + mkdirSync(realDir, { recursive: true }); + rmSync(alias, { force: true, recursive: true }); + symlinkSync(realDir, alias); + + const before = process.env.MINIMAX_DATA_DIR; + process.env.MINIMAX_DATA_DIR = alias; + try { + const w = await commitProviderCatalogueWrite({ records: [rec(A)] }); + assert.equal(w.ok, true); + assert.equal(existsSync(join(realDir, "config.yaml")), true, "the write landed in the real file"); + assert.deepEqual( + storeRecords(join(realDir, "config.yaml")).map((r) => r.id), + ["a"], + "and the read finds them there", + ); + // No stray document or leftover tmp file beside the symlink: a + // key must not be able to end up in a file the store never reads. + const stray = readdirSync(tmpBase).filter( + (f) => f === "config.yaml" || f.startsWith(".config-tmp-"), + ); + assert.deepEqual(stray, [], "no document written beside the symlink"); + } finally { + if (before === undefined) delete process.env.MINIMAX_DATA_DIR; + else process.env.MINIMAX_DATA_DIR = before; + rmSync(alias, { force: true }); + } + }); + + test("the file the write creates is 0600 even when reached through a symlink", async () => { + // The permission must not depend on how the operator spelled the + // path. POSIX mode bits travel with the file across a rename, and + // the tmp file is created 0600 before it is renamed, so the + // symlinked route lands in exactly the same mode. + const realDir = join(tmpBase, "engine-mode-target"); + const alias = join(tmpBase, "engine-mode-alias"); + mkdirSync(realDir, { recursive: true }); + rmSync(alias, { force: true, recursive: true }); + symlinkSync(realDir, alias); + + const before = process.env.MINIMAX_DATA_DIR; + process.env.MINIMAX_DATA_DIR = alias; + try { + const r = await commitProviderCatalogueWrite({ records: [rec(A)] }); + assert.equal(r.ok, true); + const mode = statSync(join(realDir, "config.yaml")).mode & 0o777; + assert.equal(mode, 0o600, `expected 0600 through the symlinked dir, got ${mode.toString(8)}`); + } finally { + if (before === undefined) delete process.env.MINIMAX_DATA_DIR; + else process.env.MINIMAX_DATA_DIR = before; + rmSync(alias, { force: true }); + } + }); +}); + +// ===================================================================== +// DEATH LINE — PUT atomicity +// ===================================================================== + +describe("PUT atomicity — no state in which the store is half a catalogue", () => { + test("a REFUSED write leaves the previous catalogue exactly as it was", async () => { + await commitProviderCatalogueWrite({ records: [rec(A), rec(B)] }); + const before = readFileSync(configPath, "utf8"); + + // The write that cannot land: the store became unparseable between + // two reads. The refusal is the point — the previous document is + // still the whole truth, so a client's next GET returns the + // catalogue it already had. + writeFileSync(configPath, "{{{ broken\n", "utf8"); + const broken = readFileSync(configPath, "utf8"); + const refused = await commitProviderCatalogueWrite({ records: [rec({ ...A, label: "CHANGED" })] }); + assert.equal(refused.ok, false); + assert.equal(readFileSync(configPath, "utf8"), broken, "not one byte of the refusal touched the file"); + + // And the store still answers with the pre-write catalogue once + // the document is readable again. + writeFileSync(configPath, before, "utf8"); + const { readProviderStore } = await import("../../../server/engine/provider-store.js"); + const s = readProviderStore({ configPath }); + assert.deepEqual(s.records.map((x) => x.label), ["A", "B"], "no field of the refused write survived"); + }); + + test("a FAILED write leaves the previous catalogue exactly as it was", async () => { + await commitProviderCatalogueWrite({ records: [rec(A), rec(B)] }); + const before = readFileSync(configPath, "utf8"); + // A post-read I/O failure: the target's parent is a regular file, + // so the tmp write fails with ENOTDIR after the plan was built. + const blocked = join(engineDir, "sub", "config.yaml"); + writeFileSync(join(engineDir, "sub"), "x", "utf8"); + const failed = await commitProviderCatalogueWrite({ configPath: blocked, records: [rec({ ...A, label: "CHANGED" })] }); + assert.equal(failed.ok, false); + assert.equal(failed.code, "ENGINE_STORE_WRITE_FAILED"); + assert.equal(readFileSync(configPath, "utf8"), before, "the real store is untouched"); + rmSync(join(engineDir, "sub"), { force: true }); + }); + + test("concurrent commits leave ONE WHOLE state, never a mixture", async () => { + // The classic torn-write shape: provider a from one body and + // provider b from the other. With a single rename per write that + // cannot be constructed — a reader sees one document or the other. + const bodyA = [rec(A), rec(B)]; + const bodyB = [rec(B), rec({ ...A, label: "A2" })]; + const results = await Promise.all([ + commitProviderCatalogueWrite({ records: bodyA }), + commitProviderCatalogueWrite({ records: bodyB }), + ]); + assert.ok(results.every((r) => r.ok)); + const after = storeRecords(); + // Whichever landed, the result is one of the two BODIES — never a + // per-provider mix. + const matchesA = + after.length === 2 && after[0].id === "a" && after[0].label === "A" && after[1].id === "b"; + const matchesB = + after.length === 2 && after[0].id === "b" && after[1].id === "a" && after[1].label === "A2"; + assert.ok(matchesA || matchesB, `store is a mixture: ${JSON.stringify(after.map((x) => [x.id, x.label]))}`); + // And it parses: a torn YAML document could not be read at all. + assert.ok(storeDoc()._webui_provider_migration, "the winner is a complete document"); + }); + + test("an interleaved read never sees a partial document", async () => { + await commitProviderCatalogueWrite({ records: [rec(A)] }); + const { readProviderStore } = await import("../../../server/engine/provider-store.js"); + const writes = []; + for (let i = 0; i < 5; i++) { + writes.push(commitProviderCatalogueWrite({ records: [rec({ ...A, label: `A${i}` })] })); + } + writes.push(Promise.resolve().then(() => readProviderStore({ configPath }))); + const [, , , , , read] = await Promise.all(writes); + assert.equal(read.ok, true, "a concurrent reader always gets a parseable document"); + }); +}); diff --git a/packages/webui/test/lib/engine/streaming-send.test.js b/packages/webui/test/lib/engine/streaming-send.test.js index d7717a86..8ae20a4b 100644 --- a/packages/webui/test/lib/engine/streaming-send.test.js +++ b/packages/webui/test/lib/engine/streaming-send.test.js @@ -1675,8 +1675,14 @@ describe("routes/chat.js#handleSend on the runtime transport", () => { assert.equal(seen.status, 409); assert.equal( seen.body, - '{"ok":false,"error":"a turn is already running for this session","reason":"cid-busy"}', - "the pre-M3 409 body, byte for byte", + // P16: the refusal is user-facing — the composer renders `error` + // verbatim in its banner, and the internal detail reads "a turn is + // already running for this session", which names neither the + // decision nor the next action. `reason` stays the machine key. + '{"ok":false,"error":"This conversation is already running a turn. ' + + 'The message was NOT delivered — wait for the turn to finish, then send it again.",' + + '"reason":"cid-busy"}', + "the 409 body, byte for byte", ); assert.equal( siblingSeen && siblingSeen.ok, diff --git a/packages/webui/test/lib/providers-config.test.js b/packages/webui/test/lib/providers-config.test.js index 0be5a87a..6908b7fe 100644 --- a/packages/webui/test/lib/providers-config.test.js +++ b/packages/webui/test/lib/providers-config.test.js @@ -651,78 +651,21 @@ describe("publicView — apiKey masked in every response path", () => { }); // --------------------------------------------------------------------- -// writeProvidersConfig — atomic persistence. +// writeProvidersConfig — MOVED (batch B11) // --------------------------------------------------------------------- - -describe("writeProvidersConfig — atomic persistence", () => { - test("writes the user-level file with v2 schema", () => { - const r = providersConfig.writeProvidersConfig({ - version: 2, - providers: [ - { - id: "p", - label: "L", - protocol: "openai", - auth: { type: "byok", apiKey: "sk-realkey-aaa" }, - models: [{ id: "m1" }], - }, - ], - }); - assert.equal(r.ok, true); - const written = JSON.parse( - readFileSync(providersConfig.getUserLevelPath(), "utf8"), - ); - assert.equal(written.version, 2); - assert.equal(written.providers[0].id, "p"); - // Pinned: plaintext key persists to disk (it has to, the engine - // needs it) — but the masking contract only governs RESPONSES. - assert.equal(written.providers[0].auth.apiKey, "sk-realkey-aaa"); - }); - - test("rejects unknown protocol in any provider", () => { - const r = providersConfig.writeProvidersConfig({ - version: 2, - providers: [{ id: "p", protocol: "ollama", auth: { type: "byok" } }], - }); - assert.equal(r.ok, false); - assert.equal(r.code, "BAD_BODY"); - }); - - test("rejects duplicate provider id", () => { - const r = providersConfig.writeProvidersConfig({ - version: 2, - providers: [ - { id: "p", protocol: "openai", auth: { type: "byok", apiKey: "sk-aaaa" }, models: [] }, - { id: "p", protocol: "openai", auth: { type: "byok", apiKey: "sk-bbbb" }, models: [] }, - ], - }); - assert.equal(r.ok, false); - assert.equal(r.code, "BAD_BODY"); - }); - - test("rejects empty body", () => { - const r1 = providersConfig.writeProvidersConfig(null); - assert.equal(r1.ok, false); - const r2 = providersConfig.writeProvidersConfig({}); - assert.equal(r2.ok, false); - }); - - test("atomic write leaves no .tmp file behind", () => { - providersConfig.writeProvidersConfig({ - version: 2, - providers: [ - { - id: "p", - protocol: "openai", - auth: { type: "byok", apiKey: "sk-realkey-aaa" }, - models: [], - }, - ], - }); - const tmp = `${providersConfig.getUserLevelPath()}.tmp`; - assert.equal(existsSync(tmp), false, "no leftover .tmp file"); - }); -}); +// +// The v2 schema gate did not move with the function: it is now +// `planCatalogueFromBody` in `server/routes/providers.js`, and its +// persistence is `engine/provider-store.js#buildProviderStoreWrite` + +// `commitProviderStoreWrite`. Every assertion this block used to make +// about the SCHEMA (unknown protocol rejected, duplicate id rejected, +// empty body rejected, no temp file left behind, the plaintext key is +// on disk because the engine needs it) is now made against the store, +// in `test/lib/engine/provider-store.test.js` and +// `test/lib/engine/provider-writes.test.js`. +// +// `normaliseConfig` is still here and still pins all four schema +// refusals — the gate is the same code, the caller moved. // --------------------------------------------------------------------- // testProvider — local validation gate BEFORE network. diff --git a/packages/webui/test/routes/chat-first-turn-session-guard.check.mjs b/packages/webui/test/routes/chat-first-turn-session-guard.check.mjs index 49093c30..50bdfd6b 100644 --- a/packages/webui/test/routes/chat-first-turn-session-guard.check.mjs +++ b/packages/webui/test/routes/chat-first-turn-session-guard.check.mjs @@ -643,3 +643,95 @@ describe("POST /api/send — parallel turns in one tab", () => { assert.deepEqual(chatLines(cs), ["› once", "● ok", "› later", "● ok"]); }); }); + +// ------------------------------------------------------------------ +// P16 — the SAME tab sending again into its own live conversation. +// +// The case above is refused by the engine-session index (`runsBySid`), which +// the runner backfills mid-turn. The one below is the wiring P16 fixed: it +// runs the REAL chat.js → runMcodeAcp → sessions.js → state-bus chain, so +// it fails if the `moveRunSession` re-key is removed from `mcode-acp.js` — +// the registry-level suite cannot see that call, and a test that restates +// the fix inside its own runner proves nothing about production. +// +// What the UAT saw (2026-10-03 16点轮 异常 #1): a second message sent into +// a running conversation was ACKed, the engine ran it (the produced file +// contained the idiom named only in that message), and the webui transcript +// and the persisted record never contained it — because the turn's echo went +// into a live `cs.chat` that the run-mirror's finalize then wrote over from +// a snapshot taken before it. "The engine ran it and the webui does not know +// it" is the exact shape this contract forbids. +// ------------------------------------------------------------------ +describe("POST /api/send — P16: a send into this tab's own live turn", () => { + test("is refused, and reaches neither the engine nor the transcript", async () => { + const cid = "cid-P16-inflight"; + const cs = makeClient(cid); + + const res1 = fakeRes(); + const turn1 = drain.track(handleSend(fakeReq({ content: "first" }), res1, { cs, cid })); + // Mid-turn the record is promoted from its draft uuid to the engine id, + // and the claim has to follow it. Wait for the PROMOTED identity, not + // for the draft: that is the state in which the UAT's second send + // arrived, and the one the re-key exists for. + const sid = await waitFor( + () => (cs.mcodeSessionId ? cs.sessionId : null), + "the draft to be promoted to the engine identity", + ); + assert.match(sid, /^mvs_fake_/); + // The claim is registered under the identity the VIEW now presents. This + // is the production assertion: it reads the registry, not a runner the + // test controls. + assert.equal( + sb.getRunForSession(cid, sid), + sb.getRunsForCid(cid)[0]?.[1] ?? null, + "the live claim must be findable under the promoted conversation id", + ); + + // The second send, from the SAME tab, into the SAME conversation. + const res2 = fakeRes(); + await handleSend(fakeReq({ content: "守株待兔，水墨国风，滚动叙事长页" }), res2, { cs, cid }); + + assert.equal(res2._status, 409, "a send into a live turn must be refused, not acked"); + const body = JSON.parse(res2._body); + assert.equal(body.ok, false); + assert.ok( + ["cid-busy", "session-busy"].includes(body.reason), + `unexpected refusal reason: ${body.reason}`, + ); + assert.match(body.error, /NOT delivered/, "the refusal must state it was not delivered"); + + // ---- the reverse half ------------------------------------------------- + // One prompt is parked on the fake transport. A second one would mean the + // engine was handed a turn the webui had already refused. + assert.equal( + FakeMcodeAcpClient.pending.length, + 1, + "a refused send must never reach the engine", + ); + assert.deepEqual( + chatLines(cs), + ["› first"], + "a refused send must not be echoed into the live transcript", + ); + assert.equal(sb.activeRunCount(), 1, "the refused send must not claim a slot"); + + // The record on disk carries the turn that ran, and nothing else. + const stored = sessions.loadSessions().find((s) => s.id === sid); + assert.ok(stored, "the promoted record must exist"); + assert.ok( + !stored.chat.some((line) => String(line).includes("守株待兔")), + "a refused send must not reach the persisted record", + ); + + // The live turn finishes normally and releases the re-keyed claim. + FakeMcodeAcpClient.release(); + await turn1; + assert.equal(res1._status, 200); + await waitFor(() => sb.activeRunCount() === 0, "the re-keyed claim to be released"); + assert.equal( + sb.getRunForSession(cid, sid), + null, + "the re-key must not leak the claim past the turn that held it", + ); + }); +}); diff --git a/packages/webui/test/routes/chat-inflight-send.check.mjs b/packages/webui/test/routes/chat-inflight-send.check.mjs new file mode 100644 index 00000000..64d8a924 --- /dev/null +++ b/packages/webui/test/routes/chat-inflight-send.check.mjs @@ -0,0 +1,264 @@ +// webui/test/routes/chat-inflight-send.check.mjs +// +// P16 — a message sent into a conversation that is already running a turn. +// +// The defect this suite exists for (UAT 2026-10-03 16点轮 异常 #1): a second +// send into a live conversation was ACKED and handed to the engine while the +// webui kept no record of it. The turn's echo landed in the live `cs.chat`, +// the run-mirror's finalize then wrote the record from a `loadSessions()` +// snapshot taken before it, and the user was left with a message the engine +// had executed and the history did not contain. +// +// The cause is identity drift, not a missing check. A first turn's draft +// record is promoted to the engine `mvs_` id mid-turn, and `cs.sessionId` +// follows it. The run was claimed under the retired draft key, so the next +// send presents a key no live run holds: `beginRun` cannot see the running +// turn, and its only remaining guard — `runsBySid` — is populated by a +// separate backfill that has not necessarily landed yet. A guard that +// cannot see the turn must not ack it. +// +// The invariants pinned here: +// 1. a send into a live conversation is REFUSED with 409, whatever identity +// the conversation presents (draft key, promoted `mvs_` id); +// 2. the reverse half — a refused send never reaches the engine, is never +// echoed into the transcript, and never reaches the persisted record; +// 3. the re-key does not leak the claim: once the turn ends, the same +// conversation accepts a send again; +// 4. #139's parallel-conversation behaviour is untouched: a second +// conversation of the SAME tab is a different key and still runs; +// 5. the refusal body names the decision in words a user can act on, and +// keeps a machine-readable `reason` for the client's third banner state. +// +// This suite depends on t.mock.module → --experimental-test-module-mocks. + +import { test, describe, before, beforeEach } from "node:test"; +import assert from "node:assert/strict"; +import { Readable } from "node:stream"; +import { + setupMocks, + absPath, + registerSessionsStore, + registerMcodeAcpMock, + getSessionsStore, +} from "../helpers/_setup.js"; + +function fakeReq(body) { + return Readable.from([Buffer.from(JSON.stringify(body), "utf8")]); +} +function fakeRes() { + return { + _status: 200, + _headers: {}, + _body: null, + writeHead(s, h) { + this._status = s; + if (h) this._headers = h; + }, + end(b) { + this._body = b; + }, + }; +} + +/** Parse a route response body; a JSON string is a failure, not a crash. */ +function bodyOf(res) { + try { + return JSON.parse(res._body); + } catch { + return null; + } +} + +let handleSend, makeClientState, clients, bindDraftToMcodeSid; + +before(async (t) => { + await setupMocks(t, { mavis: { applyMavisUsageToCs: async () => {} } }); + const sb = await import(absPath("lib/state-bus.js")); + makeClientState = sb.makeClientState; + clients = sb.clients; + bindDraftToMcodeSid = (await import(absPath("lib/sessions.js"))).bindDraftToMcodeSid; + handleSend = (await import(absPath("routes/chat.js"))).handleSend; +}); + +beforeEach(() => { + clients.clear(); + registerSessionsStore({ initial: [] }); +}); + +/** + * A transport that hangs on its first prompt and mimics the ACP runner's + * mid-turn binding: the draft record is promoted to the engine id, and the + * run claim is re-keyed with it (the production `moveRunSession` call lives + * next to `bindDraftToMcodeSid` in `mcode-acp.js`, on both transports). + */ +function hangingRunner(sid, { backfillSid = true, rekey = true } = {}) { + const seen = []; + let release; + const gate = new Promise((r) => { + release = r; + }); + const runner = async (content, opts) => { + seen.push(content); + if (seen.length === 1) { + const sb = await import(absPath("lib/state-bus.js")); + if (backfillSid) sb.updateRunSid(opts.cid, sid, opts.owningWebuiSessionId); + bindDraftToMcodeSid(opts.cs, sid); + if (rekey && opts.cs.sessionId !== opts.owningWebuiSessionId) { + sb.moveRunSession(opts.cid, opts.owningWebuiSessionId, opts.cs.sessionId); + } + await gate; + } + return { status: "succeeded", answer: "mocked", sessionId: sid }; + }; + return { runner, seen, release, finish: () => release() }; +} + +describe("P16 — a send into a running conversation", () => { + test("is refused with a 409, and the refusal names the decision", async () => { + const { runner, seen, finish } = hangingRunner("mvs_busy"); + registerMcodeAcpMock({ runMcodeAcp: runner, runMcodeRuntime: runner }); + + const cid = "cid-busy"; + const cs = makeClientState(); + cs.chat = []; + clients.set(cid, cs); + const first = handleSend(fakeReq({ content: "first" }), fakeRes(), { cs, cid }); + await new Promise((r) => setTimeout(r, 30)); + + // The identity the view now presents: the promoted engine id. + assert.equal(cs.sessionId, "mvs_busy", "the draft must have been promoted mid-turn"); + + const res = fakeRes(); + await handleSend(fakeReq({ content: "second" }), res, { cs, cid }); + + assert.equal(res._status, 409, "a send into a live turn must be refused, not acked"); + const body = bodyOf(res); + assert.equal(body.ok, false); + assert.equal( + body.reason, + "cid-busy", + "the machine-readable reason drives the composer's third banner state", + ); + assert.match( + body.error, + /NOT delivered/, + "the user-facing message must state the message was not delivered", + ); + assert.doesNotMatch( + body.error, + /another window/, + "the internal detail's wording is wrong for the common case (same tab, second send)", + ); + + // ---- the reverse half: nothing about the refused send reached the engine + assert.deepEqual(seen, ["first"], "a refused send must never reach the engine"); + assert.deepEqual( + cs.chat, + ["› first"], + "a refused send must not be echoed into the live transcript", + ); + for (const record of getSessionsStore()) { + assert.ok( + !record.chat.some((line) => line.includes("second")), + "a refused send must not reach the persisted record", + ); + } + + finish(); + await first; + }); + + test("is refused even when the engine-id backfill has not landed yet", async () => { + // The drift alone is enough to hide the running turn from `beginRun`: + // with `runsBySid` still empty the only guard is the conversation key, + // which the promotion changed. This is the exact shape the UAT hit. + const { runner, seen, finish } = hangingRunner("mvs_nobackfill", { + backfillSid: false, + }); + registerMcodeAcpMock({ runMcodeAcp: runner, runMcodeRuntime: runner }); + + const cid = "cid-nobackfill"; + const cs = makeClientState(); + cs.chat = []; + clients.set(cid, cs); + const first = handleSend(fakeReq({ content: "first" }), fakeRes(), { cs, cid }); + await new Promise((r) => setTimeout(r, 30)); + + const res = fakeRes(); + await handleSend(fakeReq({ content: "守株待兔，水墨国风" }), res, { cs, cid }); + + assert.equal(res._status, 409); + assert.deepEqual(seen, ["first"], "the engine must not be given a second concurrent turn"); + assert.ok( + !cs.chat.some((line) => line.includes("守株待兔")), + "the refused message must not appear in the transcript", + ); + + finish(); + await first; + assert.ok( + !getSessionsStore().some((r) => r.chat.some((l) => l.includes("守株待兔"))), + "the refused message must not survive into the persisted record", + ); + }); + + test("the re-keyed claim is released when the turn ends (no leak)", async () => { + const { runner, finish } = hangingRunner("mvs_release"); + registerMcodeAcpMock({ runMcodeAcp: runner, runMcodeRuntime: runner }); + + const cid = "cid-release"; + const cs = makeClientState(); + cs.chat = []; + clients.set(cid, cs); + const first = handleSend(fakeReq({ content: "first" }), fakeRes(), { cs, cid }); + await new Promise((r) => setTimeout(r, 30)); + assert.equal(cs.sessionId, "mvs_release"); + + finish(); + await first; + + // The same conversation, now idle, must accept a send. A claim that + // outlived its own turn would refuse every later send in it forever — + // the failure mode an alias-blind release would have introduced here. + const res = fakeRes(); + await handleSend(fakeReq({ content: "after" }), res, { cs, cid }); + assert.equal(res._status, 200, "a finished turn must leave the conversation sendable"); + assert.equal( + fakeRes()._status, + 200, + "sanity: a fresh response object defaults to 200, so 409 above is real", + ); + }); + + test("a second conversation of the same tab still runs in parallel (#139)", async () => { + const { runner, seen, finish } = hangingRunner("mvs_a"); + registerMcodeAcpMock({ runMcodeAcp: runner, runMcodeRuntime: runner }); + + const cid = "cid-parallel"; + const csA = makeClientState(); + csA.chat = []; + const csB = makeClientState(); + csB.chat = []; + // Two existing conversations — distinct claim keys, which is what #139 + // bought. Two unsaved drafts share the `null` key and are refused, which + // is a different question and is covered by the guard's own suite. + csB.sessionId = "web-B"; + registerSessionsStore({ + initial: [{ id: "web-B", title: "B", chat: [], workspace: null }], + }); + clients.set(cid, csA); + clients.set(cid, csB); + + const firstA = handleSend(fakeReq({ content: "A1" }), fakeRes(), { cs: csA, cid }); + await new Promise((r) => setTimeout(r, 30)); + const resB = fakeRes(); + const firstB = handleSend(fakeReq({ content: "B1" }), resB, { cs: csB, cid }); + await new Promise((r) => setTimeout(r, 30)); + + assert.equal(resB._status, 200, "a DIFFERENT conversation of the same tab is not a busy send"); + assert.deepEqual(seen.sort(), ["A1", "B1"]); + + finish(); + await Promise.all([firstA, firstB]); + }); +}); diff --git a/packages/webui/test/routes/model.check.mjs b/packages/webui/test/routes/model.check.mjs index a8cbbdc4..cf560a03 100644 --- a/packages/webui/test/routes/model.check.mjs +++ b/packages/webui/test/routes/model.check.mjs @@ -15,6 +15,8 @@ import { test, describe, before, after } from "node:test"; import assert from "node:assert/strict"; import { Readable } from "node:stream"; import { tmpdir } from "node:os"; +import { readFileSync } from "node:fs"; +import { fileURLToPath } from "node:url"; import { setupMocks, absPath } from "../helpers/_setup.js"; import { mkTmpDir } from "../helpers/tmp.js"; import yaml from "js-yaml"; @@ -1942,3 +1944,133 @@ describe("handleSetModel — contextWindow (U6)", () => { assert.equal(cs.model.contextWindow, 1000000); }); }); + +// ============================================================ +// M3-B14 — THE GATE, AS THE ROUTE SEES IT. +// +// The executor-level tests in `test/lib/engine/model-writes.test.js` +// cover the verdict, the channel precision and the 501 payload against +// synthetic providers. What is left for this file is the two properties +// only the route can be wrong about: +// +// 1. THE ROUTE DOES NOT CATCH THE GATE. If it did, a provider that +// cannot perform an effort write would produce a 200 with a +// `warning` string — #110's fake success in the exact shape the +// capability gate was built to prevent, and harder to notice than +// a 501 because the picker would still move. +// 2. THE GATE'S REPORT NEVER REACHES THE WIRE. The executors gained a +// `gate` field in M3-B14; the response body is byte-identical to +// the pre-B14 one, and this is what says so. +// +// The refusal path itself is NOT driven from here. Reaching it needs a +// provider that denies the dedicated writers, and no registered provider +// does — the file boots the real registry once, without a `?bust=` +// parameter, so there is no seam to swap one in. Asserting a 501 here +// would mean inventing a fake registry, and the real-registry +// "still 200 under both transports" assertion below is the fact that +// actually matters for a shipped user. +// ============================================================ + +const ENV_TRANSPORT = process.env.MCODE_WEBUI_TRANSPORT || "acp"; + +/** A live session on the effort channel: the shape #58 really gates. */ +function gatedCs() { + const cs = fakeCs("minimax_api/MiniMax-M3.1-Flash-Preview", [ + { + type: "select", + id: "model", + name: "Model", + currentValue: "minimax_api/MiniMax-M3.1-Flash-Preview", + options: [ + { value: "minimax_api/MiniMax-M3.1-Flash-Preview", name: "M3.1-Flash-Preview" }, + ], + }, + { type: "select", id: "thinkingEffort", name: "Thinking effort", currentValue: "low" }, + ]); + cs.mcodeSessionId = "mvs_b14_0000000000000000000000"; + return cs; +} + +describe("handleSetModel — the M3-B14 gate, from the route", () => { + test("an effort write on a live session is 200 under the real registry, on both transports", async () => { + // The shipped behaviour change, stated as the half that must NOT + // change: no registered provider denies `setThinkingEffort`, so + // adding the gate removes nothing for any user today. + const cs = gatedCs(); + const res = fakeRes(); + await modelRoute.handleSetModel( + fakeReq({ model: "minimax_api/MiniMax-M3.1-Flash-Preview", thinking: "high" }), + res, + { cs, cid: "cid-b14" }, + ); + assert.equal(res._status, 200, ENV_TRANSPORT); + const body = JSON.parse(res._body); + assert.equal(body.ok, true); + assert.equal(body.mcodeSynced, true); + assert.equal(body.thinkingSynced, true); + assert.equal(body.warning, undefined, "no refusal was invented"); + }); + + test("the response body is byte-identical to the pre-M3-B14 one — no `gate` field", async () => { + // Asserted as a KEY SET rather than a snapshot string, because the + // keys are the contract and the order is not: what must not appear is + // a new field, and the one this batch could most plausibly have added + // is the gate's own report. + const cs = gatedCs(); + const res = fakeRes(); + await modelRoute.handleSetModel( + fakeReq({ model: "minimax_api/MiniMax-M3.1-Flash-Preview", thinking: "high" }), + res, + { cs, cid: "cid-b14" }, + ); + assert.deepEqual( + Object.keys(JSON.parse(res._body)).sort(), + ["mcodeSynced", "model", "ok", "thinking", "thinkingSynced"], + ); + }); + + test("neither handler catches — a gate refusal must reach app.js, not a 200", () => { + // Static-source tripwire, and the only kind available without a + // render/registry seam. The thing it forbids is specific: a `catch` + // around the push. A `catch` here would turn the engine gate's 501 + // into `warning: ` on a 200 — the one outcome B9's module + // header calls the purest form of fake success, and the one a reader + // cannot spot because the picker would still move. + const src = readFileSync(fileURLToPath(absPath("routes/model.js")), "utf8"); + for (const handler of ["handleSetModel", "handleSetPermissions"]) { + const start = src.indexOf(`export async function ${handler}`); + assert.ok(start > 0, `${handler} not found`); + // The handler ends at the next top-level `export` (or EOF). + const next = src.indexOf("\nexport ", start + 1); + const body = src.slice(start, next === -1 ? undefined : next); + assert.doesNotMatch(body, /\bcatch\b/, `${handler} must not catch the gate's error`); + } + }); +}); + +describe("handleSetPermissions — the M3-B14 gate, from the route", () => { + test("a permission write is 200 under the real registry, unchanged", async () => { + const cs = gatedCs(); + const res = fakeRes(); + await modelRoute.handleSetPermissions(fakeReq({ mode: "ask" }), res, { cs, cid: "cid-b14" }); + assert.equal(res._status, 200, ENV_TRANSPORT); + const body = JSON.parse(res._body); + assert.deepEqual(Object.keys(body).sort(), ["mcodeSynced", "ok", "permissions"]); + assert.equal(body.mcodeSynced, true); + assert.equal(body.warning, undefined); + }); + + test("no session is still 200 with the local-only warning, gate or no gate", async () => { + // The path that returns before the gate. A 501 here would be a + // regression the other way: a request that never reaches the engine + // cannot be a fake success, so there is nothing for the capability to + // be honest about, and the recorded pick is the truthful answer. + const cs = fakeCs(); + const res = fakeRes(); + await modelRoute.handleSetPermissions(fakeReq({ mode: "ask" }), res, { cs, cid: "cid-b14" }); + assert.equal(res._status, 200); + const body = JSON.parse(res._body); + assert.equal(body.mcodeSynced, false); + assert.equal(body.warning, "no mcode session yet — applies to the next one"); + }); +}); diff --git a/packages/webui/test/routes/provider-presets.check.mjs b/packages/webui/test/routes/provider-presets.check.mjs index d19ea2e4..c51ab163 100644 --- a/packages/webui/test/routes/provider-presets.check.mjs +++ b/packages/webui/test/routes/provider-presets.check.mjs @@ -24,6 +24,7 @@ import { test, describe, before, after, beforeEach } from "node:test"; import assert from "node:assert/strict"; import {rmSync, writeFileSync, existsSync, readFileSync} from "node:fs"; +import yaml from "js-yaml"; import { join } from "node:path"; import { pathToFileURL } from "node:url"; @@ -38,17 +39,25 @@ const presets = await import(absPath("lib/provider-presets.js")); let _tmpDataDir; let _tmpCwd; +let _tmpEngineDir; let _origDataDir; +let _origEngineDir; let _origCwdEnv; let _origCwd; before(async () => { _tmpDataDir = mkTmpDir("webui-presets-route-"); _tmpCwd = mkTmpDir("webui-presets-route-cwd-"); + // M3-B11: enabling a preset now commits to the ENGINE store, so the + // suite needs an isolated engine data dir or it would write the + // developer's real ~/.minimax/config.yaml. + _tmpEngineDir = mkTmpDir("webui-presets-engine-"); _origDataDir = process.env.MCODE_WEBUI_DATA_DIR; + _origEngineDir = process.env.MINIMAX_DATA_DIR; _origCwdEnv = process.env.MCODE_WEBUI_MODELS_CONFIG; _origCwd = process.cwd(); process.env.MCODE_WEBUI_DATA_DIR = _tmpDataDir; + process.env.MINIMAX_DATA_DIR = _tmpEngineDir; process.env.MCODE_WEBUI_MODELS_CONFIG = ""; process.chdir(_tmpCwd); }); @@ -56,20 +65,73 @@ before(async () => { after(async () => { if (_origDataDir === undefined) delete process.env.MCODE_WEBUI_DATA_DIR; else process.env.MCODE_WEBUI_DATA_DIR = _origDataDir; + if (_origEngineDir === undefined) delete process.env.MINIMAX_DATA_DIR; + else process.env.MINIMAX_DATA_DIR = _origEngineDir; if (_origCwdEnv === undefined) delete process.env.MCODE_WEBUI_MODELS_CONFIG; else process.env.MCODE_WEBUI_MODELS_CONFIG = _origCwdEnv; try { process.chdir(_origCwd); } catch {} if (_tmpDataDir) try { rmSync(_tmpDataDir, { recursive: true, force: true }); } catch {} if (_tmpCwd) try { rmSync(_tmpCwd, { recursive: true, force: true }); } catch {} + if (_tmpEngineDir) try { rmSync(_tmpEngineDir, { recursive: true, force: true }); } catch {} }); beforeEach(() => { - const cwdFile = join(_tmpCwd, "models.json"); - if (existsSync(cwdFile)) rmSync(cwdFile); - const userFile = join(_tmpDataDir, "providers.json"); - if (existsSync(userFile)) rmSync(userFile); + // BOTH halves of the pre-B11 dual source are cleared: the deprecated + // providers.json (the fallback authority) and the engine store (the + // authority once the migration marker is stamped). + for (const f of [ + join(_tmpCwd, "models.json"), + join(_tmpDataDir, "providers.json"), + join(_tmpEngineDir, "config.yaml"), + ]) { + if (existsSync(f)) rmSync(f); + } }); +/** + * The provider records the store holds, read straight off disk. + * + * The store is a YAML document keyed by engine provider key, so an + * assertion about "what the user saved" reads the `_webui_provider` + * record each webui-owned entry carries rather than the engine + * projection beside it: the record is what the catalogue API + * serialises. + * + * @returns {object[]} + */ +function readStoreRecords() { + const file = join(_tmpEngineDir, "config.yaml"); + if (!existsSync(file)) return []; + const doc = yaml.load(readFileSync(file, "utf8")) || {}; + return Object.values(doc.custom_provider || {}) + .map((entry) => entry && entry._webui_provider) + .filter(Boolean); +} + +/** + * Rewrite one record's apiKey IN the store, the way a user editing the + * dialog would. The pre-B11 suite hand-edited providers.json; the + * equivalent gesture now targets the file the store actually lives in. + * + * @param {string} id + * @param {string} apiKey + * @returns {void} + */ +function writeStoreApiKey(id, apiKey) { + const file = join(_tmpEngineDir, "config.yaml"); + const doc = yaml.load(readFileSync(file, "utf8")) || {}; + for (const entry of Object.values(doc.custom_provider || {})) { + if (entry && entry._webui_provider && entry._webui_provider.id === id) { + entry._webui_provider.auth.apiKey = apiKey; + // The engine projection is absent for a key-less provider (there + // is nothing for the engine to call), so the guard is the normal + // case for a freshly materialised preset, not an edge. + if (entry.options) entry.options.apiKey = apiKey; + } + } + writeFileSync(file, yaml.dump(doc, { indent: 2, lineWidth: -1, noRefs: true }), "utf8"); +} + function fakeReq(url) { return { url }; } @@ -91,9 +153,9 @@ function getBody(res) { // ===================================================================== describe("handleGetPresets — /api/providers/presets GET", () => { - test("returns all 11 presets with enabled=false when nothing is configured (ticket 06)", () => { + test("returns all 11 presets with enabled=false when nothing is configured (ticket 06)", async () => { const res = fakeRes(); - providersRoute.handleGetPresets(null, res, {}); + await providersRoute.handleGetPresets(null, res, {}); assert.equal(res._status, 200); const body = getBody(res); assert.equal(body.ok, true); @@ -105,9 +167,9 @@ describe("handleGetPresets — /api/providers/presets GET", () => { assert.deepEqual(body.enabledIds, []); }); - test("preset entries carry id, label, protocol, auth (no key), models", () => { + test("preset entries carry id, label, protocol, auth (no key), models", async () => { const res = fakeRes(); - providersRoute.handleGetPresets(null, res, {}); + await providersRoute.handleGetPresets(null, res, {}); const body = getBody(res); const zhipu = body.presets.find((p) => p.id === "zhipu"); assert.ok(zhipu); @@ -122,7 +184,7 @@ describe("handleGetPresets — /api/providers/presets GET", () => { assert.ok(zhipu.models.length > 0); }); - test("enabled=true once the preset id is configured", () => { + test("enabled=true once the preset id is configured", async () => { // Pre-populate the user-level file with a provider that // matches a preset id. writeFileSync( @@ -141,14 +203,14 @@ describe("handleGetPresets — /api/providers/presets GET", () => { }), ); const res = fakeRes(); - providersRoute.handleGetPresets(null, res, {}); + await providersRoute.handleGetPresets(null, res, {}); const body = getBody(res); const zhipu = body.presets.find((p) => p.id === "zhipu"); assert.equal(zhipu.enabled, true); assert.ok(body.enabledIds.includes("zhipu")); }); - test("custom (non-preset) configured providers do NOT show as enabled", () => { + test("custom (non-preset) configured providers do NOT show as enabled", async () => { writeFileSync( join(_tmpDataDir, "providers.json"), JSON.stringify({ @@ -165,7 +227,7 @@ describe("handleGetPresets — /api/providers/presets GET", () => { }), ); const res = fakeRes(); - providersRoute.handleGetPresets(null, res, {}); + await providersRoute.handleGetPresets(null, res, {}); const body = getBody(res); assert.equal(body.enabledIds.length, 0, "custom providers are not preset-flagged"); for (const p of body.presets) { @@ -224,9 +286,7 @@ describe("handleEnablePreset — /api/providers/preset/:id/enable POST", () => { fakeRes(), {}, ); - const onDisk = JSON.parse( - readFileSync(providersConfig.getUserLevelPath(), "utf8"), - ); + const onDisk = { providers: readStoreRecords() }; const kimi = onDisk.providers.find((p) => p.id === "kimi"); assert.ok(kimi, "kimi persisted"); assert.equal(kimi.enabled, true); @@ -243,7 +303,7 @@ describe("handleEnablePreset — /api/providers/preset/:id/enable POST", () => { {}, ); const res = fakeRes(); - providersRoute.handleGetProviders(null, res, {}); + await providersRoute.handleGetProviders(null, res, {}); const body = getBody(res); const bailian = body.providers.find((p) => p.id === "bailian"); assert.ok(bailian, "bailian visible after enable"); @@ -285,11 +345,7 @@ describe("handleEnablePreset — /api/providers/preset/:id/enable POST", () => { {}, ); // User fills the apiKey via a normal PUT. - const onDiskPath = providersConfig.getUserLevelPath(); - let onDisk = JSON.parse(readFileSync(onDiskPath, "utf8")); - const mimoIdx = onDisk.providers.findIndex((p) => p.id === "mimo"); - onDisk.providers[mimoIdx].auth.apiKey = "sk-realkey-user-filled-key"; - writeFileSync(onDiskPath, JSON.stringify(onDisk, null, 2), "utf8"); + writeStoreApiKey("mimo", "sk-realkey-user-filled-key"); // Second enable must NOT clobber the key. const res = fakeRes(); @@ -302,8 +358,7 @@ describe("handleEnablePreset — /api/providers/preset/:id/enable POST", () => { const body = getBody(res); assert.equal(body.alreadyEnabled, true); - onDisk = JSON.parse(readFileSync(onDiskPath, "utf8")); - const mimo = onDisk.providers.find((p) => p.id === "mimo"); + const mimo = readStoreRecords().find((p) => p.id === "mimo"); assert.equal( mimo.auth.apiKey, "sk-realkey-user-filled-key", @@ -344,7 +399,7 @@ describe("handleEnablePreset — /api/providers/preset/:id/enable POST", () => { assert.equal(body.provider.models[0].id, "custom-model"); // The file on disk still has the custom record unchanged. - const onDisk = JSON.parse(readFileSync(providersConfig.getUserLevelPath(), "utf8")); + const onDisk = { providers: readStoreRecords() }; const minimax = onDisk.providers.find((p) => p.id === "minimax"); assert.equal(minimax.label, "My Custom minimax"); assert.equal(minimax.auth.apiKey, "sk-realkey-custom"); @@ -375,7 +430,7 @@ describe("handleEnablePreset — /api/providers/preset/:id/enable POST", () => { {}, ); - const onDisk = JSON.parse(readFileSync(providersConfig.getUserLevelPath(), "utf8")); + const onDisk = { providers: readStoreRecords() }; const ids = onDisk.providers.map((p) => p.id).sort(); assert.deepEqual(ids, ["my-other-custom", "openrouter"]); // Other-custom record untouched. @@ -412,7 +467,7 @@ describe("handleEnablePreset — /api/providers/preset/:id/enable POST", () => { {}, ); const res = fakeRes(); - providersRoute.handleGetProviders(null, res, {}); + await providersRoute.handleGetProviders(null, res, {}); const body = getBody(res); const claude = body.providers.find((p) => p.id === "claude-code"); assert.ok(claude); diff --git a/packages/webui/test/routes/providers.check.mjs b/packages/webui/test/routes/providers.check.mjs index 2994efe5..b902126f 100644 --- a/packages/webui/test/routes/providers.check.mjs +++ b/packages/webui/test/routes/providers.check.mjs @@ -23,10 +23,12 @@ import { test, describe, before, after, beforeEach } from "node:test"; import assert from "node:assert/strict"; import { Readable } from "node:stream"; import {rmSync, writeFileSync, existsSync, readFileSync} from "node:fs"; +import yaml from "js-yaml"; import { join } from "node:path"; import { pathToFileURL } from "node:url"; import { mkTmpDir } from "../helpers/tmp.js"; +import { setupMocks } from "../helpers/_setup.js"; const absPath = (rel) => pathToFileURL(join(import.meta.dirname, "..", "..", "server", rel)).href; @@ -36,17 +38,27 @@ const providersConfig = await import(absPath("lib/providers-config.js")); let _tmpDataDir; let _tmpCwd; +let _tmpEngineDir; let _origDataDir; +let _origEngineDir; let _origCwdEnv; let _origCwd; before(async () => { _tmpDataDir = mkTmpDir("webui-providers-route-"); _tmpCwd = mkTmpDir("webui-providers-route-cwd-"); + // M3-B11: the catalogue now lives in the ENGINE config, so this suite + // needs an isolated engine data dir. Without one the route would + // write the developer's real ~/.minimax/config.yaml — the same + // isolation contract test/lib/engine/capability-snapshot.test.js + // states, for the same reason. + _tmpEngineDir = mkTmpDir("webui-providers-engine-"); _origDataDir = process.env.MCODE_WEBUI_DATA_DIR; + _origEngineDir = process.env.MINIMAX_DATA_DIR; _origCwdEnv = process.env.MCODE_WEBUI_MODELS_CONFIG; _origCwd = process.cwd(); process.env.MCODE_WEBUI_DATA_DIR = _tmpDataDir; + process.env.MINIMAX_DATA_DIR = _tmpEngineDir; process.env.MCODE_WEBUI_MODELS_CONFIG = ""; process.chdir(_tmpCwd); }); @@ -54,20 +66,52 @@ before(async () => { after(async () => { if (_origDataDir === undefined) delete process.env.MCODE_WEBUI_DATA_DIR; else process.env.MCODE_WEBUI_DATA_DIR = _origDataDir; + if (_origEngineDir === undefined) delete process.env.MINIMAX_DATA_DIR; + else process.env.MINIMAX_DATA_DIR = _origEngineDir; if (_origCwdEnv === undefined) delete process.env.MCODE_WEBUI_MODELS_CONFIG; else process.env.MCODE_WEBUI_MODELS_CONFIG = _origCwdEnv; try { process.chdir(_origCwd); } catch {} if (_tmpDataDir) try { rmSync(_tmpDataDir, { recursive: true, force: true }); } catch {} if (_tmpCwd) try { rmSync(_tmpCwd, { recursive: true, force: true }); } catch {} + if (_tmpEngineDir) try { rmSync(_tmpEngineDir, { recursive: true, force: true }); } catch {} }); beforeEach(() => { - const cwdFile = join(_tmpCwd, "models.json"); - if (existsSync(cwdFile)) rmSync(cwdFile); - const userFile = join(_tmpDataDir, "providers.json"); - if (existsSync(userFile)) rmSync(userFile); + // BOTH halves of the pre-B11 dual source are cleared: the deprecated + // providers.json (the fallback authority) and the engine store (the + // authority once the marker is stamped). Leaving either behind would + // let one test's write decide the next test's fixture. + for (const f of [ + join(_tmpCwd, "models.json"), + join(_tmpCwd, "env.json"), + join(_tmpCwd, "env-only.json"), + join(_tmpDataDir, "providers.json"), + join(_tmpEngineDir, "config.yaml"), + ]) { + if (existsSync(f)) rmSync(f); + } }); +/** + * The provider records the store holds, read straight off disk. + * + * The store is a YAML document keyed by engine provider key, so the + * assertions below read the `_webui_provider` record each webui-owned + * entry carries rather than the engine projection beside it: the record + * is what the catalogue API serialises, so it is the thing a test must + * compare against. + * + * @returns {object[]} + */ +function readStoreRecords() { + const file = join(_tmpEngineDir, "config.yaml"); + if (!existsSync(file)) return []; + const doc = yaml.load(readFileSync(file, "utf8")) || {}; + return Object.values(doc.custom_provider || {}) + .map((entry) => entry && entry._webui_provider) + .filter(Boolean); +} + function fakeReq(body) { return Readable.from([Buffer.from(JSON.stringify(body), "utf8")]); } @@ -89,9 +133,9 @@ function getBody(res) { // ===================================================================== describe("handleGetProviders — /api/providers GET", () => { - test("empty config returns ok + empty providers + sources", () => { + test("empty config returns ok + empty providers + sources", async () => { const res = fakeRes(); - providersRoute.handleGetProviders(null, res, {}); + await providersRoute.handleGetProviders(null, res, {}); assert.equal(res._status, 200); const body = getBody(res); assert.equal(body.ok, true); @@ -101,7 +145,7 @@ describe("handleGetProviders — /api/providers GET", () => { assert.ok(body.userPath, "userPath present"); }); - test("user-level file is read on every call (hot reload)", () => { + test("the deprecated file is read on every call (hot reload)", async () => { writeFileSync( join(_tmpDataDir, "providers.json"), JSON.stringify({ @@ -118,14 +162,14 @@ describe("handleGetProviders — /api/providers GET", () => { }), ); const res = fakeRes(); - providersRoute.handleGetProviders(null, res, {}); + await providersRoute.handleGetProviders(null, res, {}); const body = getBody(res); assert.equal(body.providers.length, 1); assert.equal(body.providers[0].id, "u1"); assert.equal(body.providers[0].label, "User One"); }); - test("apiKey is masked in every provider (no plaintext anywhere)", () => { + test("apiKey is masked in every provider (no plaintext anywhere)", async () => { const key = "sk-realkey-this-is-the-secret-1234"; writeFileSync( join(_tmpDataDir, "providers.json"), @@ -150,7 +194,7 @@ describe("handleGetProviders — /api/providers GET", () => { }), ); const res = fakeRes(); - providersRoute.handleGetProviders(null, res, {}); + await providersRoute.handleGetProviders(null, res, {}); const body = getBody(res); // Pinned: the plaintext key MUST NOT appear in any response shape. const json = res._body; @@ -165,13 +209,13 @@ describe("handleGetProviders — /api/providers GET", () => { assert.equal(p1.auth.baseURL, ""); }); - test("sources.{env,cwd,user} point at the resolved paths", () => { + test("sources.{env,cwd,user} point at the resolved paths", async () => { const envFile = join(_tmpCwd, "env.json"); writeFileSync(envFile, JSON.stringify({ providers: [] })); process.env.MCODE_WEBUI_MODELS_CONFIG = envFile; try { const res = fakeRes(); - providersRoute.handleGetProviders(null, res, {}); + await providersRoute.handleGetProviders(null, res, {}); const body = getBody(res); assert.equal(body.sources.env, envFile, "env override is reported"); // When env override is set, the cwd path is NOT read — the @@ -190,7 +234,7 @@ describe("handleGetProviders — /api/providers GET", () => { // ===================================================================== describe("handlePutProviders — /api/providers PUT", () => { - test("valid body persists to user-level file and returns masked shape", async () => { + test("valid body persists to the engine store and returns masked shape", async () => { const res = fakeRes(); await providersRoute.handlePutProviders( fakeReq({ @@ -216,9 +260,7 @@ describe("handlePutProviders — /api/providers PUT", () => { // Plaintext key NEVER appears anywhere in the response. assert.equal(res._body.includes("realkey"), false); // File persisted. - const onDisk = JSON.parse( - readFileSync(providersConfig.getUserLevelPath(), "utf8"), - ); + const onDisk = { providers: readStoreRecords() }; assert.equal(onDisk.providers[0].id, "p1"); assert.equal(onDisk.providers[0].auth.apiKey, "sk-realkey-aaaa"); }); @@ -294,14 +336,14 @@ describe("handlePutProviders — /api/providers PUT", () => { assert.equal(put._status, 200); // GET picks it up. const get = fakeRes(); - providersRoute.handleGetProviders(null, get, {}); + await providersRoute.handleGetProviders(null, get, {}); const body = getBody(get); const found = body.providers.find((p) => p.id === "newprov"); assert.ok(found, "newprov visible after PUT"); assert.equal(found.models.length, 1); }); - test("keep-existing-key: empty apiKey in PUT preserves the key on disk", async () => { + test("keep-existing-key: empty apiKey in PUT preserves the key in the store", async () => { // Seed: write a provider with a plaintext key. await providersRoute.handlePutProviders( fakeReq({ @@ -338,9 +380,7 @@ describe("handlePutProviders — /api/providers PUT", () => { fakeRes(), {}, ); - const onDisk = JSON.parse( - readFileSync(providersConfig.getUserLevelPath(), "utf8"), - ); + const onDisk = { providers: readStoreRecords() }; const kp = onDisk.providers.find((p) => p.id === "kp"); assert.equal(kp.auth.apiKey, "sk-original-plaintext-aaaa"); assert.equal(kp.label, "KP renamed"); @@ -380,9 +420,7 @@ describe("handlePutProviders — /api/providers PUT", () => { fakeRes(), {}, ); - const onDisk = JSON.parse( - readFileSync(providersConfig.getUserLevelPath(), "utf8"), - ); + const onDisk = { providers: readStoreRecords() }; const kp = onDisk.providers.find((p) => p.id === "kp2"); assert.equal(kp.auth.apiKey, "sk-new-plaintext-bbbb"); }); @@ -425,9 +463,7 @@ describe("handlePutProviders — /api/providers PUT", () => { fakeRes(), {}, ); - const onDisk = JSON.parse( - readFileSync(providersConfig.getUserLevelPath(), "utf8"), - ); + const onDisk = { providers: readStoreRecords() }; const row = onDisk.providers.find((p) => p.id === "abs"); assert.equal(row.auth.apiKey, "sk-on-disk-original-aaaa"); assert.equal(row.label, "Absent renamed"); @@ -476,9 +512,7 @@ describe("handlePutProviders — /api/providers PUT", () => { fakeRes(), {}, ); - const onDisk = JSON.parse( - readFileSync(providersConfig.getUserLevelPath(), "utf8"), - ); + const onDisk = { providers: readStoreRecords() }; const row = onDisk.providers.find((p) => p.id === "envprov"); // The user-level record MUST NOT carry the env secret. The // env secret is deployment-managed and stays at the env layer. @@ -598,7 +632,7 @@ describe("handleTestProvider — /api/providers/test POST", () => { // ===================================================================== describe("SSE broadcast — providers.updated payload is masked", () => { - test("the named SSE event carries the masked provider shape", () => { + test("the named SSE event carries the masked provider shape", async () => { writeFileSync( join(_tmpDataDir, "providers.json"), JSON.stringify({ @@ -614,7 +648,7 @@ describe("SSE broadcast — providers.updated payload is masked", () => { ], }), ); - const frame = providersRoute._peekProvidersUpdatedFrame(); + const frame = await providersRoute._peekProvidersUpdatedFrame(); // Plaintext apiKey NEVER in the SSE frame. assert.equal(frame.includes("realkey"), false); assert.equal(frame.includes("secret"), false); @@ -746,3 +780,143 @@ describe("custom headers — route passthrough (ticket 85)", () => { assert.equal(lastHeaders["x-evil"], undefined, "the whole record is rejected"); }); }); + +// --------------------------------------------------------------------- +// PROOF — the route really calls the engine facade +// --------------------------------------------------------------------- +// +// Everything above runs against the REAL engine modules, which is what +// makes those tests worth having. It also means none of them can +// distinguish "the route called the facade" from "the route kept its +// own copy of the logic and the facade happens to agree" — a route that +// inlined a second implementation of the same decision would pass all +// of them. +// +// The proof is a marker. `mock.module` replaces the write half of the +// facade with a stub that throws a unique error, the route is +// re-imported under a fresh `?bust=N` (without it the route keeps its +// previous LIVE BINDING to the real module and the marker is never +// thrown), and the test asserts the error escapes by IDENTITY. The +// CONTROL below then runs the same request with no mock and asserts +// the real commit landed — so the two PROOF cases cannot both be +// passing for the wrong reason. + +let _bust = 0; + +/** + * A fresh copy of `routes/providers.js`. + * + * @returns {Promise} + */ +const loadRoute = async () => + import(`${absPath("routes/providers.js")}?bust=${_bust++}`); + +describe("PROOF — the provider route is bound to the engine facade", () => { + test("PROOF: a marker error from the write facade escapes handlePutProviders", async (t) => { + await setupMocks(t, { acp: {} }); + const marker = new Error("B11-MOCK-WAS-NOT-HONOURED"); + t.mock.module(absPath("engine/provider-writes.js"), { + namedExports: { + commitProviderCatalogueWrite: async () => { + throw marker; + }, + // Every other name the route imports from this module is the + // real one. A namespace mock REPLACES the whole module, so + // anything not listed here would be undefined at the call site + // and the test would fail for a reason that has nothing to do + // with the marker. + assertProviderWriteCapability: () => ({ + endpoint: "PUT /api/providers", + provider: "local-runtime-v2", + capability: "authCredentials", + subItem: "updateUserModelProvider", + enforcement: "hard", + gate: "checked", + }), + planProviderCatalogueWrite: (existing, incoming) => + (incoming || []).map((p) => { + if (!p || !p.auth) return p; + if (p.auth.apiKey) return p; + const prev = (existing || []).find((e) => e && e.id === p.id); + return { ...p, auth: { ...p.auth, apiKey: prev ? prev.auth.apiKey : "" } }; + }), + resolveProviderWriteProvider: () => ({ id: "local-runtime-v2" }), + }, + }); + const route = await loadRoute(); + let caught = null; + try { + await route.handlePutProviders(fakeReq({ version: 2, providers: [] }), fakeRes(), {}); + } catch (err) { + caught = err; + } + assert.ok(caught, "the route swallowed the facade error — either the mock did not take, or the route grew a catch"); + assert.equal(caught, marker, "the error is the mock's, by identity"); + }); + + test("PROOF: a marker error from the read facade escapes handleGetProviders", async (t) => { + await setupMocks(t, { acp: {} }); + const marker = new Error("B11-READ-MOCK-WAS-NOT-HONOURED"); + t.mock.module(absPath("engine/provider-reads.js"), { + namedExports: { + readEngineProviderCatalogue: async () => { + throw marker; + }, + checkProviderReadCapability: () => ({ + endpoint: "GET /api/providers", + provider: "local-runtime-v2", + capability: "authCredentials", + subItem: "listUserModelProviders", + enforcement: "soft", + gate: "checked", + degraded: false, + reason: null, + }), + }, + }); + const route = await loadRoute(); + let caught = null; + try { + await route.handleGetProviders(null, fakeRes(), {}); + } catch (err) { + caught = err; + } + assert.ok(caught, "the GET route reached its own data plane instead of the facade"); + assert.equal(caught, marker, "the error is the mock's, by identity"); + }); + + test("CONTROL: with no facade mock, PUT runs the real commit and lands in the store", async (t) => { + // The other half of the proof. A `?bust=` re-import under a fresh + // test hook gives a route bound to the REAL facade, so the request + // runs the real plan → commit sequence against the real store. If + // this answered from a mock, the two PROOF cases above would be + // proving nothing. + await setupMocks(t, { acp: {} }); + const route = await loadRoute(); + const res = fakeRes(); + await route.handlePutProviders( + fakeReq({ + version: 2, + providers: [ + { + id: "ctl", + label: "Control", + protocol: "openai", + auth: { type: "byok", apiKey: "sk-control-aaaa", baseURL: "https://ctl" }, + models: [], + }, + ], + }), + res, + {}, + ); + assert.equal(res._status, 200); + const body = getBody(res); + assert.equal(body.ok, true); + assert.deepEqual(body.engineSync.keys, ["ctl"]); + const stored = readStoreRecords(); + assert.equal(stored.length, 1); + assert.equal(stored[0].id, "ctl"); + assert.equal(stored[0].auth.apiKey, "sk-control-aaaa"); + }); +}); diff --git a/packages/webui/test/server/mode-write-501.test.js b/packages/webui/test/server/mode-write-501.test.js index 4f3cf519..a281cc33 100644 --- a/packages/webui/test/server/mode-write-501.test.js +++ b/packages/webui/test/server/mode-write-501.test.js @@ -12,6 +12,18 @@ // POST /api/protocol/set-config-option (generic) → 501 structured // POST /api/protocol/set-config-option (bridged) → 200, unchanged // +// M3-B14 added a THIRD bridged id, `thinkingEffort`, so one row of the +// matrix above moved: `POST /api/protocol/set-config-option` with +// `key: "thinkingEffort"` is no longer the generic example, it is a +// bridged one, and it answers 200 under BOTH transports. The generic +// example is now `contextWindow`. Both the move and the new case are +// asserted here, because a test that quietly keeps the old example +// would report this batch's behaviour change as a regression — and a +// test that quietly drops it would make the change undocumented. +// `test/routes/model.check.mjs` covers the other endpoint of the same +// config id (#58), whose gate is deliberately narrower. +// +// // acp transport — no behaviour change at all // both endpoints, every config id → 200, unchanged // @@ -216,14 +228,20 @@ describe(`M3-B9 · the mode-write endpoints on the ${TRANSPORT} transport`, () = ? "a generic config id answers 501 with the structured capability body" : "a generic config id is untouched on acp", async () => { + // `contextWindow` since M3-B14. It used to be `thinkingEffort`, and + // that is the whole point of this edit: the id this test names as + // the GENERIC one is now a BRIDGED one, so keeping the old example + // would have made the suite assert the batch's own behaviour change + // as a regression. `contextWindow` is the honest generic id — the + // engine has no channel for it either, and no bridge claims one. const r = await post("/api/protocol/set-config-option", { sessionId: SID, - key: "thinkingEffort", - value: "high", + key: "contextWindow", + value: "128000", }); if (!RUNTIME) { assert.equal(r.status, 200); - assert.equal(r.body.key, "thinkingEffort"); + assert.equal(r.body.key, "contextWindow"); assert.equal(rpcCalls.length, 1); return; } @@ -236,6 +254,27 @@ describe(`M3-B9 · the mode-write endpoints on the ${TRANSPORT} transport`, () = }, ); + // M3-B14's behaviour change on #68, over HTTP. The third bridged id is + // `thinkingEffort` -> `setThinkingEffort`, so this exact request used + // to answer 501 and now answers 200 and reaches the engine. Nothing in + // the shipped webapp calls #68, so there is no client to break; what + // the change buys is that the two endpoints agree about a config id + // instead of one delivering it and the other refusing it. + test("M3-B14 — `thinkingEffort` is a BRIDGED id, so #68 delivers it", async () => { + const r = await post("/api/protocol/set-config-option", { + sessionId: SID, + key: "thinkingEffort", + value: "high", + }); + assert.equal(r.status, 200, "both transports: this id is not the generic one any more"); + assert.equal(r.body.ok, true); + assert.equal(r.body.key, "thinkingEffort"); + assert.equal(r.body.value, "high"); + assert.equal("fallback" in r.body, false, "and it is not a 501 being described"); + assert.equal(rpcCalls.length, 1); + assert.equal(rpcCalls[0].configId, "thinkingEffort", "the engine was really reached"); + }); + // ------------------------------------------------------------------------- // The boundary itself: same route, same body, two config ids, two // outcomes. Without this the two cases above could each be passing for @@ -249,8 +288,8 @@ describe(`M3-B9 · the mode-write endpoints on the ${TRANSPORT} transport`, () = }); const generic = await post("/api/protocol/set-config-option", { sessionId: SID, - key: "thinkingEffort", - value: "high", + key: "contextWindow", + value: "128000", }); assert.equal(bridged.body.key, "permissionMode"); assert.equal(generic.body.key === "permissionMode", false); @@ -266,4 +305,30 @@ describe(`M3-B9 · the mode-write endpoints on the ${TRANSPORT} transport`, () = assert.equal(rpcCalls.length, 2); } }); + + test("M3-B14 — all THREE bridged ids are delivered, and only they", async () => { + // The table's whole content, over HTTP, on one provider. Before this + // batch two of these three were 501 under the runtime transport. + for (const key of ["model", "permissionMode", "thinkingEffort"]) { + const r = await post("/api/protocol/set-config-option", { + sessionId: SID, + key, + value: key === "model" ? "m:minimax_api:MiniMax-M2.7:u" : "auto", + }); + assert.equal(r.status, 200, key); + assert.equal(rpcCalls.at(-1).configId, key); + } + const refused = await post("/api/protocol/set-config-option", { + sessionId: SID, + key: "contextWindow", + value: "128000", + }); + if (RUNTIME) { + assert.equal(refused.status, 501, "a non-bridged id is still refused"); + assert.equal(rpcCalls.length, 3, "and it never reached the engine"); + } else { + assert.equal(refused.status, 200, "acp has no provider, so acp changes nothing"); + assert.equal(rpcCalls.length, 4); + } + }); }); diff --git a/packages/webui/test/server/send-run-guard.test.js b/packages/webui/test/server/send-run-guard.test.js index dec9373a..baf8c1f0 100644 --- a/packages/webui/test/server/send-run-guard.test.js +++ b/packages/webui/test/server/send-run-guard.test.js @@ -131,20 +131,43 @@ test("run guard — a draft claim follows its new record id", async (t) => { await t.test("moveRunSession re-points the claim onto the created record", () => { assert.equal(bus.beginRun("tab", null, null).ok, true); assert.equal(bus.moveRunSession("tab", null, "web-new"), true); - assert.equal(bus.getRunForSession("tab", null), null); - assert.ok(bus.getRunForSession("tab", "web-new")); + // P16: the retired key still resolves to the same run. A conversation + // whose id changed under a live turn is still that conversation — the + // next send into it must find the running turn, not a free key. Before + // the alias this read null, and a send arriving in that window was acked + // and handed to an engine session that was already executing, with its + // echo lost from the transcript and the persisted record. + assert.ok( + bus.getRunForSession("tab", null), + "the key the run was claimed under must still resolve to it", + ); + assert.equal( + bus.getRunForSession("tab", null), + bus.getRunForSession("tab", "web-new"), + "both keys must name the one run", + ); // The duplicate send the un-moved claim would have let through. const dup = bus.beginRun("tab", null, "web-new"); assert.equal(dup.ok, false); assert.equal(dup.reason, "cid-busy"); + // …and the same answer when the send presents the RETIRED key. + assert.equal(bus.beginRun("tab", null, null).ok, false); assert.equal(bus.activeRunCount(), 1); }); await t.test("releasing under either key frees the turn exactly once", () => { + // The route's `finally` still holds the key it CLAIMED under, which is + // the retired one once the record has been promoted. An alias-blind + // release would miss here and strand the claim: every later send in + // that conversation would be refused until the process ends. + bus.endRun("tab", null); + assert.equal(bus.activeRunCount(), 0, "releasing under the retired key must free the run"); + assert.equal(bus.getRunForSession("tab", "web-new"), null); + // The current key works too, and a released run leaves no alias behind. + assert.equal(bus.beginRun("tab", null, "web-new").ok, true); bus.endRun("tab", "web-new"); assert.equal(bus.activeRunCount(), 0); - // The stale key is not a live claim. - assert.equal(bus.beginRun("tab", null, null).ok, true); + assert.equal(bus.beginRun("tab", null, null).ok, true, "the retired key is free again"); bus.endRun("tab", null); assert.equal(bus.activeRunCount(), 0); }); diff --git a/packages/webui/webapp/components/composer.tsx b/packages/webui/webapp/components/composer.tsx index 24c5bc5a..b55d88f2 100644 --- a/packages/webui/webapp/components/composer.tsx +++ b/packages/webui/webapp/components/composer.tsx @@ -16,7 +16,11 @@ import { createPortal } from "react-dom"; import * as api from "@/lib/api"; import { clientId } from "@/lib/cid"; import { bridgedControlAvailability, readEngineCapabilities } from "@/lib/engine-capabilities"; -import type { ControlAvailability, EngineCapabilities } from "@/lib/engine-capabilities"; +import type { + BridgedConfigId, + ControlAvailability, + EngineCapabilities, +} from "@/lib/engine-capabilities"; import { effortControlShape, effortOptionsWithDefault, @@ -53,7 +57,7 @@ import { shouldCompleteSlashWord, } from "@/lib/slash-routing"; import { decodeTranscript } from "@/lib/transcript"; -import { isSendUnconfirmed } from "@/lib/api"; +import { isConversationBusy, isSendUnconfirmed } from "@/lib/api"; import { probeSend, shouldRestoreDraft, @@ -262,9 +266,16 @@ export function Composer({ // write, so a visible control would be advertising an action that // cannot happen. See `lib/engine-capabilities.ts` for the fail-open // rule and `webapp/test/engine-capabilities-degradation.test.ts` for - // the coverage of both halves. + // the coverage of all three halves. + // + // M3-B14 added `thinkingControl`. The thinking-effort selector is the + // one whose absence is hardest to spot, because the model chip beside + // it also carries a level on models whose thinking rides the model + // wire form: a hidden effort selector next to a working model chip is + // a correct pair, not a bug. const permissionControl = useEngineControlAvailability("permissionMode"); const modelControl = useEngineControlAvailability("model"); + const thinkingControl = useEngineControlAvailability("thinkingEffort"); const hasConversation = decodeTranscript(state?.chat ?? []).length > 0; /** Nothing to send yet — the send button is rendered but inert. */ const empty = value.trim().length === 0 && attachments.length === 0; @@ -501,6 +512,15 @@ export function Composer({ // (`lib/send-confirmation.ts`), and let that answer decide both the // words and whether the text comes back. const unconfirmed = isSendUnconfirmed(cause); + // A 409 `cid-busy` / `session-busy` is the server saying this + // conversation is already running a turn and the message was NOT + // delivered. It is a refusal — restore the text, and say so in + // words that name the turn rather than in a raw server string + // (P16). It is emphatically NOT the unconfirmed path: nothing is in + // flight on the engine, so the probe is not asked and its "do not + // resend, the engine is running it" wording would be the exact + // opposite of the truth. + const busy = !unconfirmed && isConversationBusy(cause); const outcome: SendProbeOutcome | null = unconfirmed ? await probeSend(content) : null; @@ -565,7 +585,7 @@ export function Composer({ } setComposerDraft(dispatchDraftKey, { error: errorMessage, - errorKind: unconfirmed ? "unconfirmed" : "rejected", + errorKind: unconfirmed ? "unconfirmed" : busy ? "busy" : "rejected", unconfirmed: outcome, }); } finally { @@ -902,8 +922,16 @@ export function Composer({ {/* Thinking-effort picker (ticket 04). Only rendered when the active model carries a `thinkingLevels` list; the picker is gated so models without reasoning controls - never expose a no-op control. */} - {thinkingLevelsForActive.length > 0 ? ( + never expose a no-op control. M3-B14 adds the engine + gate on top of that: `thinkingControl` hides the whole + control when the provider declares no dedicated + thinking-effort writer, which is the same rule the + other two bridged controls follow and the reason a + click here never answers 501. Both halves are in one + condition because both are "may this control be + offered", and nesting them would only make the + degraded case harder to read in a diff. */} + {thinkingControl.available && thinkingLevelsForActive.length > 0 ? ( {error || errorKind ? ( - // Three different facts need three different sentences. An expired + // Four different facts need four different sentences. An expired // deadline is not a refusal, so it never wears the "could not // send" headline nor the error colour — saying either would be a // claim about a side effect that may already have happened, and it - // is what pushed the user into resending (webui-parity 81 D-2). + // is what pushed the user into resending (webui-parity 81 D-2). A + // busy conversation is a refusal too, but the reader is told the + // turn is running and the text was NOT delivered, so the message + // is not resendable yet — a distinct sentence, not the generic + // failure line with a raw server string glued to it (P16). {errorKind === "unconfirmed" ? t(unconfirmedBannerKey(unconfirmedOutcome)) - : `${t("error.send")}: ${error}`} + : errorKind === "busy" + ? t("error.busy") + : `${t("error.send")}: ${error}`} ) : null} @@ -1094,9 +1130,7 @@ const SelectPanel = forwardRef< * control that appears a moment later is worse than one that was always * there, because the user can click it in between. */ -function useEngineControlAvailability( - configId: "model" | "permissionMode", -): ControlAvailability { +function useEngineControlAvailability(configId: BridgedConfigId): ControlAvailability { const [declaration, setDeclaration] = useState(null); useEffect(() => { let live = true; diff --git a/packages/webui/webapp/lib/api.ts b/packages/webui/webapp/lib/api.ts index 9675f4c9..1f7884b9 100644 --- a/packages/webui/webapp/lib/api.ts +++ b/packages/webui/webapp/lib/api.ts @@ -68,6 +68,50 @@ export function isSendUnconfirmed(cause: unknown): cause is SendUnconfirmedError ); } +/** + * A non-2xx answer from the API, with its status and machine-readable + * `reason` kept. + * + * Why the status matters: the composer's banner is chosen by WHAT the + * server decided, not by the wording of its message. A 409 whose `reason` + * is `cid-busy` / `session-busy` is the server saying "this conversation + * is already running, your message was not delivered" — a different fact + * from "your send failed" (retrying is wrong for one, right for the other) + * and a completely different fact from "the engine may already be running + * it, do not resend". Before this type the 409 arrived as a bare + * `new Error(string)`, so the composer could only render it as a generic + * failure and the user read a refused send as a broken one (P16). + * + * The `reason` is optional: an endpoint that answers 4xx without one (a + * malformed body, an older server) still produces a usable `ApiHttpError`, + * and the composer falls back to the generic banner for it. + */ +export class ApiHttpError extends Error { + readonly status: number; + readonly reason: string | null; + + constructor(status: number, message: string, reason: string | null = null) { + super(message); + this.name = "ApiHttpError"; + this.status = status; + this.reason = reason; + } +} + +/** + * The server's stable machine key for "this conversation is already + * running a turn", or null for any other answer. + * + * Read structurally (like `isSendUnconfirmed`) so a second copy of the + * class across module realms still answers correctly. + */ +export function isConversationBusy(cause: unknown): boolean { + if (typeof cause !== "object" || cause === null) return false; + const http = cause as { status?: unknown; reason?: unknown }; + if (http.status !== 409) return false; + return http.reason === "cid-busy" || http.reason === "session-busy"; +} + async function request( path: string, init?: RequestInit & { json?: unknown; timeoutMs?: number }, @@ -114,7 +158,11 @@ async function request( payload && typeof payload === "object" && "error" in payload ? String((payload as { error: unknown }).error) : `HTTP ${response.status}`; - throw new Error(message); + const reason = + payload && typeof payload === "object" && typeof (payload as { reason?: unknown }).reason === "string" + ? String((payload as { reason: string }).reason) + : null; + throw new ApiHttpError(response.status, message, reason); } return payload as T; } diff --git a/packages/webui/webapp/lib/composer-draft.ts b/packages/webui/webapp/lib/composer-draft.ts index 6fa658d4..4bc01d9a 100644 --- a/packages/webui/webapp/lib/composer-draft.ts +++ b/packages/webui/webapp/lib/composer-draft.ts @@ -73,12 +73,20 @@ export interface ComposerDraft { * `rejected` — the server refused the send (4xx, network error before the * request left). A real failure; the text goes back in the box. * + * `busy` — the server refused because THIS CONVERSATION is already running + * a turn (409 `cid-busy` / `session-busy`). The send never reached the + * engine and never will, so it is a refusal like `rejected` — but the + * remedy is "wait for the turn to end", not "the send is broken", and + * conflating the two is what made a refused message read as a lost one + * (P16: a message sent into a running conversation looked like it had + * vanished, and the banner told the user not to resend it). + * * `unconfirmed` — the acknowledgement never arrived and the follow-up read * against the server could not establish whether the turn started. The text * may already be executing. Never rendered as a failure, and the draft is * only restored when the server positively holds no record of the send. */ -export type ComposerErrorKind = "rejected" | "unconfirmed"; +export type ComposerErrorKind = "rejected" | "busy" | "unconfirmed"; const EMPTY_DRAFT: ComposerDraft = { value: "", diff --git a/packages/webui/webapp/lib/engine-capabilities.ts b/packages/webui/webapp/lib/engine-capabilities.ts index 86da78f8..d75fabb3 100644 --- a/packages/webui/webapp/lib/engine-capabilities.ts +++ b/packages/webui/webapp/lib/engine-capabilities.ts @@ -26,16 +26,22 @@ // is the one listed missing. // - the capability is `none` → HIDE. // -// Two controls are the reason this file exists: the permission-mode -// selector and the model selector. Both are declared by -// `MODE_WRITE_BRIDGED_CONFIG_IDS` on the server, the two config ids the -// mode-write gate exempts from the generic-write refusal, so a provider -// that refuses generic config options still serves both. That table is -// mirrored here — one small literal — and +// Three controls are the reason this file exists: the permission-mode +// selector, the model selector and the thinking-effort selector. All +// three are declared by `MODE_WRITE_BRIDGED_CONFIG_IDS` on the server — +// the config ids the mode-write gate exempts from the generic-write +// refusal — so a provider that refuses generic config options still +// serves all three. That table is mirrored here — one small literal — and // `webapp/test/engine-capabilities-degradation.test.ts` reads the server // module's source and fails if the two ever disagree. A mirror without // that tripwire would be exactly the kind of drift this repository has // been bitten by before. +// +// M3-B14 added the third. The thinking-effort selector is the control +// whose absence is hardest to notice if it is missing, because the +// model chip next to it also carries a level on some models: leaving it +// visible and letting it answer 501 is the one outcome this file exists +// to prevent. /** One declared capability, as the server serialises it. */ export interface EngineCapabilityEntry { @@ -48,7 +54,7 @@ export interface EngineCapabilityEntry { export type EngineCapabilities = Record | null; /** - * The two config ids the mode-write gate bridges, and the engine + * The three config ids the mode-write gate bridges, and the engine * sub-item each asks for instead of the generic one. * * Mirrors `MODE_WRITE_BRIDGED_CONFIG_IDS` in @@ -58,9 +64,10 @@ export type EngineCapabilities = Record | null; export const BRIDGED_CONFIG_SUB_ITEMS = Object.freeze({ model: "selectModel", permissionMode: "setPermissionMode", + thinkingEffort: "setThinkingEffort", } as const); -/** The two config ids with a dedicated engine write behind them. */ +/** The config ids with a dedicated engine write behind them. */ export type BridgedConfigId = keyof typeof BRIDGED_CONFIG_SUB_ITEMS; /** What a control should do, and why — `reason` is for logs, not for the user. */ @@ -108,7 +115,7 @@ export function controlAvailability( return { available: true, reason: null }; } -/** `controlAvailability` for one of the two bridged controls. */ +/** `controlAvailability` for one of the three bridged controls. */ export function bridgedControlAvailability( declaration: EngineCapabilities, configId: BridgedConfigId, @@ -119,8 +126,8 @@ export function bridgedControlAvailability( /** * Read the declaration once per page and share it. * - * Module-level cache with an in-flight promise, because the two controls - * mount together and a per-component fetch would double the request on + * Module-level cache with an in-flight promise, because the three controls + * mount together and a per-component fetch would triple the request on * every composer mount. The cache is deliberately NOT invalidated: a * provider's declaration does not change while the page is open, and a * poller here would be a new failure surface for no benefit. diff --git a/packages/webui/webapp/lib/i18n.ts b/packages/webui/webapp/lib/i18n.ts index 94986301..5f68d921 100644 --- a/packages/webui/webapp/lib/i18n.ts +++ b/packages/webui/webapp/lib/i18n.ts @@ -156,6 +156,14 @@ const en = { "ask.title": "Question", "error.send": "Could not send the message", + /* P16 — the third send state. A message sent into a conversation that is + already running is REFUSED by the server (409 cid-busy / session-busy): + the engine never receives it, so the text comes back to the box and the + only thing left to say is that the turn has to finish first. Distinct + from `error.send` (the send is not broken) and from the unconfirmed + banners (nothing is in flight — resending is safe and necessary). */ + "error.busy": + "Not delivered: this conversation is already running a turn. The text is back in the input box — send it again once the turn finishes.", "error.unconfirmed.accepted": "Sent, but the server never confirmed it. The engine is running this message now — do not send it again.", "error.unconfirmed.rejected": @@ -1293,6 +1301,12 @@ const zh: Record = { "ask.title": "提问", "error.send": "消息发送失败", + /* P16 —— 第三种发送态。会话进行中发的消息由服务器明确拒绝（409 + cid-busy / session-busy）：引擎根本没有收到，原文回到输入框，要说的只 + 有「等本回合跑完再发」。既不是 error.send（发送没坏），也不是下面三条 + unconfirmed（引擎侧没有任何东西在跑，重发是安全且必要的）。 */ + "error.busy": + "未送达：本会话正在跑一个回合。原文已放回输入框 —— 等回合结束后再发送即可。", "error.unconfirmed.accepted": "消息已发出，但服务器一直没有确认。引擎此刻正在执行这条消息 —— 请勿重复发送。", "error.unconfirmed.rejected": diff --git a/packages/webui/webapp/lib/send-confirmation.ts b/packages/webui/webapp/lib/send-confirmation.ts index 3dcfd668..768384c4 100644 --- a/packages/webui/webapp/lib/send-confirmation.ts +++ b/packages/webui/webapp/lib/send-confirmation.ts @@ -61,19 +61,37 @@ function echoForms(content: string): string[] { /** * Does this state show the send we asked about as taken? * - * Two independent signals, either sufficient: + * The proof is the prompt's echo line: `handleSend` writes `› ` + * into the transcript synchronously, before the turn does any work, so a + * line that is there is a send the server accepted — whether the turn is + * still running or already finished. * - * 1. A turn is running for this cid. Since a busy cid answers the send with a - * 409 immediately (`beginRun` in `handleSend`), a turn observed after a - * deadline expiry is this one. - * 2. The prompt's echo line is in the transcript — proof the server accepted - * it, whether or not the turn has already finished. + * A running turn is NOT that proof, and treating it as one is the false + * positive this function used to produce (P16). The reasoning behind the + * old first signal — "a busy cid answers with 409 immediately, so a turn + * observed after a deadline expiry is this one" — is wrong for the case + * that actually produced the report: the send was made INTO a running + * conversation, so the 409 that came back belonged to a turn that was + * already running, and the running turn the probe sees is the PREVIOUS + * one. Reading it as acceptance answered "the engine is running your + * message, do not send it again" for a message the engine never received + * and the transcript never recorded — a false negative about a side + * effect, which is the one thing the banner must never state. + * + * The running flag is still consulted, but only where it is the only + * evidence there is: a state snapshot with no transcript at all (a + * server that answered but carries no `chat`). There the running flag + * cannot be contradicted, and a real in-flight turn is the best available + * answer — refusing to restore the text in that case is what webui-parity + * 81 D-2 (`sleep 35` executed twice) was about. */ export function stateAcceptsSend(state: WebuiState, content: string): boolean { - if (state && state.running && state.running.active === true) return true; - const chat = state && Array.isArray(state.chat) ? state.chat : []; - const forms = echoForms(content); - return chat.some((line) => forms.includes(line)); + const chat = state && Array.isArray(state.chat) ? state.chat : null; + if (chat !== null) { + const forms = echoForms(content); + return chat.some((line) => forms.includes(line)); + } + return Boolean(state && state.running && state.running.active === true); } /** diff --git a/packages/webui/webapp/test/engine-capabilities-degradation.test.ts b/packages/webui/webapp/test/engine-capabilities-degradation.test.ts index 01197dd3..e385805a 100644 --- a/packages/webui/webapp/test/engine-capabilities-degradation.test.ts +++ b/packages/webui/webapp/test/engine-capabilities-degradation.test.ts @@ -19,10 +19,12 @@ // including the ones that must NOT hide anything. The rule is // fail-open, and the cases below are what make that true rather than // accidental. -// 2. THE BRIDGE MIRROR. The frontend names two engine sub-items the +// 2. THE BRIDGE MIRROR. The frontend names three engine sub-items the // server also names. Two hand-maintained copies of a set of engine // identifiers drift; the tripwire reads the server module's SOURCE -// and fails when the two disagree. +// and fails when the two disagree. M3-B14 added the third, and with +// it the case this file exists to prevent: a control that is VISIBLE +// and answers 501 on click, because the mirror lost an entry. // 3. THE WIRING. `components/composer.tsx` is a client component with // no render harness in this suite, so its half is a static-source // tripwire — the form this repository allows when no harness exists @@ -104,9 +106,10 @@ describe("controlAvailability — the fail-open rule", () => { const caps = { authCredentials: V2_MODE_KEYS.authCredentials } as EngineCapabilities; // The generic write is gone… assert.equal(controlAvailability(caps, "authCredentials", "setConfigOption").available, false); - // …and the two dedicated writers are not what it denied. + // …and the three dedicated writers are not what it denied. assert.equal(controlAvailability(caps, "authCredentials", "selectModel").available, true); assert.equal(controlAvailability(caps, "authCredentials", "setPermissionMode").available, true); + assert.equal(controlAvailability(caps, "authCredentials", "setThinkingEffort").available, true); }); test("a `partial` with no `missing` array shows the control", () => { @@ -132,39 +135,71 @@ describe("controlAvailability — the fail-open rule", () => { }); }); -describe("bridgedControlAvailability — the two controls the composer renders", () => { - test("both are available on the v2 declaration this batch ships", () => { +describe("bridgedControlAvailability — the three controls the composer renders", () => { + const ALL_IDS = ["model", "permissionMode", "thinkingEffort"] as const; + + test("all three are available on the v2 declaration this batch ships", () => { + // M3-B14's behaviour change, stated as the half that is a no-op + // today: the effort control is NOT hidden under the declaration the + // shipped providers actually send, because they deny only the + // generic write. const caps = { authCredentials: V2_MODE_KEYS.authCredentials } as EngineCapabilities; - for (const id of ["model", "permissionMode"] as const) { + for (const id of ALL_IDS) { assert.equal(bridgedControlAvailability(caps, id).available, true, id); } }); - test("both are hidden when the capability is `none`", () => { + test("all three are hidden when the capability is `none`", () => { const caps = { authCredentials: { level: "none", reason: "test: interface-absent" }, } as EngineCapabilities; - for (const id of ["model", "permissionMode"] as const) { + for (const id of ALL_IDS) { assert.equal(bridgedControlAvailability(caps, id).available, false, id); } }); - test("a provider that denies the DEDICATED writer hides that control and keeps the other", () => { + test("a provider that denies the DEDICATED writer hides that control and keeps the others", () => { // The bridge is per sub-item, so a provider can have one without the - // other — and the composer must not hide both because one is gone. - const caps = { - authCredentials: { level: "partial", missing: ["selectModel"], reason: "test: no model writer" }, - } as EngineCapabilities; - assert.equal(bridgedControlAvailability(caps, "model").available, false); - assert.equal(bridgedControlAvailability(caps, "permissionMode").available, true); + // others — and the composer must not hide all three because one is + // gone. The effort control is the one this batch added, and it is + // also the one a "deny one, hide the panel" rewrite would take with + // it, so every other id is asserted here explicitly. + for (const denied of ALL_IDS) { + const caps = { + authCredentials: { level: "partial", missing: [BRIDGED_CONFIG_SUB_ITEMS[denied]] }, + } as EngineCapabilities; + for (const id of ALL_IDS) { + assert.equal( + bridgedControlAvailability(caps, id).available, + id !== denied, + `${denied} denied, so ${id} should be ${id !== denied}`, + ); + } + } + }); + + test("denying the GENERIC write hides nothing", () => { + // The other direction, and the one that makes the bridge worth + // having: a provider that has no `setConfigOption` at all still + // serves all three dedicated writers, so no control disappears. + const caps = { authCredentials: V2_MODE_KEYS.authCredentials } as EngineCapabilities; + assert.equal(controlAvailability(caps, "authCredentials", "setConfigOption").available, false); + for (const id of ALL_IDS) { + assert.equal(bridgedControlAvailability(caps, id).available, true, id); + } }); test("an unknown config id is not a bridge — it must not inherit the exemption", () => { // Mirrors the server's own guard: a name nobody audited falls back // to the generic sub-item rather than the exemption. + // + // M3-B14: this used to be asserted with `thinkingEffort`, which is + // exactly the mutation that had to go red — so the example is now + // `contextWindow`, an id no bridge claims. Pinning the test with a + // name the batch legitimately adds would have made the suite lie. const caps = { authCredentials: V2_MODE_KEYS.authCredentials } as EngineCapabilities; const subItem = (BRIDGED_CONFIG_SUB_ITEMS as Record)[ - "thinkingEffort" + "contextWindow" ]; assert.equal(subItem, undefined); assert.equal( @@ -175,11 +210,15 @@ describe("bridgedControlAvailability — the two controls the composer renders", }); }); -describe("the frontend and the server name the same two engine sub-items", () => { +describe("the frontend and the server name the same three engine sub-items", () => { // The mirror is one small literal, and this is what keeps it honest. // A rename on either side without the other is exactly the drift the // server module's own header warns about. test("BRIDGED_CONFIG_SUB_ITEMS matches MODE_WRITE_BRIDGED_CONFIG_IDS in the server source", () => { + // The tripwire M3-B14 most needed: one entry added on each side and + // one forgotten, and the composer grows a control that answers 501. + assert.equal(Object.keys(BRIDGED_CONFIG_SUB_ITEMS).length, 3); + const block = serverModeWritesSource.match( /MODE_WRITE_BRIDGED_CONFIG_IDS\s*=\s*Object\.freeze\(\{([\s\S]*?)\}\)/, ); @@ -219,7 +258,7 @@ describe("the composer actually gates on it", () => { // suite has no render harness for it. Weak by construction, and stated // as such — what it catches is the realistic regression, which is a // later edit that drops the gate while leaving the lib alone. - test("both controls are wrapped in their availability check", () => { + test("all three controls are wrapped in their availability check", () => { assert.match( composerSource, /\{permissionControl\.available \? \(\s* { /\{modelControl\.available \? \(\s* 0 \? \(\s* 0/, + "the levels precondition must survive the capability gate", + ); }); test("the availability comes from the shared rule, not from a local reading of a 501", () => { assert.match(composerSource, /useEngineControlAvailability\("permissionMode"\)/); assert.match(composerSource, /useEngineControlAvailability\("model"\)/); + assert.match(composerSource, /useEngineControlAvailability\("thinkingEffort"\)/); assert.match( composerSource, /bridgedControlAvailability\(declaration, configId\)/, @@ -254,9 +309,23 @@ describe("the composer actually gates on it", () => { // A 4000-character sweep matches the next unrelated `disabled` prop // in the file and fails for a reason that has nothing to do with the // gate — which trains a reader to ignore this assertion. - for (const [marker, control] of [ - ["{permissionControl.available ? (", "PermissionSelect"], - ["{modelControl.available ? (", "ModelSelect"], + // + // M3-B14 added the third control and with it a distinction the first + // two did not force: `ThinkingEffortSelect` carries a PRE-EXISTING + // `disabled={running}` prop, which has nothing to do with the + // capability (it is "a turn is in flight"). So the `disabled` sweep + // is per control, and the effort control gets a stronger assertion + // instead of a weaker one: its only `disabled` must be the running + // one. A rewrite that degraded the gate into `disabled={...}` would + // still fail here. + for (const [marker, control, onlyDisabled] of [ + ["{permissionControl.available ? (", "PermissionSelect", null], + ["{modelControl.available ? (", "ModelSelect", null], + [ + "{thinkingControl.available && thinkingLevelsForActive.length > 0 ? (", + "ThinkingEffortSelect", + "disabled={running}", + ], ] as const) { const start = composerSource.indexOf(marker); assert.ok(start > 0, `${control}: the availability gate is gone`); @@ -264,14 +333,43 @@ describe("the composer actually gates on it", () => { assert.ok(end > start, `${control}: the gate no longer ends in \`: null\``); const block = composerSource.slice(start, end); assert.ok(block.includes(`<${control}`), `${control}: the gate does not wrap the control`); - assert.doesNotMatch(block, /\bdisabled\b/, `${control}: hidden, not disabled`); + if (onlyDisabled === null) { + assert.doesNotMatch(block, /\bdisabled\b/, `${control}: hidden, not disabled`); + } else { + const hits = [...block.matchAll(/\bdisabled=[^\s/>]+/g)].map((m) => m[0]); + assert.deepEqual(hits, [onlyDisabled], `${control}: only the pre-existing prop, never a capability one`); + } assert.doesNotMatch(block, /fallback|toast/i, `${control}: hidden, with no degraded rendering`); } }); - test("there are exactly two gates, and both hide", () => { - const gates = [...composerSource.matchAll(/(?:permission|model)Control\.available \? \(/g)]; - assert.equal(gates.length, 2, "expected exactly two availability gates"); + test("the effort control's `disabled` is about a running turn, not about the capability", () => { + // Stated separately because it is the one place in this file where a + // reader could reasonably think a `disabled` IS the degradation. It + // is not: `running` is a turn-state flag with a decade of history, + // and the capability degradation is the `? … : null` around it. If a + // future change makes the disabled prop read the declaration, this + // fails. + const marker = "{thinkingControl.available && thinkingLevelsForActive.length > 0 ? ("; + const start = composerSource.indexOf(marker); + const end = composerSource.indexOf(") : null}", start); + const block = composerSource.slice(start, end); + assert.doesNotMatch( + block, + /disabled=\{[^}]*([Aa]vailability|declaration|thinkingControl)/, + "the disabled prop must not be derived from the capability declaration", + ); + }); + + test("there are exactly three gates, and all three hide", () => { + // A sweep rather than a count per control, so a FOURTH gate added + // later fails here instead of quietly becoming a fourth place for + // the rule to be interpreted. + const gates = [ + ...composerSource.matchAll(/(?:permission|model)Control\.available \? \(/g), + ...composerSource.matchAll(/thinkingControl\.available && /g), + ]; + assert.equal(gates.length, 3, "expected exactly three availability gates"); }); }); @@ -325,7 +423,7 @@ describe("readEngineCapabilities — never throws, and reads once", () => { assert.equal(await readEngineCapabilities(), null); }); - test("two controls cost one request — the declaration is shared", async () => { + test("three controls cost one request — the declaration is shared", async () => { let calls = 0; globalThis.fetch = (async () => { calls += 1; @@ -333,7 +431,7 @@ describe("readEngineCapabilities — never throws, and reads once", () => { }) as typeof fetch; const [a, b] = await Promise.all([readEngineCapabilities(), readEngineCapabilities()]); const c = await readEngineCapabilities(); - assert.equal(calls, 1, "the composer mounts two controls; it must not make two requests"); + assert.equal(calls, 1, "the composer mounts three controls; it must not make three requests"); assert.equal(a, b); assert.equal(b, c, "the cache must hand back the same declaration, not a fresh fetch"); }); diff --git a/packages/webui/webapp/test/send-busy-state.test.ts b/packages/webui/webapp/test/send-busy-state.test.ts new file mode 100644 index 00000000..62b770ee --- /dev/null +++ b/packages/webui/webapp/test/send-busy-state.test.ts @@ -0,0 +1,88 @@ +// webapp/test/send-busy-state.test.ts +// +// P16 — sending into a conversation that is already running. +// +// The server refuses that send (409 `cid-busy` / `session-busy`) and never +// hands it to the engine. Before this the refusal arrived at the composer as +// a bare `new Error(string)`, so it rendered as the generic "消息发送失败" +// line with an internal English string glued to it, and the user read a +// refused message as a broken one. The state the banner must show is a +// THIRD one, distinct from both "could not send" and the unconfirmed +// banners: nothing is in flight on the engine, the text came back, and the +// only instruction is "send it again when the turn finishes". +// +// The invariants pinned here: +// 1. a 409 carrying the busy `reason` is recognised structurally, not by +// message text, and not by `instanceof`; +// 2. a 409 with any other `reason` (at-capacity) and every other non-2xx +// are NOT "busy" — collapsing them would repeat the same lie; +// 3. the two locales both carry the new key, and neither reuses the +// unconfirmed wording that forbids resending. + +import { test, describe } from "node:test"; +import assert from "node:assert/strict"; + +import { ApiHttpError, isConversationBusy, isSendUnconfirmed } from "../lib/api"; +import { translate, type Locale } from "../lib/i18n"; + +const BUSY_REASONS = ["cid-busy", "session-busy"] as const; + +describe("isConversationBusy", () => { + test("a 409 with a busy reason is a busy conversation", () => { + for (const reason of BUSY_REASONS) { + assert.equal( + isConversationBusy(new ApiHttpError(409, "not delivered", reason)), + true, + `${reason} must read as busy`, + ); + } + }); + + test("a structural object reads as busy without instanceof", () => { + // Two module realms (the app bundle and a re-bundled copy) would make + // `instanceof ApiHttpError` answer false for a real 409 — the same + // trap `isSendUnconfirmed` exists to avoid. + const crossRealm = { status: 409, reason: "session-busy" }; + assert.equal(isConversationBusy(crossRealm), true); + }); + + test("a capacity refusal is not a busy conversation", () => { + assert.equal( + isConversationBusy(new ApiHttpError(409, "server is at capacity", "at-capacity")), + false, + ); + }); + + test("only 409 with a busy reason qualifies", () => { + assert.equal(isConversationBusy(new ApiHttpError(400, "content required")), false); + assert.equal(isConversationBusy(new Error("a turn is already running for this session")), false); + assert.equal(isConversationBusy(null), false); + assert.equal(isConversationBusy("cid-busy"), false); + }); + + test("a busy refusal is never the unconfirmed deadline error", () => { + // The two are opposites: one means the engine has nothing, the other + // means the engine may already be running it. The composer branches on + // them separately, so neither may satisfy the other's check. + const busy = new ApiHttpError(409, "not delivered", "cid-busy"); + assert.equal(isSendUnconfirmed(busy), false); + assert.equal(isConversationBusy(new (class extends Error {})()), false); + }); +}); + +describe("error.busy copy", () => { + for (const locale of ["en", "zh"] as Locale[]) { + test(`${locale}: present, and does not forbid resending`, () => { + const text = translate(locale, "error.busy"); + assert.ok(text.length > 0, "the key must resolve in every locale"); + assert.ok( + !text.includes("请勿重复发送") && !/do not send it again/i.test(text), + "the busy banner must NOT tell the user not to resend — nothing is running", + ); + assert.ok( + /未送达|not delivered/i.test(text), + "the busy banner must state the message was not delivered", + ); + }); + } +}); diff --git a/packages/webui/webapp/test/send-confirmation.test.ts b/packages/webui/webapp/test/send-confirmation.test.ts index a7203ae6..64fb7c6a 100644 --- a/packages/webui/webapp/test/send-confirmation.test.ts +++ b/packages/webui/webapp/test/send-confirmation.test.ts @@ -74,8 +74,29 @@ describe("the acknowledgement deadline is typed, not worded", () => { describe("stateAcceptsSend — what counts as 'the engine took it'", () => { const CASES: { name: string; state: WebuiState; content: string; want: boolean }[] = [ { - name: "a running turn is accepted", - state: stateOf({ running: true, chat: [] }), + name: "a running turn carrying this send's echo is accepted", + state: stateOf({ running: true, chat: ["› sleep 35"] }), + content: "sleep 35", + want: true, + }, + { + // P16 — the false positive. This send was made INTO a running + // conversation, so the 409 that came back belonged to a turn that was + // already running: the running turn the probe sees is the PREVIOUS + // one, and the transcript is the only thing that can tell them apart. + // Reading the flag as acceptance answered "the engine is running your + // message, do not send it again" for a message the engine never got. + name: "a running turn with NO trace of this send is not accepted", + state: stateOf({ running: true, chat: ["› ping", "● pong"] }), + content: "sleep 35", + want: false, + }, + { + // The one place the running flag still decides: a snapshot that + // carries no transcript at all cannot contradict it, and webui-parity + // 81 D-2 (`sleep 35` ran twice) is the price of ignoring it there. + name: "a state with no transcript at all falls back to the running flag", + state: { running: { active: true } } as unknown as WebuiState, content: "sleep 35", want: true, }, @@ -127,13 +148,21 @@ describe("stateAcceptsSend — what counts as 'the engine took it'", () => { }); describe("classifySendProbe — the decision table", () => { - const RUNNING = stateOf({ running: true }); + // A turn that is running AND holds this send's echo — the proof, not the + // flag. A read showing only a running turn is the P16 false positive. + const RUNNING = stateOf({ running: true, chat: ["› x"] }); + const RUNNING_OTHER_TURN = stateOf({ running: true, chat: ["› earlier"] }); const IDLE_EMPTY = stateOf({ chat: [] }); test("one read showing the turn running settles it as accepted", () => { assert.equal(classifySendProbe([null, RUNNING], "x"), "accepted"); }); + test("a running turn that never took this send is a rejection, not acceptance", () => { + assert.equal(classifySendProbe([RUNNING_OTHER_TURN], "x"), "rejected"); + assert.equal(shouldRestoreDraft(classifySendProbe([RUNNING_OTHER_TURN], "x")), true); + }); + test("a read that came back with no trace is a rejection", () => { assert.equal(classifySendProbe([IDLE_EMPTY], "x"), "rejected"); }); @@ -160,7 +189,7 @@ describe("probeSend — bounded, and it really reads", () => { const outcome = await probeSend("x", { read: async () => { reads += 1; - return stateOf({ running: reads === 2 }); + return stateOf({ running: reads === 2, chat: ["› x"] }); }, delay: async (ms) => { waits.push(ms); @@ -183,7 +212,7 @@ describe("probeSend — bounded, and it really reads", () => { const outcome = await probeSend("x", { read: async () => { reads += 1; - return stateOf({ running: true }); + return stateOf({ running: true, chat: ["› x"] }); }, delay: async () => {}, }); diff --git a/release/public-source.json b/release/public-source.json index 456b5662..273649e2 100644 --- a/release/public-source.json +++ b/release/public-source.json @@ -3457,6 +3457,9 @@ "packages/webui/server/engine/mode-writes.js", "packages/webui/server/engine/model-reads.js", "packages/webui/server/engine/model-writes.js", + "packages/webui/server/engine/provider-reads.js", + "packages/webui/server/engine/provider-store.js", + "packages/webui/server/engine/provider-writes.js", "packages/webui/server/engine/providers/local-runtime-v2.capabilities.js", "packages/webui/server/engine/providers/local-runtime-v2.js", "packages/webui/server/engine/providers/tui-runtime-adapter.js", @@ -3482,7 +3485,6 @@ "packages/webui/server/lib/context-percent.js", "packages/webui/server/lib/credential-file.js", "packages/webui/server/lib/engine-catalogue.js", - "packages/webui/server/lib/engine-provider-sync.js", "packages/webui/server/lib/events.js", "packages/webui/server/lib/feedback/command-feedback.js", "packages/webui/server/lib/feedback/message-feedback.js", @@ -3588,6 +3590,7 @@ "packages/webui/test/integration/upload-limits.test.js", "packages/webui/test/lib/acp-cache.check.mjs", "packages/webui/test/lib/acp-client-requests.test.js", + "packages/webui/test/lib/acp-stderr-tail.test.js", "packages/webui/test/lib/acp-transport-answer.test.js", "packages/webui/test/lib/acp-turn-message-id.test.js", "packages/webui/test/lib/agent-team-detect.test.js", @@ -3603,7 +3606,6 @@ "packages/webui/test/lib/config.test.js", "packages/webui/test/lib/context-percent.test.js", "packages/webui/test/lib/engine-catalogue.test.js", - "packages/webui/test/lib/engine-provider-sync.test.js", "packages/webui/test/lib/engine/account-reads.test.js", "packages/webui/test/lib/engine/capabilities.test.js", "packages/webui/test/lib/engine/capability-reads.test.js", @@ -3613,6 +3615,11 @@ "packages/webui/test/lib/engine/mode-writes.test.js", "packages/webui/test/lib/engine/model-reads.test.js", "packages/webui/test/lib/engine/model-writes.test.js", + "packages/webui/test/lib/engine/provider-migration.test.js", + "packages/webui/test/lib/engine/provider-reads.test.js", + "packages/webui/test/lib/engine/provider-store-ownership.test.js", + "packages/webui/test/lib/engine/provider-store.test.js", + "packages/webui/test/lib/engine/provider-writes.test.js", "packages/webui/test/lib/engine/session-export.test.js", "packages/webui/test/lib/engine/session-load.test.js", "packages/webui/test/lib/engine/session-reads.test.js", @@ -3678,6 +3685,7 @@ "packages/webui/test/routes/alerts.check.mjs", "packages/webui/test/routes/chat-failed-send.check.mjs", "packages/webui/test/routes/chat-first-turn-session-guard.check.mjs", + "packages/webui/test/routes/chat-inflight-send.check.mjs", "packages/webui/test/routes/chat-run-mirror.check.mjs", "packages/webui/test/routes/chat.check.mjs", "packages/webui/test/routes/debug.check.mjs", @@ -3981,6 +3989,7 @@ "packages/webui/webapp/test/plugins-surface.test.ts", "packages/webui/webapp/test/preview-edit.test.ts", "packages/webui/webapp/test/provider-management.test.ts", + "packages/webui/webapp/test/send-busy-state.test.ts", "packages/webui/webapp/test/send-confirmation.test.ts", "packages/webui/webapp/test/session-row-hover-tray.test.ts", "packages/webui/webapp/test/session-switch-visibility.test.ts", diff --git a/scripts/test-tmp-leak.check.mjs b/scripts/test-tmp-leak.check.mjs index 0520ba7d..82c08edd 100644 --- a/scripts/test-tmp-leak.check.mjs +++ b/scripts/test-tmp-leak.check.mjs @@ -256,11 +256,15 @@ const KNOWN_PREFIXES = [ "mcode-webui-w2-cmd-", "mcode-webui-w2-gate-", "minimax-code-engine-cat-", - "minimax-code-engine-sync-", + "minimax-code-engine-migration-", + "minimax-code-engine-reads-", + "minimax-code-engine-store-", + "minimax-code-engine-writes-", "sessions-single-id-", "state-bus-restore-", "webui-acp-answer-", "webui-acp-fake-engine-", + "webui-acp-stderr-", "webui-alerts-audit-", "webui-alerts-check-", "webui-authgate-events-", @@ -298,8 +302,10 @@ const KNOWN_PREFIXES = [ "webui-parent-root-", "webui-paths-", "webui-plan-projection-", + "webui-presets-engine-", "webui-presets-route-", "webui-presets-route-cwd-", + "webui-providers-engine-", "webui-providers-cwd-", "webui-providers-route-", "webui-providers-route-cwd-",