diff --git a/AGENTS.md b/AGENTS.md index d2d96fd7c7..4be4aed2af 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -105,5 +105,9 @@ passing; rebuilding coverage for the area it covered happens under CL-8150. - [VENDORED.md](VENDORED.md) — the vendoring ledger and its rules - [DESIGN.md](DESIGN.md) — the UI design system canon; a screen that disagrees with it is wrong until a review changes the doc -- `docs/` — architecture and design docs; owner rulings and internal - decisions belong in Linear, not a doc here +- [docs/GLOSSARY.md](docs/GLOSSARY.md) — product-term to platform-term mapping +- [docs/local-dev.md](docs/local-dev.md) — running fully local with Ollama +- [docs/sidecar-provisioners.md](docs/sidecar-provisioners.md) — choosing + and configuring where sidecars run +- [docs/package-migrations.md](docs/package-migrations.md) — how a + package's own database migrations are written and applied diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 8f08f7338a..2641b3ae14 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -1,292 +1,61 @@ # Architecture -This document describes how Corbits Workbench is structured, independent of -specific technology choices — see [IMPLEMENTATION.md](IMPLEMENTATION.md) -for the concrete stack. - -## Repo shape - -| Layer | What lives here | -| ------------- | ----------------------------------------------------------------------- | -| `apps/` | Deployable services: hub (API), sidecar (execution host), web (UI) | -| `packages/` | Domain packages — where product rules live | -| `workflows/` | Workflow definition packages, deployed as assets | -| `vendor/intx` | Hand-copied, ledgered `@intx/*` source — see [VENDORED.md](VENDORED.md) | - -Packages own product rules. Apps stay generic — they compose packages -rather than own domain logic — with one tracked exception: `apps/hub` -carries a small, explicitly-listed set of mounts (`artifacts-mount.ts`, -`memory-mount.ts`, `skills-mount.ts`, `tenant-create-guard.ts`) pending -extraction into packages (tracked as -CL-6127). Outside that list, a product rule that lives inside `apps/*` is -a defect — it belongs in a package. - -## Interchange is the platform - -Everything about identity, credentials, agent execution, and workflow -orchestration belongs to Interchange (`@intx/*`), never reimplemented in -this repo: - -- **Tenancy** — the tenant hierarchy, membership, principals, roles, and - grants (`@intx/db`, `@intx/hub-api`). -- **Credentials** — resolution and storage. -- **Agents** — Interchange owns launch (`prepareProvisionedDeployment`), - the run principal, and session primitives (`@intx/agent`, - `@intx/hub-agent`, `@intx/hub-sessions`). -- **Inference** — LLM calls (`@intx/inference`). -- **Workflow runtime** — definitions, runs, and the workflow host - (`@intx/workflow`, `@intx/workflow-host`, `@intx/workflow-deploy`). -- **Mail** — the transport an agent turn uses to receive and send - messages. A workbench itself is not a mail-holding host run. - -Workbench adds product contracts on top of these primitives; it never -forks or patches Interchange internals. See [docs/TENANCY.md](docs/TENANCY.md) -for the authoritative list of what Interchange already provides versus -what is a genuine upstream gap workbench has had to work around -product-side. - -## Agent launch - -Workbench is a client of Interchange. Launching an agent is Interchange -`prepareProvisionedDeployment` onto that instance's own `kind:workflow` -asset (default ref) in the hub git store. Interchange mints the run -row, the allocation, and — on first trigger — the run principal. Chat -records the returned run id, deployment id, and mail address. It does -not pre-mint an anchor run, own an `agent_session`, or render a -per-run `sourceRef`. `@corbits/folded-runs` and -`@corbits/folded-run-one-shot` are gone. A one-shot wait-loop for Myra -drafting lives in `@corbits/agent-directory` as `runOneShotPrompt` and -still calls `prepareProvisionedDeployment`. - -Launch is asynchronous. The sidecar reconciler's ready hook and the -`workflowDispatchService` queue (wired in `apps/hub`) deliver a -message sent before the instance is ready; the UI shows a starting -state until then. Presence while starting comes from allocation and -deployment status, not a run principal. Wake of a reaped or lost -instance is a fresh `prepareProvisionedDeployment` onto the same -asset; chat updates the recorded ids. - -A turn is one native `sendUserMessage` per addressed agent, carrying -Message-ID, In-Reply-To, and References. Agent replies reach humans -through the hub `persistMail` wrapper into `@corbits/mailbox`. Turns -do not use a correlationId. Human attachments are read from the -mailbox frame, not Interchange `session_mail`. Inference sources ride -as catalog offering ids (`sourceOfferingIds` / -`defaultSourceOfferingId`). Catalog instantiate does not 409 -credential-binding workflows. - -## Tenancy model - -A **bench** (the shared team scope) and a **workbench** (a single -conversation) are both Interchange tenants underneath — a bench is simply -a tenant nothing else is parented under as a workbench, in practice the -one a person signs into. A workbench mints its own child tenant at -creation time, parented under the bench, so its membership and grants are -native and independent of the bench's own. - -Parent tenancy is largely invisible plumbing to a person using the -product: they experience "the bench" and "the workbench they're in," not -tenant ids or parent chains. Underneath, inheritance is live — plugins, -credentials, and catalog data resolve by walking the ancestor chain on -every read, never copied down at creation time. A sub-workbench or child -tenant never gets a snapshot of its parent's catalog; it always sees the -parent's current state. - -The workbench host's default inference model is chosen from that -visible catalog: tenant-owned (direct) completion offerings beat -inherited catalog seeds when any exist; an inherit-only catalog still -picks the inherited completion default. Embedding models stay -ineligible. This is a preference among what the ancestor walk already -shows, not a snapshot and not a second catalog. - -See [docs/TENANCY.md](docs/TENANCY.md) for the full contract, including -the workbench-owned discriminator that distinguishes a "real" bench from a -workbench's own child tenancy (native tenants carry no `kind` field), and -[docs/GLOSSARY.md](docs/GLOSSARY.md) for the term mapping between product -nouns and platform primitives. - -## Conversation as workbench data - -A workbench is not a parallel messaging system bolted onto Interchange, -and it is not itself a workflow run. Creating a workbench mints a child -tenant and writes settings rows — no deploy, no host, no anchor instance -(`provisionSpaceWorkbench` and the chat create path). A workbench holds a -**timeline** of `chat.workbench_messages` rows; posting is one insert plus -one publish onto the workbench's live stream, so the conversation takes -messages whether or not any agent process is running. - -**Editing an own prompt.** Edit is a composer replace, not a timeline -mutation and not a thread fork. The timeline offers Edit only on the -signed-in reader's own prompts that have text -(`packages/chat-ui/src/timeline.tsx`). Choosing it copies that message's -text into the composer through `ComposerHandle.setText` -(`packages/chat-ui/src/composer.tsx`). `setText` replaces the draft and -clears leftover composer-private state (slash, mention, pending invites, -attachments, in-flight attachment reads). Sending is the ordinary post -onto the timeline already in view (root or an open thread). It does not -PATCH the origin message and it does not call `forkThread`. - -A workbench's address is derived, not resolved. Asking an invited agent -for a turn is a separate act over Interchange mail; the agent replies by -emitting `connector.reply` events on its own stream, and a reply bridge -turns those events into timeline messages. See -[docs/CHAT.md](docs/CHAT.md) for the full message, thread, and -participant model. Per-workbench settings (name, participants, capacity, -connector overrides, notification prefs — see PRODUCT.md's "Workbench -settings") are composition on top of this tenant and its settings rows, -not a separate object; the surface lives in `packages/chat-ui`'s -`workbench-settings`. - -**In-room onboarding scene.** A named template's first-minute walkthrough -is one timeline card posted in the room's own voice, not a member's -message and not a side effect of hosting an agent. The card keeps a -stable header — the job, the promise, the ordered steps — and flips its -body in place through connect, pick-repos, and reviewing. Change-repos -returns the body to the picker without rewriting what the room already -recorded. An empty step list is omitted, not rendered as an empty rail. -The current step is named in words; colour is additive, never the only -signal. Consecutive agent-joined events collapse into one line so the -scene and the reviewers' own introductions are what a person reads -first. - -**A `kind: chat` is 1:1.** It is the one DM with its agent. Inviting a -different or additional agent into that conversation is a conflict -(HTTP 409 `kind_is_chat`); extra agents belong on a `kind: workbench` -channel. A same-definition invite reuses the resident principal and -does not clone a sibling instance. - -**Default specialist creation mints or reopens that DM.** Myra's -`create_agent` path does not invite the new definition into the -caller's conversation — Myra's DM is itself `kind: chat` and would -reject the extra agent. It mints a `kind: chat` for the definition -under the bench, or reopens the existing one for that (bench, -definition) pair, matching the product-surface find-or-reopen rule. -The specialist launches into that chat, never into Myra's. - -**Streaming a reply.** An agent's live reply reaches the timeline through -one path, deltas to pixels: - -1. Inference emits `inference.text.delta` events (vendored, `@intx/inference`) - as an agent generates a reply, each carrying the cumulative text so far. -2. The sidecar's per-agent event stream (`SidecarRouter.subscribeAgent`, - `vendor/intx/hub-sessions/src/ws/sidecar-handler.ts`) carries those - events out of the run. -3. `packages/chat/src/platform-adapter.ts` subscribes to that stream and - wraps each event as a `chat.agent` payload; `packages/chat/src/workbench-events.ts` - merges it with in-process ephemeral events (typing, settings changes) - onto one SSE stream, served from `GET /workbenches/:id/stream` - (`packages/chat/src/routes.ts`), re-checking access on every delivered - event. -4. On the web, `packages/chat-ui/src/use-workbench-stream.ts` holds the - `EventSource` (with backoff reconnect and a polling fallback), and - `packages/chat-ui/src/streaming-reply.ts` narrows `chat.agent` payloads - to `inference.text.delta` and reduces them into one growing string. -5. `packages/chat-ui/src/timeline.tsx` renders that growing string as a - synthetic in-progress message alongside the persisted timeline, until - the real message lands and replaces it. - -**Provider tool-call quirks.** Some OpenAI-compatible local models emit -a declared tool call as a JSON object in assistant text instead of a -native tool-call field. Workbench's Ollama adapter wraps the platform -OpenAI adapter (composition, not a fork) and reclassifies that exact -object into structured tool-call events when the tool name was -declared for the request. Unknown names, extra keys, arrays, mixed -prose, and fenced JSON stay text so a legitimate JSON or code answer -is not stolen. The reactor and timeline still only see the usual text -versus tool-call events. - -## Workbench Definition and hostless onboarding - -A picker "template" is a shipped **Workbench Definition**, not a second -kind of object: default agents, routines, tools, required and optional -plugins, and an ordered onboarding walkthrough. Creating from a named -row mints an empty workbench channel with no host, then instantiates -that definition into the room. The walkthrough is posted as a system -timeline card, never as a side effect of hosting an agent — so an -empty channel can onboard with nobody launched. - -Code review's definition names three reviewers and does not name Myra. -GitHub already connected is the same in-room card: it reads live -credential state and flips to repository pick. There is no separate -create-dialog path for the already-connected case. - -Settling a connector a template room is waiting on records the -connected event from a system address and does not wake an agent. A -generic in-room connect that an agent asked for still wakes that -agent. - -Credential bindings for pinned tool packages fold only at deploy. -Connecting a connector that feeds those packages relaunches live -assistants that already pin them, so the new credential is on the -run — not only on the next invite. The pending-room settle and that -relaunch are separate passes. - -## Capability growth and approval gates - -An agent's capability set grows through what it is granted, not through -code changes: Skills are installable, tenant- or principal-scoped -capabilities layered onto a definition or a workbench, resolved live -rather than baked in at deploy time (see `skills/skills`). Plugins are -installed globally rather than per bench or workbench (CL-6272.2). Every -external side effect — anything leaving the -platform's own boundary — sits behind a human approval gate. Approval -itself is native Interchange state (an `approval` row backed by a -`signal_correlation` row); the web client composes the names on top of it -(which agent, which bench) from the native pending-approvals list and run -view, and `@corbits/approvals` supplies the grant-allowance gate and the -human-readable headline, so a person can read what they're being asked to -approve — see [docs/needs-you.md](docs/needs-you.md). - -## Sidecar allocation and the provisioner model - -The **sidecar** is the execution host that runs workflow definitions on -behalf of a hub. It is a native Interchange subsystem, not a -workbench-specific service: workbench composes it the way it composes -everything else on the platform, supplying placement policy (which -sidecar a given run lands on) rather than reimplementing execution. -Provisioning and allocation follow Interchange's own contracts; workbench -does not maintain a parallel scheduler. - -## MCP connect - -Remote MCP servers connect through Plugins (curated presets and -add-by-URL). The connect-time probe and later tool calls share one -origin-pinned fetch: every first hop must be the stored origin or an -explicit extra origin for that pin — not a host-suffix match — and a 3xx -is never followed, even to an allowlisted origin. Canva is the one -shipped extra: the stored MCP origin may also first-hop the protocol -origin that is not the stored `apiBaseUrl`. - -OAuth presets that list advertised scopes send those scopes on RFC 7591 -dynamic client registration; presets that omit the list stay on the SDK's -protected-resource metadata fallback. When `/start` fails, the return -distinguishes `client_rejected` (the authorization server refused -Workbench as a client — including RFC 7591 `invalid_redirect_uri` and -sibling client-metadata codes, even when the SDK maps an unknown code -onto a generic server error) from `discovery_failed` (the authorization -server could not be reached). A successful callback re-probes with the -new token and may put that probe's tool count on the Plugins return so -the row can show it. - -Live Canva OAuth against Canva's own servers is not verified; this is -the shipped control flow, not a proven live handshake. - -## Related docs - -- [docs/GLOSSARY.md](docs/GLOSSARY.md) — product-term to platform-term - mapping -- [docs/TENANCY.md](docs/TENANCY.md) — tenancy contracts and Interchange - gaps -- [docs/CHAT.md](docs/CHAT.md) — the conversation/message/thread model -- [docs/workbench-tenancy.md](docs/workbench-tenancy.md) — workbench tenant - mint, listing, and move mechanics -- [docs/needs-you.md](docs/needs-you.md) — the approval surfacing model -- [docs/connect-cards.md](docs/connect-cards.md) — in-room connect cards - and template-room settle -- [VENDORED.md](VENDORED.md) — the vendoring ledger for `@intx/*` - -## Open questions - -- Sidecar placement policy specifics (how a run is assigned to a - particular sidecar instance under multi-sidecar deployment) are not - detailed in the docs reviewed for this pass. +Workbench is a client of [Interchange](https://github.com/faremeter/interchange). +Interchange owns identity, tenancy, credentials, agent launch, and the +workflow runtime; Workbench is the hub composition, the web client, and a +handful of domain packages built on top. + +## Services + +- **`apps/hub`** — the Interchange hub server. It mounts stock + `@intx/hub-api` routes (auth, tenants, principals, grants, workflow + deploy/run) plus one composition block that adds Corbits libraries — + `@corbits/mailbox` (chat), `@corbits/memory` (recall), `@corbits/artifacts` + (files), `@corbits/cron` (scheduling) — and the remaining in-repo + packages: `connections` (credential catalog and OAuth), `webhook-triggers` + (inbound webhook ingress), `workflows` (author routes), and + `agent-directory`. +- **`apps/sidecar`** — a byte-for-byte copy of Interchange's own sidecar. + It executes workflow runs and agent turns on the hub's behalf; Workbench + makes no local changes to it. +- **`apps/web`** — the React client. On sign-in it converges its own + tenant, workflow deployments, credentials, and grants over stock hub + routes; the hub never seeds data for it. + +## Tenancy + +A workbench is a plain Interchange tenant — there is no separate +"workbench" tenancy layer. Every user gets a tenant on sign-up; a +principal is a person or an agent that can act inside one. + +## Chat is mail + +There is no chat-specific data model. A conversation is a thread in +`@corbits/mailbox`: `GET /me/inbox/threads` lists threads, `POST +/me/inbox/send` sends a message (as a person or, addressed as an agent +principal, as that agent). Fan-out, read state, and threading are the +library's; the hub only resolves the caller's tenant/principal and mailbox +address. + +## Agents and workflows + +Myra (`agents/assistant`) is the only agent Workbench ships. There is no +workflow catalog to browse — Myra creates a workflow, tool, or skill +dynamically, as code, when a job needs one, and deploys it through the +stock workflow-deploy route. `tools/*` and `skills/*` are ordinary +`@corbits/*` packages an agent can pin. + +## Data + +Custom tables — `connections`, `webhook-triggers`, `agent-directory`, and +so on — live on their own Postgres schema, each with foreign keys back to +Interchange's `tenant`/`principal` tables in `public`. See +[docs/package-migrations.md](docs/package-migrations.md) for how a +package's own migrations are written and applied. Anything Interchange +already does natively is deleted rather than kept as a parallel path. + +## Vendoring + +Interchange capabilities not yet published to npm are vendored +byte-for-byte under `vendor/intx`, tracked with a kill date in +[VENDORED.md](VENDORED.md). The upstream repository is never modified. diff --git a/DESIGN.md b/DESIGN.md index 783e361c16..6e15770a18 100644 --- a/DESIGN.md +++ b/DESIGN.md @@ -84,7 +84,7 @@ dialog. The sidebar `+` opens this route. First-run after credential does not: `/` hops to Myra's one DM (`openAgentDm` / find-or-reopen). There is no parallel Myra home route and no Describe door. -**`/inbox` is gone as a page** (CL-6151). The path stays as a redirect +**`/inbox` is gone as a page.** The path stays as a redirect home so old links still resolve; it is not a live groups inbox. ## Tables & Lists @@ -207,7 +207,7 @@ Three named easings cover the system (all sourced from `@corbits/react-ui`'s that grows or shrinks _in place_ — the search bar's morph, a rail resizing, a composer height change — where overshoot would drag every neighbour in the row along with it. This supersedes the earlier reading of `spring` as the - search morph's curve (CL-6410 review). + search morph's curve. Named durations are also react-ui tokens: `--duration-micro` (150ms) for a hover/pressed state or icon swap, `--duration-standard` (200ms) for a toast or @@ -313,9 +313,9 @@ Own messages align right; everyone and everything else — other people, agents, system notices — aligns left. This is evaluated per viewer, never baked into the message itself: a shared bench is multiplayer, so the exact same message renders right for the person who sent it and left for every -other reader of that same bench (CL-6558, reversing an earlier reading of +other reader of that same bench, reversing an earlier reading of the "Workbench UI Overhaul v1" mock that called for a single flat, -never-mirrored layout). +never-mirrored layout. Alignment is the only thing that changes. No chat bubble, no border, no background fill on the message itself, and no change to the avatar/name/ @@ -351,7 +351,7 @@ still says Connect. GitHub for Code review is PAT-first today: Connect opens a guided personal-access-token paste, then the same card flips to pick repositories. A GitHub App / hosted OAuth Connect as the welcome mat is -CL-6343 (out of scope), not the shipped card. +tracked separately (out of scope), not the shipped card. The room's own onboarding card renders as a scene, not a member's message: no author row, the job as its title with the promise beneath, diff --git a/IMPLEMENTATION.md b/IMPLEMENTATION.md index c7cc84034e..048d85d7e8 100644 --- a/IMPLEMENTATION.md +++ b/IMPLEMENTATION.md @@ -1,310 +1,47 @@ # Implementation -The concrete stack behind [ARCHITECTURE.md](ARCHITECTURE.md)'s structure -and [PRODUCT.md](PRODUCT.md)'s surface. - -## Runtime and language - -- **[Bun](https://bun.sh)** >= 1.2 — package manager, task runner, and test - runner across the whole workspace. -- **TypeScript** — the implementation language throughout `apps/`, - `packages/`, and `workflows/`. -- **[Hono](https://hono.dev)** — the HTTP framework the hub and its mounted - package routers use. -- **[Vite](https://vitejs.dev)** — builds the web app. - -## Data - -- **Postgres** (17, with the **pgvector** extension) — the platform - database, reached through `DATABASE_URL`. -- **[Drizzle](https://orm.drizzle.team)** — the ORM/query layer, both in - vendored `@intx/db` and in workbench-owned packages that need their own - tables (e.g. `packages/chat`'s `chat` schema). -- Package-owned migrations run through each package's own migration - runner; see [docs/package-migrations.md](docs/package-migrations.md). - -## Trust boundaries - -**[arktype](https://arktype.io)** schemas validate every request body, env -value, and piece of external data at the point it crosses into the system -— never an `as T` cast on untrusted input. `.env` validation at `bun run -dev` startup reports every missing or malformed value at once, using the -same discipline. - -## UI - -**[@corbits/react-ui](https://github.com/corbitsdev/react-ui)** is the -shared component library workbench consumes rather than reimplementing — -core, reusable components live there; only workbench-specific composition -(the shell, page layout, feature-specific screens) lives in this repo. It -is pinned to a specific upstream commit rather than a floating version -range. - -`@corbits/chat-ui` owns the conversation surface. The composer's host -seam is `ComposerHandle`: `insertText` splices at the caret (Mention); -`setText` replaces the whole draft (Edit a previous prompt) and is the -layer that clears leftover slash, mention, invite, and attachment state -so a replaced draft cannot send under the old picker's rules. - -## Vendored `@intx/*` - -Interchange capabilities are consumed as published `@intx/*` npm packages -wherever a publish covers what's needed. Where a capability is not yet -published, it is vendored — hand-copied into `vendor/intx/`, one -row per path in [VENDORED.md](VENDORED.md), the authoritative ledger. Each -row carries the upstream commit it was copied from, why it isn't a -published package yet, an owner, a **kill date**, and a dated test -(`check:killdates`) that starts failing after that date — forcing either a -re-pin to a fresher upstream commit or a cutover to the published package. -Vendoring is hand-copied files only, never a git submodule; the upstream -repository is never modified. Local modifications to vendored code (e.g. -repointing an export map from `intx-src` resolve conditions to direct -TypeScript source, since workbench forbids custom resolve conditions) are -recorded per-package in each vendored package's own `VENDORED-FROM` file. - -## Commands - -| Command | What it does | -| ------------------ | -------------------------------------------------------------------------------------------------------------------------------- | -| `bun run dev` | Validates `.env`, verifies the database, applies pending migrations, builds the web UI if needed, starts the hub and one sidecar | -| `bun run setup` | Applies pending database migrations (`scripts/db-setup.ts`) | -| `bun run reset` | Drops the platform database schema and clears on-disk asset directories (local `DATABASE_URL` only, unrecoverable) | -| `bun run check` | The full gate: `typecheck && lint && fmt:check && test` — must pass before every commit | -| `bun run test` | Workspace unit/integration tests | -| `bun run test:e2e` | End-to-end smoke tests (`scripts/e2e/*.test.ts`) | -| `bun run fmt` | `oxfmt` write | - -`bun run dev` seeds the administrator account and ensures the root -tenant; it does not insert agents, tools, workflows, or skills. An empty -database is a valid hub. `bun run setup` applies migrations against the -running database and is safe to re-run. Provider API keys are never read -from the environment — hub boot plants no credentials; inference waits -until someone connects a provider. - -## Acceptance mechanism: the e2e browser walkthrough - -`bun run test:e2e` is the acceptance bar for a working system, not just -unit coverage. Each file in `scripts/e2e/` spawns a real hub process -(some also spawn a real sidecar) against a scratch database and drives it -entirely through its own HTTP API — no mocking inside the process under -test. The `smoke-*.test.ts` suite covers, one scenario per file: sign-up -authorization, onboarding provisioning, a full chat round-trip (create, -post, read, invite), an artifact upload-and-retrieve round-trip, and a -signed webhook delivery launching a real run. - -Each suite owns and rebuilds its own `_e2e` sibling database on -every run, so it can never touch a developer's working database. The -suite needs zero real credentials: `startHub` only forwards an explicit -env allowlist, so a real `ANTHROPIC_API_KEY` in the shell never reaches -the spawned hub, and every inference source in a test points at the hub's -own `noop-inference` endpoint or an unreachable placeholder host — -enforced by `assertNeverRealProvider` in `scripts/e2e/harness.ts`. CI infers -required-ness from `CI=true` so a missing `DATABASE_URL` fails loudly there -instead of silently skipping. Start Postgres locally with -`docker compose -f compose.test.yml up -d`. +## Stack + +TypeScript throughout, run on [Bun](https://bun.sh). Hono for HTTP routes, +Postgres (with pgvector) as the database, Drizzle as the query builder. +The web client is React. arktype validates every trust boundary — env, +request bodies, external data. `oxlint`/`oxfmt` lint and format; `bun run +check` (typecheck, lint, fmt check, test) is the one gate CI runs. + +## Database + +The platform schema (`public`) comes entirely from `@intx/db`'s own +migrations — this repo authors no SQL for it. Every package that owns +product tables gets its own Postgres schema, its own literal-SQL migration +list, and its own ledger table tracking which of its migrations have run; +`scripts/db-setup.ts` applies the platform's migrations and then each +installed package's, in an explicit order. See +[docs/package-migrations.md](docs/package-migrations.md) for the exact +rules and shapes. Custom tables are hard-removed when retired — no drop +migrations, no dead columns kept around; a schema cutover means resetting +the local database, not writing a data migration. + +## Vendoring + +`vendor/intx` holds Interchange packages this repo needs at a newer +commit than is published to npm. Each is a hand copy, never a submodule +or a fork, with a kill date and ledger row in [VENDORED.md](VENDORED.md). +The upstream repository is read-only from here — a needed change lands +upstream first, then gets re-vendored. ## Deployment -Deployment is explicit via **Pulumi**, targeting **Railway**. CI runs -tests only — nothing auto-deploys on `main`; a deploy is a deliberate, -separate action. - -**Where sidecars run** is one variable, `SIDECAR_PROVISIONERS`. Unset — -the default — the hub registers the `process` backend alone and runs each -exclusive sidecar as a child process on its own host, so a single-server -install works with no configuration. `docker` and `e2b` are the other -shipped backends. See -[docs/sidecar-provisioners.md](docs/sidecar-provisioners.md) for the -comparison, the required settings per backend, and how to add another. - -## Workbench Definition (shipped) - -`WorkbenchDefinition` (`WorkbenchDefinitionSchema` in -`templates/index.ts`) is the one type for a named picker row: -default agents, routines, tools, plugins `{required, optional}`, and -ordered `onboardingSteps`. A template is a shipped definition, not a -second kind. `instantiateWorkbenchTemplate` resolves the definition -against a bench over injected ports, including `beginOnboarding(steps)`. - -Create (`apps/web`'s `instant-agent-create.ts`) mints an empty -`kind: "workbench"` channel with no host and no `definitionId`, -instantiates, then `POST /workbenches/:id/onboarding` -(`packages/chat/src/routes.ts`) posts the walkthrough from -`system@` — a `connect-github` block carrying the -definition's title, promise, and step labels. The card is not posted as -a side effect of hosting an agent. - -`settleConnectedService` (`packages/chat/src/connect-pending.ts`) -clears both `connections/pending` and `template/pendingConnections`. A -template-key-only match posts `connection.connected` from the system -address and does not `dispatchTurn`. A generic pending match still -wakes the asking agent. - -That settle is not the whole of a tool-package connect. Bindings for -pinned packages (`pinnedPackageCredentialBindingsFor` in `apps/hub`) -fold only at deploy, so a live assistant launched before the key was -pasted cannot `resolve` the new handle from the old snapshot. -Connecting a `feedsTools` connector (hub `settleServiceConnection`, -e.g. Manus → `@corbits/manus-tools`) therefore also fire-and-forget -relaunches live assistants that pin that package -(`reconcilePinnedToolPackages`). Persist-only — stamp the pin onto the -launch row and leave the run — is the bug that was fixed. Connectors -with no `feedsTools` skip the pass. Same posture as CL-6687 inference -reconcile on provider connect; see -[docs/credential-wiring.md](docs/credential-wiring.md) for the -inference path and the provider plugins (`http-x-manus-api-key` among -them). - -## GitHub connect (shipped) - -The in-room `connect-github` card and the Plugins/Connections GitHub row -are **PAT-first** (CL-6345): the person pastes a personal access token; -the host tests and stores it through `@corbits/connections`' generic -`github/complete` route. The card then flips in place to pick repos -(`startReviewingRepos`), including when GitHub is already connected — -the in-room card reads live state; there is no `/new` already-connected -dialog. A GitHub App / hosted OAuth welcome mat is CL-6343 and is not -the current product path — do not document OAuth as the first Connect -step, and do not treat a PAT paste as a defect against an OAuth-first -welcome mat that has not shipped. - -The room posts that card from `system@` -(`POST /workbenches/:id/onboarding` in `packages/chat/src/routes.ts`). -`packages/chat-ui` renders a system-sender `connect-github` block as a -scene: no author row, no avatar, no "Member" label; the job title and -optional promise stay put while the body flips. Step labels come from -the block's `steps` array — an empty or omitted array does not draw a -step list. The current-step marker ("You're here") is applied only when -there are exactly three steps (connect / pick / review); `data-state` on -the step is colour, never the only signal. After the server has recorded -repos, the body is the reviewing state (repo names plus a `change repos` -control). Clicking `change repos` is client-local in -`connect-github-block-container` and does not mutate the recorded -selection. `onReviewingStarted` posts canned introductions from -`packages/code-review/src/introductions.ts` under each reviewer's own -address in roster order. Consecutive `workbench.agent-joined` rows -collapse to one line in `packages/chat-ui/src/timeline.tsx`. - -Optional `GITHUB_APP_CLIENT_ID` / `GITHUB_APP_CLIENT_SECRET` exist for -that future hosted path; leaving them unset is normal. See -`docs/connect-cards.md` and PRODUCT.md's Code review first minute. - -## create_agent and specialist DMs - -`@corbits/agent-directory-tools`' `create_agent` creates a specialist -definition in the caller's tenant and, by default, opens that -specialist's own 1:1 — never an invite into Myra's DM. - -- **Default** (`invite` omitted or true): POST - `/api/workflow-chat/participants/mint-dm` (`mintAgentDm` in - `@corbits/chat`). That find-or-reopens the `kind: chat` for - `(bench, definition)`, matching `POST /workbenches`. The agent - launches into that chat, not the caller's. -- **`invite: false`**: create the definition only — no mint-dm and no - invite. -- **Extra agent into `kind: chat`**: POST - `/api/workflow-chat/participants/invite` (and the session invite - path) returns **409** `kind_is_chat` when the target is a DM and the - definition is not that chat's first/same agent. Same-definition - retry reuses the resident. - -A create-succeeded / mint-failed split is a completed tool result that -names both halves, not a bare error. - -## Canva MCP connect (shipped) - -Canva is the `canva` MCP preset (`templates/connectors.ts`): -`https://mcp.canva.com/mcp`, `connectionMode: "oauth"`, with the 16 -advertised PRM scopes space-joined onto RFC 7591 DCR `clientMetadata.scope` -(`createMcpOAuthProvider` in `packages/connections/src/mcp-oauth.ts`). -Other presets omit `oauthScopes` and stay on the SDK's SEP-835 PRM -fallback. - -Connect-time probe and credential fetch share -`mcpOriginPinnedFetch` (`packages/credential-providers/src/mcp-origin-pinned-fetch.ts`): -pin to the stored origin, extra first hop only -`https://mcp.canva.com` → `https://canva.ai` (not a host suffix), -`redirect: "manual"` so a 302 is never followed. `/start` classifies -DCR/client refusal as `client_rejected` versus unreachable discovery as -`discovery_failed` (`packages/connections/src/mcp-oauth-routes.ts`). -RFC 7591 `invalid_redirect_uri` (and `invalid_client_metadata`, -`invalid_client`, `unauthorized_client`) count as `client_rejected`; the -route clones 4xx/5xx JSON before the MCP SDK 1.30.0 maps unknown codes -onto `ServerError`. - -A successful OAuth callback probes with the new token and, on success, -appends `toolCount` to the Plugins return query. The Canva row -(`packages/plugins-ui/src/mcp-preset-cards.tsx`) shows that count when -it is a non-negative integer; otherwise the row stays "Connected". -`@corbits/mcp-tools` per-request timeout is two minutes -(`MCP_REQUEST_TIMEOUT_MS` in `tools/mcp/src/mcp-client.ts`) — -above the SDK's 60s default, below a five-minute chat turn. - -These are unit-tested control-flow facts. Live Canva OAuth against -Canva's own servers is **not** verified; do not document a proven live -handshake. - -## Workbench-host default inference - -`selectDefaultInferencePreferences` -(`packages/chat/src/inference-preferences.ts`) builds the preference -list a workbench host launches with (`workbenchHostInferencePreferences` -on the chat adapter; also `tenantDefaultModel` on agent-definition -routes). It keeps credentialed completion-capable offerings -(`preferCompletionCapable` in `@corbits/connections/model-capability` -— embedding names never win) and, when any survivor is -`origin.direct`, picks from that direct set only. Inherit-only catalogs -still sort among inherited completion rows. - -Ollama connect is the case that used to lose: discovery -(`GET /api/tenants/:id/models`) includes inherited `CATALOG_SEEDS` rows, -so the curated name (`CATALOG_SEEDS.ollama.models[0]`) could win a -name-sort without living on the instance. `resolveOllamaModelSource` -(`packages/onboarding/src/complete-credential.ts`) therefore also reads -tenant-owned `GET /api/tenants/:id/catalog/models` (paginated -`ModelResponse` envelope) when discovery contains that curated name. A -non-empty owned list restricts candidates to those names; an empty owned -list keeps inherited discovery. Other providers still pin -`CATALOG_SEEDS[provider].models[0]`. - -Ollama's OpenAI-compatible endpoint can also emit a declared tool call -as a JSON object in `content` instead of native `tool_calls`. -`@intx/inference`'s OpenAI parser only reads `delta.tool_calls`, so -that JSON would otherwise land on the timeline as assistant text and -the tool would never run. `@corbits/ollama-adapter` -(`reclassifyInlineToolJsonEvents` in -`packages/ollama-adapter/src/inline-tool-json.ts`) rewrites a content -stream that is exactly that object into `inference.tool_call.*` -events, gated on the tools declared for the request. Salvage requires -a plain object whose keys are only `name` plus exactly one of -`parameters` or `arguments` (optional `id`), and a `name` in the -declared set. Unknown names, no-tools requests, extra keys, arrays, -mixed prose, fenced JSON, and incomplete objects stay text. - -The assistant definition (`agents/assistant`) also tells Myra to -invoke tools only through tool calls — never by writing a JSON object -with a tool name into the reply — and not to `memory_search` a bare -greeting. - -## Related docs - -- [README.md](README.md) — quickstart, local setup, repo layout, e2e detail -- [AGENTS.md](AGENTS.md) — working conventions, commit sequencing, coverage - floor -- [VENDORED.md](VENDORED.md) — the vendoring ledger -- [docs/package-migrations.md](docs/package-migrations.md) — how a - package's own migrations run -- [docs/model-seeding.md](docs/model-seeding.md) — how the catalog seed - data is curated +Deployment mechanics are not yet settled: -## Open questions +- Target hosting platform and process topology (one hub, how many + sidecars, where they run) are undecided. +- The sidecar provisioner an install uses in production (`process`, + `docker`, or `e2b`) is a per-deployment choice — see + [docs/sidecar-provisioners.md](docs/sidecar-provisioners.md). +- Secrets management for `CREDENTIAL_ENCRYPTION_KEY` and + `PRINCIPAL_KEY_ENCRYPTION_KEY` in a real deployment is undecided. +- Migration/rollout ordering across `hub`, `sidecar`, and `web` on deploy + is undecided. +- Backup/restore for `HUB_DATA_DIR` and the database is undecided. -- The single-command install path mentioned in README.md ("Workbench will - install and run with a single command... does not exist yet") is not - yet built; source checkout remains the only supported path as of this - writing. -- Whether Pulumi stacks/config live in this repo or a separate - infrastructure repo is not established in the docs reviewed for this - pass. -- Live Canva MCP OAuth (DCR, redirect allowlist, and post-OAuth probe - against `mcp.canva.com` / `canva.ai`) is not verified as of CL-7083. +Do not assume any of the above from this document alone. diff --git a/PRODUCT.md b/PRODUCT.md index 3fc9fdfd08..ac4033a9db 100644 --- a/PRODUCT.md +++ b/PRODUCT.md @@ -1,269 +1,33 @@ # Product -Corbits Workbench is the multiplayer workspace for humans and agents — a -team and its AI agents working side by side, with the same conversations -and routines. It is the default implementation of the Corbits -Platform, built on [Interchange](https://github.com/faremeter/interchange). +Workbench's product is Myra: an AI coworker who lives in your inbox and +gets things done through tools, not a chat toy bolted onto a workflow +builder. -## The one concept +## Myra -Everything in Workbench collapses to a single idea: **a workbench is a -conversation tenant.** People and agents are both principals -(Interchange). Opening an agent opens the one 1:1 DM with that agent -(`kind: chat`) — two clicks never clone a second DM. Opening a channel -opens a multi-principal room (`kind: workbench`). There is no separate -"project" or "space" object sitting above the conversation; the -conversation is the unit of work. +Myra (`agents/assistant`) is the one agent Workbench ships. She isn't a +template you configure — she's a coworker you talk to. Ask her for a +capability the team doesn't have yet, and she writes it: a new tool, a +new skill, or a scheduled workflow, deployed as code, granted only the +access it needs. There's no catalog of pre-built workflows to browse +because there doesn't need to be one. -This is why the sidebar is one recency list of conversations (pins -first) — agent DMs mixed with channels — not two labeled sections, and -not a list titled Workbenches. That noun is the product name and the -mint verb. The same Interchange agent can sit in its DM and in -many channels; product reopens or invites, and does not clone the -definition or mint a sibling instance per room. See -[docs/GLOSSARY.md](docs/GLOSSARY.md) for the full term mapping and -[docs/CHAT.md](docs/CHAT.md) for how a conversation is built underneath. +## Tools and skills -## Who it's for +Myra's capabilities come from `@corbits/*` tool and skill packages — +connecting a mail account, searching the web, reading a linked repo, +running a scheduled digest. Anyone can author a new one; Myra reuses an +existing teammate's capability before duplicating it. -Teams adopting agentic workflows who want their agents working in the same -place their people already talk — not a separate agent console bolted onto -a chat tool. A signed-in user belongs to one or more **benches** (shared -team spaces); within a bench, they open, create, and work in workbenches. +## MCP -## The guided, single-column experience +Myra can also reach any external MCP server: connect one in Plugins and +its tools become callable the same way a native `@corbits/*` tool is, +with no separate integration per server. -Workbench is intentionally not a multi-pane IDE. The product surface is one -column at a time: +## Chat is mail -- **A sidebar of conversations** lists agent DMs and channels together - by recency, pins first, in the selected bench. Agent rows are DMs - (`kind: chat`); channel rows are shared rooms (`kind: workbench`). - There are not two labeled empty sections, and the list is not titled - Workbenches. -- **"New workbench" always creates.** It opens `/new` — the shipped - prompt-primary picker — never home, and never a picker of existing - things to join. Plus mints an empty channel. Nobody is auto-hosted. -- **Agents are principals, not templates.** Opening Sales opens Sales — - the one 1:1 tenant with that agent. The same agent can sit in its DM - and in many channels. Inviting an agent provisions a native - Interchange run of that definition; product reopens or invites, it - does not clone a sibling instance per room. Myra is the - first-run guide in her DM, not a special home slot and not a - parallel home route. When she creates a specialist (`create_agent`), - the default is to mint or reopen that specialist's own DM - (`kind: chat`) — never to invite them into Myra's conversation. A - DM stays 1:1; another agent belongs in a channel. -- The active workbench occupies the main column; a contextual panel beside - it carries account-wide surfaces (approvals, recent activity) that stay - visible regardless of which workbench is open. - -## First run - -A brand-new account is walked through login and a credential connect, then -lands in Myra's one DM rather than `/new` or an empty shell: - -1. **Login.** -2. **Credential** — connect a model provider (one-click OAuth for - supported providers, or a pasted API key); see `packages/onboarding`. -3. **Myra's DM** — `/` hops an empty bench to that agent's one DM - (`openAgentDm` / find-or-reopen). There is no parallel Myra home - route. `/new` stays the create door (sidebar `+`), not this hop. - -Connecting a local Ollama uses a completion model that instance has -actually pulled, so Myra's first DM message is a real agent turn — even -on a machine that only has models like llama3.2 or qwen3. An inherited -catalog seed the instance never pulled is not the default when the -tenant already owns a pulled completion model. Embedding models never -become that default. A local model that writes a tool it meant to -invoke as JSON in the reply still becomes a human turn: the person -sees Myra's words (and ordinary tool activity), not that JSON as the -message. - -Create stays on `/new` (`apps/web/src/pages/new-workbench-picker.tsx`): -a prompt box is the primary act: typing a goal and submitting mints an -empty channel and sends that text as the first message; blank plus -invites nobody. Named-template rows underneath mint that same empty -channel with no host and no mint `definitionId`, then instantiate the -picked Workbench Definition's own agents (Myra joins only when the -definition names her) — one-click shortcuts, not a kind-then-Create -second step. There is no Describe door and no -`describe-first-workbench.tsx`. - -A bench that already has one or more workbenches skips first-run and -lands on `workbenches[0]` (see `apps/web/src/pages/home-page.tsx`). Myra -is the first-run guide in her DM, not a home slot. - -The shell's first-run destinations stay small on purpose: -Mission Control is pinned above the footer rail; the rail itself is -Routines, Files, Skills, Agents, and Plugins. Insights and Evals appear -on that rail only after honest usage exists. New benches should not -meet an empty Insights / Evals gallery before they have anything to -put there. - -Myra is every tenant's default agent and the coordinator of its other -agents, not merely the first-run guide. When a request needs a -capability the bench doesn't have, she doesn't decline it or improvise -an inline answer: she proposes a teammate — an agent, workflow, or -skill — authors it as code, deploys it through the normal deploy flow, -grants it only the access its job requires, and adds it to the bench -or a DM, reporting back once it's live. She checks for an existing -teammate that already fits before creating one, so a bench never ends -up with two agents doing the same job. - -### Code review's first minute - -Code review is the product scene for the definition-driven path: -minting the Code review workbench opens an empty room with no host — -its Workbench Definition names three reviewers and no Myra. The room -itself posts the onboarding card as a scene, not a member's message: -no author row, the job as its title, the promise beneath, and the -walkthrough's steps listed with the current one marked in words. A -walkthrough with no steps hides the step list rather than drawing an -empty one. Connect GitHub with a personal access token (the shipped -path today). The same in-room card reads live connection state and -flips in place to pick repositories — already connected GitHub is -that card, not a `/new` dialog — then Start reviewing. Once repos are -recorded the same card shows what it is reviewing, with a change-repos -link back to the picker — a reviewing card never still says Connect. -Once reviewing starts, each reviewer posts its own canned introduction -under its own address, in roster order — the first thing a person -reads is who is reviewing and what for, never a join dump. Consecutive -agent-joined rows collapse into one line naming everyone. A GitHub App -/ hosted OAuth welcome mat is future work (CL-6343), not current -product. Inviting teammates into the room is a later slice, not part -of this first minute. - -Settle for a template-key-only wait posts from a system sender and -does not wake an agent. Generic `connections/pending` still wakes the -asking agent. Neither path posts the connected notice as the connecting -person. - -## Reusing an own prompt - -A person can **Edit** their own previous prompt. Edit copies that text -into the composer, replacing any leftover draft — slash command, mention, -invite, and attachments. Sending posts a new message on the same -timeline: it does not rewrite the original, and it does not fork a -thread. Other people's prompts have no Edit. - -## Plugins and Skills - -A **Skill** is a named, reusable capability — instructions an agent can -pin and a workbench can install, backed by the platform's native -`kind:"skill"` asset (see `skills/skills`). Skills are visible only to -the principal or tenant that owns them; nothing crosses tenant boundaries -implicitly. Plugins extend what a workbench can do the same way Skills -extend what an agent knows — both are installable, both are scoped to the -bench or workbench that installs them, and neither requires touching -platform internals. - -The Plugins rail is how a person connects remote MCP servers: curated -preset cards plus an add-by-URL path. Canva is an OAuth preset — Connect -sends them through that app's sign-in, then back to Plugins. After a -successful OAuth return, the row can show how many tools the connect -probe found; a missing or non-integer count stays a bare "Connected". If -sign-in cannot start, Plugins distinguishes an unreachable authorization -server from the app rejecting Workbench as a client (redirect URL or -registration). Agent MCP tool calls are allowed two minutes so a slow -design tool can finish inside a chat turn. - -Live Canva OAuth against Canva's own servers is **not** verified as of -CL-7083. This documents the shipped connect path, not a proven live -handshake. - -## Workbench settings - -Each workbench has its own full-stage settings surface, not a dialog — -reached from the workbench itself, never a separate console. It covers -what is specific to that one conversation: name, purpose, and pinned -state; its agent and human participants; per-agent name, instructions, -and capabilities; dedicated vs. shared inference capacity (CL-6117); -per-workbench connector and plugin overrides against the account default -(CL-6099); inference model/provider fallback order; applying a saved -config profile; per-person notification preferences for that workbench; -and archiving. See ARCHITECTURE.md's "Conversation as workbench data" -for how a workbench's settings relate to its tenant, and -`packages/chat-ui`'s `workbench-settings` for the implementation. - -## Routines, through conversation - -A **Routine** is a scheduled workflow — an authored definition whose -frozen projection carries a native `ScheduleTrigger`. Cadence is set -from inside conversation, not from a separate scheduling console: a -person names what should happen again, on what schedule, and where the -result should land. See `@corbits/workflows` for the schedule/cron -helpers and the hub's `workflow-scheduler.ts` for the poller. Owner -ruling (2026-09-17, LESS IS MORE): the shipped workflow catalog is -deleted — `WORKFLOW_CATALOG` now carries only Myra, and every other -workflow a workbench runs is one Myra creates dynamically for that -workbench's own needs. - -## Inbox and approvals - -Approvals sit outside any single workbench. Inbox does not: - -- **Inbox is not a product page.** The groups UI (action / mention / - delivery) is gone. `/inbox` stays routable only as a redirect home - (CL-6151) so old links and bookmarks land somewhere real. The hub - inbox API (`packages/inbox`, `/api/tenants/:tenantId/inbox`) may still - exist as a backend; it is not a live groups page. -- **Approvals ("needs you")** surface a paused agent run waiting on a - human decision. There is no dedicated Approvals page — pending approvals - show up in the Activity band, a permanent section of the contextual - panel visible from every page, resolved by name ("`` in - ``") rather than a raw id. See - [docs/needs-you.md](docs/needs-you.md). - -## Insights - -Workbench exposes tenant-level usage and activity data — cost, token -consumption, and run activity — surfaced as read-only queries over data a -usage sink persists from the live inference stream. Missing data is shown -as an explicit absence, never a fabricated zero. See `packages/insights`. - -## Vocabulary - -User-facing surfaces (UI, docs, support) use exactly these nouns: - -- **Workbench** — the product name, and the mint verb ("New workbench"). - A workbench is a conversation tenant: a DM or a channel. Do not title - the sidebar list Workbenches. -- **Agent** — a coworker principal. Opening the row reopens that agent's - one DM. Never "template." Myra is the first-run guide in her DM. - Creating a specialist opens (or reopens) that specialist's own DM, - not a seat in Myra's. -- **DM** — the one 1:1 conversation with an agent. Never cloned by a - second open, and never a room for a second agent. -- **Channel** — a shared room between people and agents. Plus mints an - empty one; nobody is auto-hosted. Named templates instantiate their - Workbench Definition's own agents into that room (Myra joins only - when the definition names her). -- **Bench** — the shared team scope a person signs into and switches - between; shown in the bench switcher, never called a "workspace" or - "org" in copy. - -Internally these map onto platform primitives (principal, tenant, kind: -chat vs kind: workbench) — see [docs/GLOSSARY.md](docs/GLOSSARY.md) for -the authoritative table. Code and API paths generally keep the -platform's own names; "workbench" is the one exception (CL-6260), since -its package (`@corbits/chat`) is ours, not the platform's — only -user-facing surfaces use the rest of the product vocabulary above. - -## Open questions - -- Template-key-only settle (system sender, no agent wake) is shipped; - generic `connections/pending` still wakes the asking agent. -- Settle attribution via a synthetic system sender is shipped (CL-6741); - `connection.connected` is never posted as the signed-in user. See - [docs/connect-cards.md](docs/connect-cards.md). -- A leftover agent 401 after GitHub already succeeded, and a stale - Connect after success, remain first-minute bugs — see - [docs/connect-cards.md](docs/connect-cards.md); do not document those - as shipped. -- Live Canva MCP OAuth (sign-in, DCR, and post-OAuth probe against - Canva's own servers) is not verified; do not document a proven live - Canva handshake. -- The precise boundary of what Insights surfaces to a non-admin bench - member (all tenant activity vs. only their own) is not spelled out in - `packages/insights`'s own docs as of this writing. +Talking to Myra, or to a teammate, is an ordinary mail thread — no +separate "chat" concept to learn, no bespoke room model. What you send +lands in an inbox; what she does happens through approvals you can see. diff --git a/README.md b/README.md index bcf2ea52e9..5e2e8b5160 100644 --- a/README.md +++ b/README.md @@ -1,228 +1,74 @@ # Corbits Workbench -**The multiplayer workspace for humans and agents.** - -Corbits Workbench is the open-source workspace where your team and your AI -agents work side by side in shared [benches](docs/GLOSSARY.md) — same -threads, same workflows, same inbox. It is the default implementation of -the Corbits Platform — a set of pluggable libraries with -[Interchange](https://github.com/faremeter/interchange) at the core — -offering agents, workflows, and multi-tenant capabilities for organizations -embracing agentic solutions. Grounded in your team's own knowledge, with -every external side effect behind human approval. - -## Install - -Run Workbench from a source checkout: clone the repo and follow the -Quickstart and Running locally sections below. A single-command Homebrew -install is tracked separately and is not part of this path. +The multiplayer workspace for humans and AI agents. Workbench is a web +client of [Interchange](https://github.com/faremeter/interchange): it +signs you into a plain Interchange tenant, and chat is mail threads over +`@corbits/mailbox`. The one shipped agent, Myra, creates whatever +workflows, tools, or skills a job needs on the fly. ## Quickstart -Requires [Bun](https://bun.sh) >= 1.2. The exact version CI and this repo -develop against is pinned in [`.bun-version`](.bun-version). +Requires [Bun](https://bun.sh) >= 1.2 (pinned in `.bun-version`) and a +local [Postgres](https://www.postgresql.org) 17 with the pgvector +extension — `brew install postgresql@17 pgvector` on macOS, or bring up +the same Postgres CI uses: `docker compose -f compose.test.yml up -d`. ```sh bun install -bun run typecheck && bun run lint && bun run test -``` - -## Running locally - -Requires [Bun](https://bun.sh) >= 1.2 and a local -[Postgres](https://www.postgresql.org) with the pgvector extension — any -Postgres 17 you point `DATABASE_URL` at works. On macOS: - -```sh -brew install postgresql@17 pgvector -brew services start postgresql@17 -``` - -Or, to match what CI runs against exactly, bring up -[`compose.test.yml`](compose.test.yml) instead: - -```sh -docker compose -f compose.test.yml up -d -``` - -Then: - -```sh cp .env.example .env bun run dev ``` -Recommended: run `bun run setup:memory` for a machine-specific -recommendation on turning on the memory plane (embeddings-backed recall) -and its optional reranker — native Ollama, Docker, or a remote endpoint, -whichever this machine can actually use. When a local embed path exists, -that command writes the missing `EMBED_*` keys into `.env`. `bun run dev` -also mounts `@corbits/memory` when `OLLAMA_BASE_URL` is set, or when -native Ollama is on PATH even if you skipped setup. Without any of those, -memory tools answer "not set up" instead of erroring; rows written before -an embed URL is set are never retroactively embedded. See -[docs/local-dev.md](docs/local-dev.md#memory-plane) for the full -degradation story. - -`bun run dev` validates your `.env` (reporting every missing or malformed -value at once), verifies the database in `DATABASE_URL` is reachable and -actually speaks Postgres, applies any pending platform migrations, builds -the web UI if it has not been built yet, and starts the hub. The hub -provisions authenticated sidecars on demand using the configured backend. -Every required setting lives in `.env.example` with its expected shape. -`bun run dev` may sign up the local administrator (`HUB_ADMIN_EMAIL` / -`HUB_ADMIN_PASSWORD`, defaulting to alice@example.com / password123 -when unset) through the same auth HTTP API the UI uses. Hub boot itself -inserts no users or tenants. An empty database is a valid hub: boot -does not insert agents, tools, workflows, or skills. Product state -arrives through onboarding and explicit seed callers, not production -boot. -Provider API keys are never read from the environment: hub boot plants no -credentials, so setting `ANTHROPIC_API_KEY` or any other provider's env -var has no effect. Inference waits until someone connects a provider. - -That applies to every account, not just the administrator: anyone who -signs up gets a personal bench with no default routines -deployed, and first-run tells them exactly that. Onboarding walks them -through picking a provider — Anthropic, OpenAI, Google, OpenRouter, Hugging -Face, Groq, or another of the curated providers in -[`packages/connections/src/catalog-seed-data.ts`](packages/connections/src/catalog-seed-data.ts) -— and pasting their own key (or, for OpenRouter, completing a PKCE OAuth -connect), which is stored right away — no separate seeding step, no -docs to read. A wrong key isn't caught up front; it surfaces the first time -it's actually dialed for real inference, through the same in-chat "Fix this -connection" flow any credential failure uses. The bench's default agents -deploy in the background — "Your workbench is ready — agents will come -online shortly," no "Connecting…" wait in the browser. Whichever provider they connect gets its own curated -catalog entry planted the same way onboarding plants a connected -provider's catalog; see -[docs/model-seeding.md](docs/model-seeding.md) for how that catalog data is -curated and kept up to date. - -### Resetting local state - -To test onboarding (the self-serve signup and first-login provisioning flow) -from a clean slate, or to recover from a broken local database, run: +`bun run dev` validates `.env`, applies pending migrations, builds the web +UI if needed, and starts the hub. It signs up a local administrator +(`HUB_ADMIN_EMAIL` / `HUB_ADMIN_PASSWORD`, default +`alice@example.com` / `password123`) through the same auth API the UI +uses — hub boot itself creates no users, tenants, or product data. + +Required env vars (see `.env.example` for the full, commented list): +`DATABASE_URL`, `BASE_URL`, `SESSION_SECRET`, `HUB_DATA_DIR`, +`HUB_ALLOW_GIT_INSIDE_WORK_TREE`, `HUB_STATIC_DIR`. Everything else — +OAuth sign-in, the memory plane's embedding provider, sidecar +provisioners — is optional and documented inline in that file. To run +fully offline, connect the Ollama provider from the UI instead of a +cloud key; see [docs/local-dev.md](docs/local-dev.md). + +To wipe local state (drops the schema and the hub's on-disk asset +directory, refuses against anything but a local database): ```sh bun run reset ``` -`bun run reset` drops the platform database schema and removes the hub's -on-disk asset directory (which also holds provisioned sidecar state). -Nothing is recreated until the next `bun run dev` — that recreates the -schema. Hub boot itself inserts no users or tenants. Local `bun run -dev` may then sign up the administrator through the same auth HTTP API -the UI uses, so you can sign in. It does not insert agents, tools, -workflows, or skills. Product state arrives through onboarding and -explicit seed callers. - -It refuses to run against anything but a local `DATABASE_URL` (localhost, -127.0.0.1, or `::1`) — there is no override, since the schema drop is -unrecoverable. - -### OAuth sign-in - -Email/password sign-in always works. To let people sign in with an -existing Google or GitHub account instead, set that provider's client id -and secret in `.env` — `GOOGLE_CLIENT_ID`/`GOOGLE_CLIENT_SECRET` and/or -`GITHUB_CLIENT_ID`/`GITHUB_CLIENT_SECRET`, each an independent pair. The -matching button then appears on the sign-in screen automatically; leave -both values of a pair unset to leave that provider off. Setting only one -value of a pair is a boot-time error — the hub refuses to start rather than -silently disable the provider. See `.env.example` for where to register -each OAuth app and what redirect URI to configure. - ## Repo layout -| Path | What lives here | -| ------------ | ---------------------------------------------------------------------- | -| `apps/` | Deployable services (hub API, web UI, Interchange sidecar) | -| `packages/` | Domain packages — Workbench UI/composition and other product logic | -| `tools/` | Individually publishable `@corbits/*` agent-tool packages | -| `agents/` | Individually publishable `@corbits/*` interactive-agent packages | -| `skills/` | Individually publishable `@corbits/*` skill packages | -| `workflows/` | Workflow definition packages, deployed as assets | - -Chat — the shared conversation surface humans and agents both use — is -documented separately in [docs/CHAT.md](docs/CHAT.md). +| Path | What lives here | +| ------------- | --------------------------------------------------------------- | +| `apps/` | `hub` (API), `web` (React client), `sidecar` (execution host) | +| `packages/` | Domain packages and Workbench-specific composition | +| `tools/` | Publishable `@corbits/*` agent-tool packages | +| `agents/` | Publishable `@corbits/*` agent packages (`assistant` is Myra) | +| `skills/` | Publishable `@corbits/*` skill packages | +| `workflows/` | Workflow definition packages, deployed as assets | +| `vendor/intx` | Hand-copied Interchange fallbacks; see [VENDORED.md](VENDORED.md) | ## Development -- `bun run check` — the full gate: typecheck, lint (oxlint + `oxfmt - --check`), structural checks, test -- `bun run test` — workspace tests -- `bun run test:e2e` — end-to-end smoke tests (see below) -- `bun run fmt` — oxfmt write - -Conventions and agent guidance: [AGENTS.md](AGENTS.md). Contributions: -[CONTRIBUTING.md](CONTRIBUTING.md). - -Docs map: [PRODUCT.md](PRODUCT.md) (what and why), [ARCHITECTURE.md](ARCHITECTURE.md) -(system structure), [IMPLEMENTATION.md](IMPLEMENTATION.md) (concrete stack). - -### End-to-end smoke tests - -`bun run test:e2e` runs `scripts/e2e/*.test.ts`: each file spawns a real hub -process (some also spawn a real sidecar) against a scratch database and -drives it entirely through its own HTTP API — no mocking inside the -process under test. `scripts/e2e/smoke-*.test.ts` are the CL-6004 smoke -suite, one scenario per file, each independent and safe to run alone or -in any order: - -| File | Proves | -| -------------------------- | --------------------------------------------------------------------------------------------------------------------- | -| `smoke-auth.test.ts` | Email sign-up mints a session cookie that authorizes a session-gated route | -| `smoke-onboarding.test.ts` | A brand-new signup starts benchless and mints its first tenant through an ordinary `POST /api/tenants` | -| `smoke-chat.test.ts` | A workbench is created, a message posts and reads back intact, and the invited-agent listing has its documented shape | -| `smoke-webhook.test.ts` | A signed delivery to the public webhook ingress route launches a real run for a routine bound to that trigger | - -**Needs:** a reachable Postgres named by `DATABASE_URL` in `.env` (see -[Running locally](#running-locally)). Each suite reuses a sibling -`_e2e` database that it owns outright — drops and rebuilds its -own schema on every run — so it can never touch your working database. -Without `DATABASE_URL` the suite skips with a warning naming -`docker compose -f compose.test.yml up -d`; CI infers -required-ness from `CI=true` so that skip fails loudly there instead of -silently passing. - -**Needs zero real credentials.** The whole suite — smoke and non-smoke -files alike — runs with no API keys, no paid credentials, and no network -call to a real inference provider: `startHub` only forwards an explicit -env allowlist, so a real `ANTHROPIC_API_KEY` sitting in your shell never -reaches the spawned hub, and every inference source a test configures -points at the hub's own `noop-inference` endpoint or an unreachable -placeholder host. `scripts/e2e/harness.ts` exports -`assertNeverRealProvider` for any test that builds a baseURL/apiKey by -hand, so an accidental live-provider reference fails loudly instead of -attempting a real network call. Never document "set a key to run this" — -if a scenario would need one, stop at the keyless contract it can still -prove (see `smoke-onboarding.test.ts`) or don't write that scenario. - -Run one file directly with `bun test scripts/e2e/smoke-auth.test.ts` -(`DATABASE_URL` still required). - -### DB-gated package and hub suites - -Beyond the e2e suite, most packages and `apps/hub` carry their own -DB-gated tests (migrations, Drizzle-backed stores, route integration -tests) that skip via `describeIfDb` when `DATABASE_URL` is unset. A -skip is never silent: `scripts/e2e/db-gate.ts`'s `dbGate` — what -`describeIfDb` is built on — prints an unmissable summary naming every -skipped suite at the end of the run, and infers required-ness from -`CI=true` the same way the e2e suite above does, turning a skip into -a hard failure on CI jobs that provision Postgres. - -To run these locally against the same Postgres CI uses: - ```sh -docker compose -f compose.test.yml up -d -DATABASE_URL=postgres://postgres:postgres@localhost:5432/workbench bun test packages apps/hub/test +bun run check # typecheck, lint, fmt:check, test — the full gate +bun run test:e2e # end-to-end smoke tests against a scratch database +bun run fmt # oxfmt write ``` +`test` and `test:e2e` need a reachable `DATABASE_URL`; DB-gated suites +skip locally with a banner and fail loudly in CI. + +More: [AGENTS.md](AGENTS.md), [ARCHITECTURE.md](ARCHITECTURE.md), +[IMPLEMENTATION.md](IMPLEMENTATION.md), [PRODUCT.md](PRODUCT.md). + ## License The application (`apps/` and the rest of this repo) is GPLv2 with the [AI Exception](GPLv2-AI-Exception.md) — see [LICENSE.md](LICENSE.md). -Libraries under `packages/` and `workflows/` are each licensed -LGPL-2.1-or-later, with the license text alongside their source. +Libraries under `packages/`, `tools/`, `agents/`, `skills/`, and +`workflows/` are each licensed LGPL-2.1-or-later. diff --git a/VENDORED.md b/VENDORED.md index e4491bcb48..df8b04b626 100644 --- a/VENDORED.md +++ b/VENDORED.md @@ -11,209 +11,52 @@ never a convenience. - Every vendored path has exactly one row in the ledger below. Code copied into the tree without a ledger row is not vendored; it is a bug. - Every entry carries a **kill date** — the date by which the vendored copy is - replaced by a published package or deliberately renewed — and a dated test - that fails after that date. An entry with no kill date is not an entry. -- The ledger row, the kill date, and its dated test land in the same commit as - the copied files. -- Local changes to vendored code land in this repository through normal review. - The upstream repository is never modified, committed to, or pushed to. -- Retiring a vendored copy closes the entry: delete the row, the files, and the - kill-date test together. + replaced by a published package or deliberately renewed. +- The ledger row and the kill date land in the same commit as the copied + files. +- Local changes to vendored code land in this repository through normal + review. The upstream repository is never modified, committed to, or pushed + to. +- Retiring a vendored copy closes the entry: delete the row and the files + together. ## Ledger -| Vendored path | What was copied | Upstream repo @ commit | Why not a published package | Owner | Kill date | Kill-date test | -| ----------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------ | ---------- | ----------------- | -| `apps/sidecar` | Byte-identical to upstream `apps/sidecar` at this pin — no local delta; every Workbench-only module (`oauth-login.ts`, `originating-workbench.ts`, `hibernated-agent-identity-vault.ts`, `hub-link-watchdog.ts`, `workflow-deployment-record.ts`, `shutdown.ts`, `concurrency.ts`, `boot-restore-push-hold.ts`, the `workflow-host-wiring/`/`workflow-substrate-factory/` split directories, `step-strategy.ts`, `workbench-director.ts`, and their tests) deleted outright. | [faremeter/interchange](https://github.com/faremeter/interchange) @ `79adc433` (origin/main, 2026-09-17) | An app is never npm-published, so no publish can cover the execution host; retired by an upstream `@intx/sidecar` publish, or by renewing this row deliberately | sawyer | 2026-11-03 | none — no check scripts | -| `vendor/intx/agent` | `@intx/agent` source (`src/`, manifest, tsconfig) | [faremeter/interchange](https://github.com/faremeter/interchange) @ `79adc433` (origin/main, 2026-09-17) | npm 0.3.0 predates the operator-configurable doom-loop threshold (`afd0c82b`, `c421c092`) the re-vendored `workflow-host` configures; the CL-7190 `beforeToolExtension` delta is absorbed upstream at this pin — no local delta; retired by the next `@intx/agent` publish | sawyer | 2026-11-03 | `check:killdates` | -| `vendor/intx/db` | `@intx/db` source (`src/`, `migrations/`, drizzle config, manifest, tsconfigs) | [faremeter/interchange](https://github.com/faremeter/interchange) @ `79adc433` (origin/main, 2026-09-17) | npm 0.3.0 predates this pin's schema; the CL-6324 `wire_projection` column/loader and the CL-6452 `workflow_definition.origin` column are GONE upstream at this pin (CL-8206) — every hub-side reader of either is a hard cutover in this change, not a port — and the unpublished per-principal signing-key series (`principal_key`) this pin previously re-vendored is now upstream-native, so that delta is dropped; the pin keeps a CL-7505 serving-time refresh delta (`model-source-resolution.ts`: `buildSource`/`resolveModelSources`/`resolveInstanceModelSources` accept an optional `servingRefresh` hook, with a `credential_needs_reauth` skip so an unrefreshable `oauth_token` credential is never served) and a CL-7510 delta (`"openai-responses"` in `schema/catalog.ts`'s `modelProvider.plugin` enum, and an adapter registry-key dispatch rewrite in `model-source-resolution.ts`'s `buildSource` so `ollama`/`codex`/`xai-oauth` resolve their own registry key while the catalog `plugin` column keeps the wire-format id); retired when upstream absorbs the deltas | sawyer | 2026-11-03 | `check:killdates` | -| `vendor/intx/harness` | `@intx/harness` source (`src/`, manifest, tsconfig) | [faremeter/interchange](https://github.com/faremeter/interchange) @ `79adc433` (origin/main, 2026-09-17) | npm 0.3.0 predates the connector reply drain (`driveConnectorReplies`, `ConnectorReplyDrain`, `AgentEventStream`; `11590e66`) the sidecar's warm mail loop drives; no local delta; retired by the next `@intx/harness` publish | sawyer | 2026-11-03 | `check:killdates` | -| `vendor/intx/hub-agent` | `@intx/hub-agent` source (`src/`, manifest, tsconfig) | [faremeter/interchange](https://github.com/faremeter/interchange) @ `79adc433` (origin/main, 2026-09-17) | npm 0.3.0 predates the `agentDir` path export (`927556de`) the sidecar's deploy-tree lookup uses, and its own `@intx/mail-memory`/`@intx/harness` pins must resolve the vendored copies; one local delta (CL-6396): `SessionEventSink` / `publishWorkflowInferenceEvent` stamp optional `childRunId` on `agent.event`; another (CL-7508): the `OAuthLoginExecutor` link option (`ws/hub-link.ts`) routing inbound `oauth.login.start` frames to the sidecar host's loopback login executor and answering every path with `oauth.login.result` frames (`started`, then one terminal arm), with a fail-closed placeholder executor so an unwired host still errors instead of hanging the hub, threaded through `sidecar-orchestrator.ts`'s config, plus cancellation: `oauth.login.cancel` frames and link disconnects close the staged login's pinned-port listener via the executor's `cancel` handle; the pin also carries the package's first colocated test (`ws/hub-link-oauth-login.test.ts` with a `bun test` script) pinning the cancel dispatch against a local ws server; retired by the next publish | sawyer | 2026-11-03 | `check:killdates` | -| `vendor/intx/hub-api` | `@intx/hub-api` source (`src/`, manifest, tsconfig) | [faremeter/interchange](https://github.com/faremeter/interchange) @ `79adc433` (origin/main, 2026-09-17) | npm 0.3.0 predates this pin; the null-principal `resolveApproval` (CL-6345) and the bearer-authenticated workflow-deploy mirror (CL-workflow-deploy-bearer) are still local, but the per-principal signing-key seam (`principalKeyStore`) this pin previously re-vendored is now upstream-native and dropped; keeps the CL-7505 serving-time refresh delta (`run-source-resolution.ts`: `resolveDefinitionSources` accepts and threads an optional `servingRefresh` hook); retired when upstream absorbs the remaining deltas | sawyer | 2026-11-03 | `check:killdates` | -| `vendor/intx/hub-sessions` | `@intx/hub-sessions` source (`src/`, manifest, tsconfig) | [faremeter/interchange](https://github.com/faremeter/interchange) @ `79adc433` (origin/main, 2026-09-17) | npm 0.3.0 covers the base package but not the pack-acceptance fixes (`ownsWorkflowRunRepo`, `anchorAddressForPackSource`, `decideTerminalRunFlip`), malformed tool-call-name sanitization (CL-6478), or the sealed-run terminal-status backfill (CL-6595); the CL-6324 wire-projection writer, the CL-7190 `registerSignalCorrelation` guard, and the CL-6396 `childRunId` forward are absorbed upstream at this pin (`SessionService.sendUserMessage` is gone entirely — every caller now delivers through `SidecarRouter.routeMail`) and dropped; keeps a CL-7505 serving-time refresh delta (`credential-push.ts`: `pushInstanceSourceUpdate`/`pushSourceUpdates`/`pushSourceUpdatesSubtree` accept and thread an optional `servingRefresh` hook); a CL-7508 loopback-login delta (`ws/sidecar-handler.ts`: the `oauth.login.result` frame classified as a bypass/terminal correlated response, the `pendingOAuthLogins` request map with a whole-login timeout that also sends `oauth.login.cancel` so the sidecar tears its pinned-port listener down, plus a disconnect sweep, the `requestOAuthLogin` router method gated behind a new `oauthLogin.isLocalSidecar` locality policy, and the typed gate outcome when no connected sidecar is local; review fix: the terminal `completed` arm resolves the pending login with its tokens instead of falling into the error arm); and a corbitsdev/workbench#709 prepared-deploy delta (`session-service.ts` threads `deployContent` + `toolPackagePins` through the prepared-deploy params and fans every top-level step out to `stageWorkflowStep` before the deployment frame, `workflow-allocation-service.ts` rehydrating both from the frozen launch spec with a fail-closed prompt check); retired when upstream absorbs the remaining deltas | sawyer | 2026-11-03 | `check:killdates` | -| `vendor/intx/inference` | `@intx/inference` source (`src/`, manifest, tsconfig) | [faremeter/interchange](https://github.com/faremeter/interchange) @ `79adc433` (origin/main, 2026-09-17) | npm 0.3.0 predates doom-loop detection (`8da4c827`, `afd0c82b`, `c421c092`); local delta: `providers/google-genai-files.ts` builds its upload body as `new Uint8Array(bytes)` because TS 6's lib.dom `BodyInit` rejects `Uint8Array` (upstream compiles ESNext-only under TS 5.9); the CL-7190 `message_response` resume-branch delta is absorbed upstream at this pin and dropped; retired by the next publish | sawyer | 2026-11-03 | `check:killdates` | -| `vendor/intx/mail-memory` | `@intx/mail-memory` source (`src/`, manifest, tsconfig) | [faremeter/interchange](https://github.com/faremeter/interchange) @ `79adc433` (origin/main, 2026-09-17) | npm 0.3.0 predates the `@intx/mailbox` extraction (`af03bb90`), on-demand body reads (`54f7c239`) and `expunge` returning the swept uids (`bcabb1f8`) that the re-vendored `workflow-host` binds against; no local delta; retired by the next publish | sawyer | 2026-11-03 | `check:killdates` | -| `vendor/intx/mailbox` | `@intx/mailbox` source (`src/`, manifest, tsconfig) | [faremeter/interchange](https://github.com/faremeter/interchange) @ `79adc433` (origin/main, 2026-09-17) | Never published: a new package at the target pin (`af03bb90`) that `workflow-host`'s substrate mailbox store and supervisor-backed transport import; no local delta; retired by its first publish | sawyer | 2026-11-03 | `check:killdates` | -| `vendor/intx/mime` | `@intx/mime` source (`src/`, manifest, tsconfig) | [faremeter/interchange](https://github.com/faremeter/interchange) @ `79adc433` (origin/main, 2026-09-17) | npm 0.3.0 predates the non-RFC message-id guard `isMessageId` (`d97e1832`), the full `References` chain (`65c6fe70`) and the lossless `decodeMail` decoder (`3b6d06b2`) that `mailbox`/`mail-memory` at the same pin import; no local delta; retired by the next publish | sawyer | 2026-11-03 | `check:killdates` | -| `vendor/intx/types` | `@intx/types` source (`src/`, manifest, tsconfig) | [faremeter/interchange](https://github.com/faremeter/interchange) @ `79adc433` (origin/main, 2026-09-17) | npm 0.3.0 predates the type surface the re-vendored trees compile against; the CL-7190 `"message_response"` signal-kind delta and the CL-6396 `childRunId` delta are absorbed upstream at this pin and dropped, as is the unpublished `signer-identity`/`principalKeyAad` signing-key surface (now upstream-native); keeps one local delta (CL-7510): `"openai-responses"` added to `catalog.ts`'s `modelProviderPlugins`; retired by the next publish | sawyer | 2026-11-03 | `check:killdates` | -| `vendor/intx/hub-common` | `@intx/hub-common` source (`src/`, manifest, tsconfig) | [faremeter/interchange](https://github.com/faremeter/interchange) @ `79adc433` (origin/main, 2026-09-17) | npm 0.3.0 predates `generateId`'s `pky_` principal-key prefix, now upstream-native at this pin; no local delta; retired by the next `@intx/hub-common` publish | sawyer | 2026-11-03 | `check:killdates` | -| `vendor/intx/workflow` | `@intx/workflow` source (`src/`, manifest, tsconfig) | [faremeter/interchange](https://github.com/faremeter/interchange) @ `79adc433` (origin/main, 2026-09-17) | One local delta (corbitsdev/workbench#709): `run.ts` treats the per-step timeout as an execution budget — trigger-idle input parks disarm the step timer across the wait and re-arm a fresh budget on resume, while the approval park stays armed; npm 0.3.0 predates the `onBodyFailure: "tolerate"` section policy (`b977ade6`) that `@corbits/agent-runtime` authors and the action/loop primitives (`3bd5b837`, `1ea2f39b`) the re-vendored `workflow-host` runs; retired by the next `@intx/workflow` publish | sawyer | 2026-11-03 | `check:killdates` | -| `vendor/intx/workflow-deploy` | `@intx/workflow-deploy` source (`src/`, manifest, tsconfig) | [faremeter/interchange](https://github.com/faremeter/interchange) @ `79adc433` (origin/main, 2026-09-17) | No local delta: npm 0.3.0 predates `inertLoopBody` and the loop-body source pin (`1ea2f39b`) that the re-vendored `hub-sessions` imports; retired by the next `@intx/workflow-deploy` publish | sawyer | 2026-11-03 | `check:killdates` | -| `vendor/intx/workflow-host` | `@intx/workflow-host` source (`src/`, manifest, tsconfig) | [faremeter/interchange](https://github.com/faremeter/interchange) @ `79adc433` (origin/main, 2026-09-17) | npm 0.3.0 covers the base package but not this pin; the CL-6396 event-channel `childRunId` stamp is absorbed upstream and dropped; keeps one local delta (CL-6448): `run-child.ts`'s `createCredentialsBackedAuthorize` re-applies `findStepGrantsEntry`, which upstream at this pin does not have — for a single-step deployment, an `onTrigger` body step's own id (keyed by the PARENT step's stepOrder in the credentials snapshot) collapses to the deployment's sole grants entry instead of throwing, mirroring the head/step collapse the body's tool materialization already uses; a multi-step snapshot stays strict; retired when upstream absorbs it | sawyer | 2026-11-03 | `check:killdates` | - -The pin is `79adc433` (upstream `origin/main`, 2026-09-17), re-pinned from -`692c3106`/`c1b5bb73` (CL-8206). npm is still `0.3.0`, so every tree a -re-pinned tree imports at a newer API is vendored too, at the same commit — -a vendored tree never mixes pins: `agent`, `db`, `harness`, `hub-agent`, -`hub-api`, `hub-common`, `hub-sessions`, `inference`, `mail-memory`, -`mailbox`, `mime`, `types`, `workflow`, `workflow-deploy`, and -`workflow-host`, plus `apps/sidecar`. The npm-consumed names whose source -is still byte-identical at this pin stay on npm `0.3.0`: `authz`, `crypto`, -`log`, `pack-transport`, `storage-isogit`, `tool-packaging`, -`inference-catalog`. The root `package.json` `overrides` point every -vendored name at `workspace:*` (so the published `@intx/harness`, -`@intx/hub-agent`, `@intx/tool-packaging`, `@intx/authz`, … resolve their -own `@intx/*` dependencies onto the vendored copies instead of a second npm -copy) and every unchanged name at `0.3.0`. - -CL-8206 re-vendored every `vendor/intx/*` tree at `79adc433` and dropped -every delta upstream has since absorbed: the per-principal signing-key -series (`vendor/intx/db`, `hub-api`, `hub-common`), CL-7190's -`beforeToolExtension` (`vendor/intx/agent`) and `"message_response"` signal -kind (`vendor/intx/types`, `vendor/intx/inference`), and CL-6396's -`childRunId` forward (`vendor/intx/hub-sessions`, `vendor/intx/types`). It -is also a hard cutover, not a port, for two capabilities upstream deleted -outright with no replacement: `@intx/db`'s frozen wire-projection storage -(the `workflow_definition.origin` column and -`workflow_definition_version.wire_projection`, CL-6324/CL-6452) and -`@intx/hub-sessions`' `SessionService.sendUserMessage` (every caller now -delivers through `SidecarRouter.routeMail`, mirroring -`apps/hub/src/native-workflow-routine-launch.ts`'s existing pattern). See -each row's own cell and the retired call sites in `packages/workflows`, -`packages/agent-directory`, `packages/webhook-triggers`, `packages/chat`, -and `apps/hub` for what was deleted rather than reimplemented. - -Local modifications (all surviving `vendor/intx/*` rows): each package's -exports map is repointed from the upstream `intx-src` resolve condition to -direct TypeScript source resolution (`types`/`default` → `./src/...`), with -`dist/` references and the `customConditions` entry in the shared tsconfig -removed — workbench forbids custom resolve conditions — and each tsconfig -carries `types: ["bun"]`. `vendor/intx/*` (CL-7226) additionally gained the -workspace-wide TypeScript project-references cutover: a `tsconfig.src.json` -with `composite: true` and `references` generated from real workspace -`dependencies`, while `tsconfig.json` stays the combined, conventionally -named project (extends `tsconfig.src.json`, adds `test`) so tools that -discover a project by looking for a file literally named `tsconfig.json` -— ESLint's `projectService`, an editor, a bare `tsc` invocation — still -find one without being told the composite project exists at all. The same -shape every other package in the workspace got in the same change, so a -vendored package's build participates in `tsc --build` like any other. -`vendor/intx/hub-api` (CL-6345) accepts -`principalId: null` on `resolveApproval` for a decision a standing-grant -allowance already authorized, skipping the per-principal resolve gate the -HTTP routes still enforce. `vendor/intx/hub-api` (CL-workflow-deploy-bearer) adds -`middleware/workflow-run-deploy-auth.ts`, an optional bearer-authenticated -mirror onto the SAME session-cookie `POST/GET .../workflows/deployments` -route `routes/workflows.ts` already mounts (that route file is otherwise -untouched — no deploy logic is duplicated). It is the missing bearer path -every other workflow-run write surface (skills, capabilities, routines, -agent-directory) already has: a workflow-run agent has no browser session, -only its sidecar bearer token and its own run address. `MountHubRoutesDeps` -and `CreateAppOpts` gain an optional `workflowRunAuthenticator`; when -supplied, the new middleware mounts ahead of `createResolveTenant` (which -already short-circuits once `principal`/`tenant` are set — the same seam the -git-token/asset bearer routes use), so a bearer-authenticated request reaches -the exact same handler as a human session: same `requireGrant("workflow:*", -"create")` gate (already in `SEED_GRANTS`, so no grant changes were needed), -same asset tenant-scoping (a foreign-tenant asset already read as `not_found` -before this change and still does), same install/probe/gate/freeze call into -`sessionService.deployWorkflowFromSource`. A request with no bearer -credential falls through unchanged to the session path. -`vendor/intx/hub-sessions` also drops the -live-status gate on `receiveWorkflowRunPack`'s anchor lookup: the gate is now -the exported pure helper `ownsWorkflowRunRepo` (a self-anchored `workflow_run` -row with a routable address), with the allocation fences unchanged. Upstream -required `status in (deployed, running)`, which wedged every terminal run that -still had mail in flight into a permanent loop — the run's own inbox-enqueue -and `markConsumed` rejection packs were refused as `path_violation`, the -sidecar withheld the ack, and the hub redelivered forever -(`docs/revendor-inventory.md`). `vendor/intx/hub-sessions` (CL-6361) also -widens that same anchor lookup to resolve a per-step pack source address -(`deriveStepAddress`'s `-@`, -`@intx/workflow-deploy`'s `orchestrator.ts`) back to its base run's -anchor address via the new pure helper `anchorAddressForPackSource`. Upstream -keys the lookup on an exact `workflow_run.address` match, which only the -anchor row ever carries; a multi-step deployment's per-step agents push their -own event-log commits (e.g. a step named `write`) under their step-suffixed -address, so every such pack was rejected `path_violation` with "source -address has no deployment anchor it owns," the sidecar withheld the ack, and -the hub redelivered forever — the same infinite-retry shape as the terminal-run -case above, one layer up the address hierarchy. `vendor/intx/workflow-host` (CL-6448) threads the parent child's -credentials-backed authorize, live `CredentialWiring` and `MailPartReader` -through the suspendable-child (onTrigger body) spawn seam: -`RunSuspendableChild`'s input and `createInMemorySpawnSuspendableChild`'s -opts gain the three optional fields, and `run-child.ts` passes them when -building the body resolver, so a body agent's tool calls gate through the -same per-step grant snapshot a top-level step's do, its tool bundles resolve -credentials, and an attachments-only inbound mail resolves its parts instead -of throwing. `findStepGrantsEntry` collapses a single-step deployment's sole -grants entry onto a body step whose own id the parent snapshot never lists. -Upstream never runs tool-bearing body agents, so the seam has no upstream -analog. `vendor/intx/workflow` carries one local delta (corbitsdev/workbench#709 step-timer budget — see its ledger row): upstream `b977ade6` ships the -`onTrigger` body-failure policy workbench had vendored as -`onBodyFailure: "continue"` (CL-6326, CL-6324) under the literal `"tolerate"`, -so the authoring site (`@corbits/agent-runtime`) says `"tolerate"` and `@intx/db` -migration `0089` rewrites the retired literal inside stored wire projections. `vendor/intx/db` and `vendor/intx/hub-sessions` (CL-6324) together -persist a definition's evaluated inert projection at approval time: -`workflow_definition_version` gains a `wire_projection` jsonb column -(migration `0087_workflow_definition_version_wire_projection.sql`, renumbered -from `0086` when upstream `692c3106` took that slot and from `0085`/`0084` before -that; the drizzle-kit journal keeps our original `when` timestamps, which -still sort after upstream's `0085`. Workbench applies the platform SQL by -sorted filename through `scripts/db-setup.ts`, which replays from scratch -and refuses a schema set up under a different file list, so a database -migrated before a renumber is reset, never patched — `scripts/db-setup.test.ts` -pins that refusal), -`createDbFrozenApprovalWriter` stamps it in the SAME transaction that -writes `approved_wire_hash`, and `loadFrozenWireProjection` reads it back -validated as a `WorkflowProjectionDefinition`. Upstream carries no -hub-side record of a deployed definition's body at all — under the -`workflow.json` retirement the body is whatever the source closure -evaluates to on the sidecar, and a source-format asset holds no envelope -to read it back from — so every hub-side launch that needs the body (a -folded run's system prompt, tool pins, model, credential bindings) had -nowhere to get it. Keyed to the approved wire hash and stored beside it, -this is one store per concept, not a second copy: the projection and the -hash that addresses it are written and read together. -`vendor/intx/hub-sessions` (CL-6379) classifies an accepted workflow-run -pack's newly-terminal runs through the new pure `decideTerminalRunFlip` -before the DB flip: a section occurrence's repo-local child run -(`turn__`) has no `workflow_run` row by design and is skipped quietly -instead of being logged as a foreign-deployment violation on every turn. -`vendor/intx/hub-sessions` (CL-6478) adds a -`sanitize-tool-name.ts` module and calls it from `event-collector.ts`'s -`tool_call` handling: `@intx/inference`'s `decodeToolName` is deliberately -total and returns a hallucinated or provider-mangled function name -verbatim, but `encodeToolName` throws when that same name is put back on -the wire for the next turn's outbound request — persisting a decoded name -unchecked meant the room's very next turn died rebuilding its request, -permanently, once the bad name was durable. Only a name `encodeToolName` -can re-invert is now persisted as-is; anything else collapses to a stable -`malformed_tool_call` placeholder before it reaches `turnPart`, so a single -bad tool-call name fails that turn cleanly instead of wedging the room. -`@intx/inference` is added to `vendor/intx/hub-sessions`'s own -`package.json` dependencies for this. -`vendor/intx/hub-sessions` (CL-6595) fixes `workflow-run-kind.ts`'s -newly-terminal detection, which skipped a run's `events.jsonl` subtree -entirely (`enumerateEventBlobs` only walks per-event `.json` files), -so a run sealed from birth — its whole event log arriving pre-combined in -one push, with no per-event blobs ever landing — never fired `markTerminal` -and stayed "running" in `workflow_run.status` forever despite the run -having genuinely finished; `validatePush` now also scans a newly-sealed -run's combined log for its terminal event, and `hub-session-lookups.ts` -gained a same-push defense-in-depth backfill via the new -`readCommittedWorkflowRunTerminalStatus` export, in case a future pack still -slips past the primary detection. -Each package's `VENDORED-FROM` file restates its own delta. - -`apps/sidecar` records `692c3106`: the fork tracks the closure-sourced -lineage and, at this pin, adopts upstream's warm-agent mailbox (the -supervisor-backed transport's inbound half, the mailbox watch registry and -mutation bridge, the connector-thread seed and reply drain), the -self-terminated-supervisor address reclaim, the live-child grants refresh -after an approval, the `agentDir` deploy-tree lookup and the disposer -`AggregateError`. Its defining delta stays: onTrigger bodies run WITH tools -(`bodyInvokeStep`), and every step env -- body or top-level -- gets the -mailbox read surface. - -### Un-vendoring `vendor/intx` - -- [ ] Delete `vendor/intx/` and remove `vendor/intx/*` from the root - `package.json` workspaces. +Pinned to [faremeter/interchange](https://github.com/faremeter/interchange) +@ `79adc433` (origin/main). + +| Vendored path | What it is | Owner | Kill date | +| ------------------------------ | ------------------------------------------------------------------------ | ------ | ---------- | +| `apps/sidecar` | Byte-identical copy of upstream's sidecar app; apps are never npm-published | sawyer | 2026-11-03 | +| `vendor/intx/agent` | `@intx/agent` at a newer commit than npm has published | sawyer | 2026-11-03 | +| `vendor/intx/db` | `@intx/db`, with one local serving-time credential-refresh delta | sawyer | 2026-11-03 | +| `vendor/intx/harness` | `@intx/harness` at a newer commit than npm has published | sawyer | 2026-11-03 | +| `vendor/intx/hub-agent` | `@intx/hub-agent`, with a local OAuth-login-over-websocket delta | sawyer | 2026-11-03 | +| `vendor/intx/hub-api` | `@intx/hub-api`, with local approval and workflow-deploy-bearer routes | sawyer | 2026-11-03 | +| `vendor/intx/hub-sessions` | `@intx/hub-sessions`, with several local run/pack-acceptance fixes | sawyer | 2026-11-03 | +| `vendor/intx/inference` | `@intx/inference`, with one local Google file-upload compile fix | sawyer | 2026-11-03 | +| `vendor/intx/mail-memory` | `@intx/mail-memory` at a newer commit than npm has published | sawyer | 2026-11-03 | +| `vendor/intx/mailbox` | `@intx/mailbox`, never yet published to npm | sawyer | 2026-11-03 | +| `vendor/intx/mime` | `@intx/mime` at a newer commit than npm has published | sawyer | 2026-11-03 | +| `vendor/intx/types` | `@intx/types`, with one local model-provider-plugin enum addition | sawyer | 2026-11-03 | +| `vendor/intx/hub-common` | `@intx/hub-common` at a newer commit than npm has published | sawyer | 2026-11-03 | +| `vendor/intx/workflow` | `@intx/workflow`, with one local step-timeout-budget delta | sawyer | 2026-11-03 | +| `vendor/intx/workflow-deploy` | `@intx/workflow-deploy` at a newer commit than npm has published | sawyer | 2026-11-03 | +| `vendor/intx/workflow-host` | `@intx/workflow-host`, with one local step-grants-collapse delta | sawyer | 2026-11-03 | + +npm's published `0.3.0` predates this pin for every row above, so each is +re-vendored at the same commit rather than mixed pins; the root +`package.json` `overrides` point each vendored name at `workspace:*` so +their own `@intx/*` dependencies resolve onto the vendored copies too. +Local deltas are restated in each package's own `VENDORED-FROM` file. + +## Un-vendoring `vendor/intx` + +- [ ] Delete `vendor/intx/` and remove it from the root `package.json` + workspaces. - [ ] Restore the `@intx/*` dependencies in `apps/*`, `packages/*`, and - `workflows/*` to the published npm version that covers each surviving - tree's delta, and drop the root `overrides` pins. -- [ ] Delete the `vendor/intx/*` ledger rows above and the local-modifications note. -- [ ] Drop the `vendor/intx/*` rows from `scripts/checks/kill-dates.txt`. + `workflows/*` to the published npm version, and drop the root + `overrides` pins. +- [ ] Delete the `vendor/intx/*` ledger rows above. - [ ] `bun install` - [ ] `bun run check` diff --git a/docs/AGENTS-PAGE.md b/docs/AGENTS-PAGE.md deleted file mode 100644 index f572b62d6e..0000000000 --- a/docs/AGENTS-PAGE.md +++ /dev/null @@ -1,85 +0,0 @@ -# Settings · Agents - -Settings · Agents (`apps/web/src/pages/agents-settings-section.tsx`) is a -bench's one surface for its agents: the definitions a person can launch -from, and the instances currently running from them. It is a Settings -section, not a rail destination (CL-5990) — agent definitions are the only -thing this surface manages; talking to an agent is a chat (Start chat), and -looping one into a conversation is a workbench mention. - -## Definitions and instances - -A **definition** (`workflow_definition`) is a reusable template: a name, an -optional description, a system prompt, and (optionally) a model, folded into -a single-step workflow asset. A definition's status is `deployed` -(launchable) or `stopped`. - -An **instance** (`workflow_run`) is a live launch of a definition — a -running agent with its own mailbox address. An instance's status is one of -`deployed`, `running`, `updating`, `error`, or `stopped`, exactly the -vocabulary `GET /api/tenants/:tenantId/workflows/runs` reports. - -The page lists both in one place, as two tabs sharing one search box and one -grid/table view toggle. An instance whose `definitionId` does not resolve -against the tenant's own definitions listing is marked **Unlinked -definition** rather than hidden — the page never silently drops a row it -cannot fully explain. - -Every list excludes legacy workbench-host definitions -(`@corbits/chat/workbench-host-naming`'s `isWorkbenchHostDefinitionName`): -pre-CL-6330 databases may still carry those rows, and they were internal -plumbing, not agents a person created. - -## Creating an agent - -The "Create agent" action opens a form for identity (name, handle, -description) and definition (system prompt, model) and posts to -`POST /api/tenants/:tenantId/agent-definitions`, added by -`@corbits/agent-directory`. The route: - -1. Builds a single-step, folded workflow definition from the submitted - fields (`buildAgentDefinitionWorkflow`) — the same shape - `@corbits/assistant-workflow` and `@corbits/chat`'s workbench host produce, - parametrized instead of fixed — and renders it as a source codebase - (`@corbits/workflows`'s `./source`'s `renderWorkflowSourceTree`), never a bare - `workflow.json` envelope. `workflow.json` is retired; nothing writes it. -2. Creates a `workflow`-kind asset and writes that source tree into it - in-process (`AssetService.populateAsset` — no git subprocess), which - produces a commit. -3. Deploys that commit through Interchange's native source pipeline - (install -> sidecar probe -> gate -> freeze) via the same - `WorkflowDeployer` `@corbits/workflows`'s `./authoring`'s - agent-authored-workflow registry calls, which projects the first-class - `workflow_definition` row over the asset (CL-7363). - -The definition lands with the schema's default status (`deployed`) and a -materialized asset, so it is immediately invitable and launchable — no -separate deploy step, and no page reload needed to see it appear. Every -subsequent edit (instructions, model, tools, skills, restore) writes a new -commit and redeploys the same way; a definition frozen before CL-7363 -stays launchable as-is and only redeploys through the native pipeline on -its next edit — no data migration. - -A deploy that finds no connected sidecar fails the write outright (502, -`unavailable`) rather than falling back to the old bare-freeze path — no -fallback, per this repo's ground rules. - -**Tools and a model provider are not exposed on the create form.** The -platform's wire contract for a workflow definition -(`WorkflowDefinitionResponse` in `@intx/types`) carries no tool-package -field, and `@intx/agent`'s `defineAgent` does not accept a caller-supplied -`toolPackagePins` on its authoring-time config — it is vendored, read-only -source. The five catalog workflow packages (`granola-call`, -`process-granola-call`, `morning-brief`, `pain-point-collateral`, -`collateral-generation`) build their `AgentDefinition`s directly against -that type instead, and do set `toolPackagePins` (CL-5999) — but this -create-form path is unrelated to those packages and still has no field to -carry one. A model is accepted as a bare canonical name only; the provider -is resolved fresh against the tenant's catalog at launch time -(`resolveDefinitionSources`), never baked into the definition. - -## The mailbox address - -An instance's mailbox address (`WorkflowRunResponse.address`) is never -rendered as visible text on this page. It is reachable only through a -"Copy address" control that writes it to the clipboard. diff --git a/docs/AGENT_TEST.md b/docs/AGENT_TEST.md deleted file mode 100644 index 4c36a57531..0000000000 --- a/docs/AGENT_TEST.md +++ /dev/null @@ -1,159 +0,0 @@ -# Agent Test Program - -A graduated series of live tests an agent (or a person) runs against a real -Workbench stack to validate that agents actually work — not that CI is -green. Every level is a checklist with pass criteria and an evidence rule. -Levels build on each other: do not certify a level until every level below -it holds on the same build. - -**Ground rules** - -- Run against a real inference provider. We use Ollama on a remote box; - the provider is interchangeable — the program is not. -- Drive the real UI (`agent-browser`) or the real API, never internals. - Follow `.claude/skills/ui-test` for evidence discipline: scoped - snapshots, reproduce twice, timings noted. -- Model latency is not failure. UI dishonesty about latency is. -- Every failure gets a Linear ticket with the exact repro; every pass is - stated plainly. Silence is the only unacceptable outcome — from the - product or from the tester. - -## L0 — The loop - -The baseline that everything else assumes. - -- [ ] Fresh signup → connect a provider → first workbench → send a - message → the agent replies. One sitting, no seed accounts. -- [ ] The reply is clean: no raw chain-of-thought, no internal package - paths, no protocol strings. -- [ ] A failed turn says why, carries a reference id, and Retry resends - without touching the composer. -- [ ] Restart the stack. The same room answers again without recreating - anything. - -## L1 — Concurrency and queuing - -Multiple messages, one room. The product must never lose or misattribute -a turn; honest queuing beats silent dropping every time. - -- [ ] Same user sends 3 messages rapid-fire before any reply lands. All - are answered (batched or serial), none lost, order legible. -- [ ] Two users (two browser sessions) message the same agent - simultaneously. Both get answers; attribution is correct. -- [ ] **One agent, five people**: five sessions message one agent within - seconds. Expected: turns queue per agent; every sender either gets - an answer or an honest notice. Measure: time-to-first-feedback for - the 5th sender; nothing silently >120s (turn deadline). -- [ ] While an agent is generating, send it another message. The second - turn queues; the first reply still lands intact. -- [ ] Typing indicators are per-agent and truthful: they show while a - turn is running, clear when it ends, and never show for an idle - agent. - -## L2 — Multi-agent coordination - -Several agents, one room. The room must stay legible. - -- [ ] @A a question; while A generates, @B another. **Both replies - land**, attributed correctly (the CL-6670 class). -- [ ] Un-addressed message in a multi-agent room: behavior is predictable - and documented (host answers). A new user could guess it. -- [ ] Three or more agents: header identity, distinct typing indicators, - no interleaving confusion in the timeline. -- [ ] **Agent↔agent mail dispatch**: agent A asks agent B for something - through the platform (mail), B's answer reaches A, and the room - shows enough of the exchange that a human can follow the work. -- [ ] Tag-dispatch fan-out: one message @-ing two agents produces two - attributed replies, not one merged or dropped. - -## L3 — Routines - -Scheduled and triggered work, including agents building it. - -- [ ] Create a routine in the UI; Run now; the result lands where - promised and the run's status reaches a terminal state everywhere - it is shown (list, detail, Mission Control, Insights) — no - eternal "Running now". -- [ ] Schedule a routine (near-future cron); it fires unattended. -- [ ] **An agent creates a routine** from a chat instruction ("check X - every morning and post here"); the routine exists, is visible, - correctly configured, and fires. -- [ ] An agent edits/disables a routine on request. -- [ ] A routine that needs a missing capability fails with a named, - actionable reason — never a silent no-op. - -## L4 — Capabilities: plugins, skills, tools - -Agents extending what they can do. - -- [ ] **Plugins pull-in**: an agent uses an already-connected plugin - (e.g. posts a GIF via Giphy) without ceremony. -- [ ] **Plugin recommendation**: asked to do something requiring an - unconnected plugin, the agent recommends connecting it by name and - the connect flow is one click away — credentials are collected - BEFORE the capability is exercised, per the guided-join rule. -- [ ] Agent-declared slash commands register on join and vanish on leave. -- [ ] **Auto-creating skills**: ask an agent to capture a working method - as a skill; the skill exists, is versioned, pinned to the agent, - and demonstrably changes behavior on next use. -- [ ] **Creating tools**: where tool authoring is exposed, an agent - drafts a tool, the human approves, and the tool is callable. Every - external side effect sits behind approval — verify the approval - actually gates. - -## L5 — Agent lifecycle - -Creating and changing agents, and the changes actually taking effect. - -- [ ] Create an agent (model, prompt) in the UI; invite; it answers in - character. One click = one agent (no duplicate records). -- [ ] **Update an agent's instructions**; the next reply in an existing - room obeys the edit (reconcile-on-wake — the CL-6588 class). Other - rooms pick it up on their next turn. -- [ ] Change an agent's model; verify the switch took (ask it to - identify; check Insights attribution). -- [ ] Create an agent on every chat-capable catalog model; each replies. - Non-chat models never appear in the picker. -- [ ] Delete/archive an agent: rooms show an honest state, history - preserved, no undead participant. - -## L6 — Memory and self-improvement - -Per-bench learning. Myra should get better at _this_ bench. - -- [ ] Tell an agent a durable fact ("our fiscal year starts in Feb"). - New conversation, same bench: the fact is used without re-telling. -- [ ] Memory is inspectable: a person can see what the bench remembers - (read-only surface) and it matches what was taught. -- [ ] **Myra self-improvement per bench**: after correcting Myra's - behavior twice ("always answer in bullet points here"), the third - ask complies without the correction. Verify the mechanism is - per-bench: another bench's Myra is unaffected. -- [ ] Memory survives stack restart and agent sleep/wake. -- [ ] Memory respects boundaries: nothing from bench A surfaces in - bench B; nothing from tenant A surfaces in tenant B. - -## L7 — Instrumentation: insights and evals - -The system's account of itself must be true. - -- [ ] **Insights**: after a known workload (e.g. exactly 5 turns across - 2 agents), turn counts, per-workbench breakdowns, latency, and - token numbers reflect it. Local models may honestly show $0 cost; - zeros anywhere else must trace to a real reason, not a broken - projection. -- [ ] Run detail/trace pages agree with the room: a delivered reply's - run shows terminal, its steps visible. -- [ ] **Evals**: run an eval against an agent (when shipped); results - are reproducible run-to-run within stated variance; an empty evals - surface says so honestly rather than implying data. -- [ ] Mission Control's numbers agree with each other and with Insights - for the same window. - -## Cadence - -- **Per merge wave**: re-run the levels the changes touch. -- **Per release candidate**: full L0–L7 on one build, one sitting. -- **Grow the list**: every production incident adds its repro here as a - permanent checklist line (this file is the regression memory of the - agent platform). diff --git a/docs/CHAT.md b/docs/CHAT.md deleted file mode 100644 index b996acf3b9..0000000000 --- a/docs/CHAT.md +++ /dev/null @@ -1,813 +0,0 @@ -# Chat - -Chat is Workbench's shared conversation surface: teams and agents read and -write the same timeline, in the same [bench](GLOSSARY.md). A message is -workbench data — a row the hub writes and reads directly — and asking an -agent for a turn is a separate act, over the mail transport Interchange -already gives every agent. It ships as two -packages — `@corbits/chat` (the HTTP surface and domain logic) and -`@corbits/chat-ui` (the React components a host renders it with) — composed -onto the hub and the web app respectively. - -## What a workbench is - -A workbench holds a **timeline**: the rows in `chat.workbench_messages` -belonging to that workbench, read back in order. Posting a message is one -insert plus one publish onto the workbench's live stream (CL-6327) — no mail, -no wake, no sidecar hop — so a workbench takes messages and renders them -whether or not a single agent process is running. - -A workbench has no run of its own (CL-6330): creating one mints a child -tenant and writes settings rows — no deploy, no anchor instance. Its -address (`@`) is derived, not resolved; a message -reaches an agent only when the workbench asks that agent for a turn. - -A workbench is also its own tenant, parented under the bench it was created -in, so its membership and permissions are native grants rather than a -chat-specific system — see [workbench-tenancy.md](workbench-tenancy.md) for -the mint, listing, and move mechanics. - -```mermaid -flowchart LR - subgraph Workbench - Room[Room timeline
workbench_messages rows] - end - Alice[Human participant] -->|POST message| Room - Room -->|timeline read| UI[Chat UI] - Room -.->|turn dispatch, mail| Bot["@handle agent participant"] - Bot -.->|connector.reply, bridged| Room -``` - -## The message model - -A message is a list of MIME parts, not a single string. The parts a workbench -supports: - -- **text** — plain message text. -- **event** — a structured, machine-readable record (e.g. a participant - joining, a settings change) rendered distinctly from authored text. -- **attachment / file** — a named blob with a media type, referenced by id - rather than inlined, so large files are never pulled into memory just to - render a message list. -- **block** — a generative UI card: a `{ type, data }` envelope whose data - is agent-authored JSON, never markup or code. The typed vocabulary lives - in `@corbits/chat`'s `blocks` module — `approve`, `steps`, `metrics`, - `poll`, `form`, and `stream` — and `@corbits/chat-ui` renders each type - through a closed component registry, parsing the data at the render - boundary so an unknown type or malformed payload degrades to a labeled - fallback card. An approve block carries only a reference to a platform - approval plus the agent's framing (title, risk, body): action labels are - fixed by the client and the decision's state lives on the approval - record, never in the message. Poll choices carry no agent-authored - tallies. Blocks render read-only today; their controls stay disabled - until the action round-trip ships. - -A message stores its `Part[]` as it is, so reading a timeline is a query, not -a decode. Asking an agent for a turn still encodes the parts onto the -platform's mail-send shape — a single text part rides as bare mail content; -anything else becomes a list of `text/plain`/`application/json` MIME -attachments — and that encoding is confined to the dispatch seam. - -The dispatched frame itself carries RFC 5322 threading (CL-7450): its -`Message-ID` is `mailMessageIdFor(sourceMessageId, domain)` — the same -derivation `mailboxFanOutForSend` already stamped for that row, so the two -always agree — and `In-Reply-To`/`References` name that row's own parent -chain (`mailAncestryOf` in `./threads.ts`). This is what lets a reply's own -`In-Reply-To` correlate back to the exact row that triggered the turn, -independent of the process-local `turnMailCorrelation` bookkeeping -`dispatchTurn` also does. Degrades to unthreaded, exactly as before, on a -composition with no mailbox domain to derive it from. - -## Every human's mailbox gets a copy (CL-7450) - -`sendWorkbenchMessage` runs a strict order: store the row -> stamp its -RFC 5322 `Message-ID` -> fan the message into every human participant's -`@corbits/mailbox` inbox, as one batch -> publish the row onto the room's -live stream -> dispatch to whichever agents the message names. The mailbox -fan-out runs BEFORE the row is published and BEFORE any agent is asked for -a turn: a fan-out failure fails the send outright — the just-inserted row is -deleted and the failure is rethrown to the caller, never swallowed into a -send that looks fully delivered when it wasn't. Only once the fan-out batch -has committed does the row publish, giving a client-visible bubble and a -durable mailbox copy the same all-or-nothing guarantee. - -This is not transactional across the row insert and the delete: the row -write and the mailbox batch are two separate commits (chat and -`@corbits/mailbox` are separate Postgres handles in the hub's own -composition), so a concurrent `GET` of the timeline in the window between -the insert and a fan-out failure's delete CAN read the row before it is -removed — no `chat.message` publish has happened yet (so no live client -sees it), but a poller hitting the REST list endpoint in that same window -can. The window is real and bounded to that one gap, not claimed away. A -fan-out failure is reported once, under one `refId`, as -`MailboxFanoutFailedError` (`./mailbox-fanout.ts`) — the send route -answers with that same ref rather than letting the hub's generic -unhandled-route-error handler report it again under a second one. - -The fan-out itself writes an "outbound" row in the sender's own mailbox and -an "inbound" row in every other human participant's, all sharing the -timeline row's own `Message-ID` (``, stamped onto -`workbench_messages.mail_message_id` — see `./mail-headers.ts`) as the -frame's threading identity; the mailbox write's own idempotency key is left -to `@corbits/mailbox`'s default (a transport key derived from that same -Message-ID plus each recipient's principal and direction), never a -caller-minted one. The whole batch is ONE `@corbits/mailbox` transaction -(`writeMailboxMessages`): every row commits together or none does, so a -retry after a genuine failure re-attempts every recipient rather than -risking a partial delivery. An agent participant never gets a mailbox row -this way — its inbox is its run's own live mail queue, reached through -`WorkbenchMail.sendMail` instead. Each row carries a -`{ kind: "workbench", id }` ref back to the room it came from, and threads -under its parent via `In-Reply-To`/`References` when the message answers -one (`mailAncestryOf` in `./threads.ts`). - -A shared workbench's rows live in the OWNING tenant, never the acting -(projected) tenant a share member sends from: the row's `Message-ID` and any -`In-Reply-To` are minted against the owning tenant's own mail domain, and -every recipient — the sender included — is addressed under that same -domain. The acting sender's principal is resolved in the OWNING tenant, not -the tenant they authenticated against; when a share member has no principal -row there at all, that sender's own outbound copy is skipped quietly (a -debug log, not a report) since it is the expected shape for a share member, -while every other human participant still gets their row. Any OTHER -participant address with no matching principal in the tenant — a stale or -removed member — is reported and skipped rather than attempted; any other -batch failure is reported and re-thrown. See `./mailbox-fanout.ts` for the -fan-out logic and its `MailboxWriter` seam, and `./platform-port.ts`/ -`routes.ts`'s `mailbox` dep for how a host wires a live `@corbits/mailbox` -instance in. - -## One turn in flight per workbench (CL-6331) - -A workbench claims itself before it asks any agent for a turn: `dispatchTurn` -runs behind a `WorkbenchTurnQueue` (`packages/chat/src/turn-queue.ts`), keyed -by `workbenchId` rather than thread — an owner call, since a workbench's -agents share one room and a second agent starting a turn while the first is -mid-review is exactly the collision this closes. A message arriving while a -turn is already in flight for its workbench queues instead of dispatching, -and the room is told live (`chat.turn-queued`, non-persisted — a queued -message's own row already carries it on the timeline; this is only the -"still waiting" signal a client renders as a queued strip). Once the -in-flight turn's claim releases, everything that queued behind it dispatches -together as **one** next turn, recipients unioned and parts concatenated in -arrival order — never as N separate turns replaying one after another. - -The claim (`packages/chat/src/turn-claims.ts`) reuses the tryClaim/release -shape `write-claims.ts` proved out for the finalized-turn write surfaces, -not the same table: a turn claim is in-memory, process-local state, released -once `dispatchTurn`'s own call settles. That is an honest, disclosed gap — -at this seam `dispatchTurn` only reaches "the mail was handed to the agent's -mailbox," not "the agent's turn actually finished" (the real turn, and its -own completion signal, move to this seam in CL-6329) — so a claim also -expires on a TTL (`turnTimeoutMs`) as the backstop against a dispatch that -never settles at all, rather than wedging a workbench behind it forever. - -A host that composes more than one send surface against the same -`ChatPlatform` (the hub wires `createChatRoutes`, the workflow-participant -router, and the Slack tag mount this way) constructs one `WorkbenchTurnQueue` -and injects it everywhere, the same "one instance, shared" pattern -`workbenchSubscribers` already follows — otherwise each surface would only -serialize against its own traffic, not the others'. - -## Turn = run: room agents as `onTrigger` sections (CL-6329) - -Every agent invited into a room deploys as an **`onTrigger` section keyed -on (agent, workbench)**: one warm run per pair, every message an -occurrence, every occurrence its own child run with its own run id and -event log. The shape lives in `@corbits/agent-runtime` -(`buildAgentRuntimeWorkflow`, `mode: "section"`), and -`platform-adapter.ts`'s `ROOM_AGENT_MODE` is the single place that pins -it — one switch feeding both the wake and the relaunch path. The -workbench itself has no host run and no occurrences to name; only invited -agents deploy as sections. - -`onBodyFailure: "tolerate"` — authored in the section shape itself — is -the failure edge: a turn that ends `failed` records the failed occurrence -and leaves the section subscribed, so one bad turn kills neither the agent -nor the room. The runtime names an occurrence's child run `turn__`, -which is what `agentRuntimeTurnRunId` derives and what a reply's `run_id` -carries. - -- **The dispatch seam** (`dispatchTurn` in `workbench-service.ts`) opens - a projection row before it touches the execution plane, so an in-flight - turn is visible from its first moment and the child run id its reply - will carry is already allocated. Firing the section's trigger is still - a `sendMail`, because a mail trigger is what the section subscribes on - — but what it starts is an occurrence, not another turn folded into one - endless step. -- **Context assembly** (`packages/chat/src/turn-context.ts`) — - `assembleTurnContext` builds the conversation a turn is asked with from - message rows: the turn's own thread (never the whole room when a thread - is named), capped to the workbench's resolved `chat/contextWindow`, with - the dropped span folded into one bounded recap rather than silently - lost. Thread membership lives in its own store, so the scope is injected - as a `TurnContextThreadScope` rather than this module reaching for a - second store. -- **The turn projection** (`packages/chat/src/agent-turns.ts`, table - `chat.agent_turns`) — one row per turn, opened as the turn starts and - closed as it settles, carrying the child run id, the messages it was - asked to answer, the message it produced, and how it ended. This is - deliberately **our** projection rather than a read of the platform's own - run tables: a room has to answer "which run produced this reply, and how - did that turn end" from its own rows, at timeline speed, whether or not - the execution plane is reachable. Occurrence allocation happens inside - the insert with a unique index behind it, so two dispatches racing for - one agent can never quietly share a child run id. `GET -/workbenches/:id/turns` and `GET /workbenches/:id/turns/:turnId` serve - it, following the same "no store, no feature" contract `pins` already - does. - -**Where a reply finds its turn.** The sidecar's `agent.event` frames carry -the agent's address and nothing finer — no occurrence id — so the reply -path matches on the newest still-`running` turn for (workbench, agent). -The one-in-flight-turn-per-workbench claim is what makes that -unambiguous in the common case; under a burst that opens more turns than -the runtime runs occurrences, older rows can be left `running`. Closing -that gap properly needs the runtime to report the occurrence id on the -event, not a heuristic here. - -**Stale turns fail, never zombie (CL-6451).** A dispatch can die without -any closing event ever reaching the hub — the workflow-host supervisor's -terminal-or-park backstop failing it, or the process dying mid-turn. An -occurrence cannot legitimately outlive the per-turn timeout the section -body enforces, so both turn stores fail any row still `running` past -that timeout plus a settle grace (`AGENT_TURN_STALE_MS`) on their next -read or write: the room shows a failed turn instead of typing forever, -and the reply path can never attribute a later reply to a dead row. - -**A model that cannot resolve is a failed turn, not a 500.** Wake and -mint map an unresolvable inference source to a consumer 4xx and a -failed-turn strip ("Jimmy's model isn't available here.") with an -inline picker of tenant-available chat models and a hop into that -agent's Settings — never a raw HTTP 500 or the technical resolution -dump on the timeline. - -Proved live end to end by `scripts/e2e/cl-6329-turn-swap-proof.ts`: two -agents replying in one room under distinct occurrences, three rapid -messages serializing into ordered turns, and a sidecar killed -mid-occurrence leaving both the room and the section alive. - -## Stopping a turn (CL-7201) - -Before this, the only bound on a wedged turn was the dispatch and -wait-until-free timeouts themselves — minutes long, and no way for a -user watching an agent go wrong to do anything but wait or reload. -`POST /workbenches/:id/turns/cancel` (`packages/chat/src/routes.ts`) -closes that gap, calling `cancelWorkbenchTurn` -(`packages/chat/src/workbench-service.ts`), which runs two independent -mechanisms together because a turn can be in either place when the user -asks to stop it: - -- **Still on our own call stack.** `dispatchTurnBatch` registers one - `AbortController` per recipient it dispatches, via a workbench-keyed - `TurnCancelRegistry` (`packages/chat/src/turn-cancellation.ts`) — - the same "one instance, shared" pattern `WorkbenchTurnQueue` follows. - Its signal is composed into each `withTimeout` call (`waitUntilFree`, - `dispatchTurn`) as an **external signal** — `withTimeout` (CL-7193) - already gives `work` an `AbortSignal` the moment its own timeout wins; - CL-7201 extends it to also fire (with the external caller's own - reason, not its own timeout message) the moment that external signal - aborts, whichever comes first. `dispatchTurn`'s abort-close handler - tells a deliberate cancellation apart from a timeout by checking - whether the abort reason is a `TurnCancelledError`, and closes the - turn row `cancelled` rather than `failed`. -- **Already off our call stack.** `sendMail` has no cancellable - primitive of its own (CL-7230) — once it resolves, the agent is - generating (or parked on an approval gate somewhere in the execution - plane this package cannot see into) with nothing left - registered to abort. `cancelWorkbenchTurn` snapshots every turn - `AgentTurnStore.findRunningTurns` reports for the workbench _before_ - triggering the registry above (so a row the abort path already - claimed is still counted), then sweeps it directly through the same - `finishTurn` compare-and-set. - -Both mechanisms race the same compare-and-set, so whichever reaches a -given row first is the only one that ever settles it or posts a notice -— `postCancelledNotice`, a `turnCancelled` text part distinct from -`postUndeliveredNotice`'s `turnFailed` (a cancellation is not a -failure, and the timeline says so). CL-7230's ceiling is honest, not -silent: settling the row is not the same as stopping the underlying -agent process. A late `connector.reply` that lands anyway finds no -`running` row left to attach to, and `postReply` will not fall back to -posting it unattached onto a 1:1 membership whose latest turn is -`cancelled`. The composer offers a Stop affordance -(`packages/chat-ui/src/composer.tsx`) whenever `isAwaitingReply` -(`streaming-reply.ts`) is true — the whole in-flight phase, including -after tokens have started streaming — independent of its own `sending` -state. A follow-up message can still be typed and queued while a turn -runs. - -## Threads: workbench → thread → sub-thread - -A workbench's timeline is itself a thread — its **root thread**, one per -workbench, created lazily on first use. Any message can be replied to, which -opens (or reuses) a **depth-1 thread** anchored on that message; any message -_inside_ a depth-1 thread can be **forked**, which opens (or reuses) a -**depth-2 sub-thread** anchored on that message. That's the whole model — -workbench → thread → sub-thread, stop. There is no depth 3 (owner ruling, -CL-5908): nesting a reply off a message that already lives in a sub-thread -is rejected with an honest `409 conflict` rather than silently growing a -third level. - -Forking is the first-class affordance CL-5948 adds — "something Slack -doesn't have": any message inside a thread offers **Fork**, spawning a -sub-thread rooted at it. Forking from a message already inside a sub-thread -never creates a third level; it redirects to a **sibling sub-thread** under -that sub-thread's same depth-1 parent instead. Both the redirect and the -409 share one piece of pure logic, `resolveThreadAnchor` in -`packages/chat/src/threads.ts`: given the root thread and the thread a -message currently lives in, it returns where a new thread should hang and -whether that would be a third level. `openReplyThread` (implicit replies) -refuses on that signal; `forkThread` (explicit forks) redirects on it — -neither reimplements depth math. - -**Correlation is mail headers, and nothing else (CL-7104).** Chat is a -mail thread, so every timeline row dispatched to an agent goes out under -its own RFC 5322 `Message-ID` — `<{row id}@{mail domain}>`, derived from -the row's primary key and stamped back onto it — and names its thread -parentage in `In-Reply-To` and `References`, built per RFC 5322 from the -`chat.threads` parent chain (the anchor of the thread it lives in, then -that anchor's own anchor; the root feed answers nothing and carries -neither header). The reply that answers it is matched by those headers. -There is no `Interchange-Correlation-ID` on the wire, no reply-to address -carrying a workbench id, and no "the last thing this address was asked" -heuristic: two turns pending against one agent are told apart because -their dispatches carry different `Message-ID`s, not because one of them -is newer. - -An agent's reply therefore lands in the thread of the message that woke -its turn — the same thread, whether that message lives on the root feed -or inside a sub-thread. The reply path resolves the turn's -`message.run.started` bracket, whose `messageId` IS the dispatched row's -`Message-ID`, back to that row; after a hub restart that dropped the -process-local bracket, the running turn's last `requestMessageIds` names -the same source. Delegation needs no separate mechanism, since a -delegating message has a thread like any other — the hop that wakes a -mentioned specialist carries the delegating row's own `Message-ID`. -Approve blocks and artifact deliveries thread under the turn that -produced them the same way. A reply that names no parent at all is -reported through `reportError` with its room and agent context and -appended to the workbench's root thread — visible, with a `refId` a -person can quote, and never attached to a guessed parent. - -A parked `message_response` gate is the same rule (CL-7104): the -question card is posted with its own `Message-ID`, the gate is armed on -that id, and the answer is dispatched as a reply naming it in -`In-Reply-To`. An answer to a different question names a different id, so -it never clears the wrong gate. - -Every outbound agent frame is also dual-written into each addressed human -participant's `@corbits/mailbox` inbox (CL-7449) — the hub's `persistMail` -lookup is wrapped with `createMailboxPersist` -(`apps/hub/src/mailbox-persist.ts`), so a durable `principal_mail` row lands -for a reply's human recipients alongside the thread's own `session_mail` -record, independent of whether the recipient's session is live. The -`workbench` ref stamped on those rows is resolved header-first: -`@corbits/chat`'s `resolveWorkbenchIdForAgentFrame` maps the frame's own -`In-Reply-To`/`References` back to the timeline row it answers and takes -that row's `workbenchId`, since an agent participating in several -workbenches has no single honest answer from its address alone; only a -header-less frame falls back to a participant scan, and only when that -scan names exactly one workbench — zero or several matches stamp no ref -and report `mailbox_ref_unresolved` instead of guessing. - -Every non-root thread carries a `parentThreadId` — the thread it hangs -directly off (the root thread's id for a depth-1 thread, a depth-1 thread's -id for a depth-2 sub-thread) — alongside its existing `parentMessageId`, -the origin message it answers or forks. `@corbits/chat-ui` reads -`parentThreadId` to render the breadcrumb (`Workbench / Thread / Sub-thread`, -at most three segments), to walk a fork back to its parent thread, and to -indent sub-threads under their parent in the threads menu; a forked -sub-thread also shows a small banner above its timeline linking back to its -origin message — the fork's visible back-reference. - -## Participants and mentions - -A workbench's participants are held in its settings as records of -`{ address, handle }`. The **handle** is the short, unique-within-workbench -name a mention actually types — `@echo`, never the underlying run's -unreadable instance id. Handles are derived from a definition's name at -invite time and de-duplicated against every handle already in the workbench -(`echo`, `echo-2`, `echo-3`, ...). - -**One room participant = one live run (CL-6451).** The `/name` / `@name` -workflow commands resolve residency first -(`findResidentAgentForDefinition`, comparing by the definition's asset so -re-deployed rows still match): a definition already resident reaches the -run the room already has, never a freshly minted sibling that would then -race the original for every message. An `@name` typed as a definition's -wire name (`@assistant`) whose participant answers to a display-name -handle (`@myra`) posts as an ordinary message and rides the normal turn -pipeline into that participant's run, queueing behind an in-flight turn -like any mention. The explicit invite affordance is the one deliberate -way to place a second instance of a definition in a room — it always -launches, which is exactly what handle de-duplication ("echo", "echo-2") -exists for. - -A **mention** is `@` followed by a participant's handle at a word boundary, -anywhere in a message's text. Mentioning an agent participant triggers -**fan-out**: the server sends that agent a single-recipient copy of the -message, addressed from the workbench itself rather than from the posting -principal. Sending from the workbench matters because an agent's reply -router answers the address a message came from — a principal address has -no mailbox to answer into, but the workbench's address is the mailbox every -participant already reads. - -## Chats and direct messages (DMs) - -`kind: "chat"` is a direct thread with exactly one counterpart, fixed at -creation and never changed afterward (`POST /workbenches/:id/invite` 409s a -chat, whichever kind of counterpart it has). The counterpart is chosen at -`POST /workbenches` time, one of: - -- **An agent** — `{ kind: "chat", definitionId }`. The named definition is - launched and joined as the chat's one participant, exactly as - `POST /workbenches/:id/invite` joins one into a workbench (`launchAndJoinAgent` - in `packages/chat/src/workbench-service.ts`, shared by both paths). -- **A person** — `{ kind: "chat", principalId }`. This is a **DM**: a - two-member workbench tenancy whose second participant is an existing bench - member, added directly with no instance to launch - (`joinHumanParticipant`, the human-counterpart analog of - `launchAndJoinAgent`) — a human participant reads the workbench's own - timeline directly, so there is no mailbox to stand up, only the - participant record and a `workbench.member-joined` audit event on the - workbench's own timeline. - -Exactly one of `definitionId`/`principalId` may be present; a `principalId` -is validated before anything is minted — it must name a real, active -`"user"`-kind principal in the calling tenant, and it can never equal the -caller's own principal id (`409 conflict`, "you cannot start a direct chat -with yourself"). Both counterpart kinds are optional on `name`: an agent -chat falls back to the agent's own handle, and a person chat falls back to -the same handle its one participant record carries — in practice always the -member's display name, since `@corbits/chat-ui`'s new-chat dialog already -has it (from the same listing Settings → People renders) and sends it as -`name` whenever the person creating the chat didn't type a custom title. - -**There is no `dm: true` wire flag.** A DM is recognized the same way -everywhere it matters — `kind === "chat"` plus the absence of an -agent-shaped participant address (`isAgentAddress` in -`packages/chat/src/mentions.ts`, which is simply "does this participant's -address contain `@`" — a human participant's address is its bare principal -id). `@corbits/chat-ui`'s workbench-settings surface trims its Agents section the -same way (`workbenchSettingsSections(kind, isDm)` in -`packages/chat-ui/src/workbench-settings/model.ts` — a DM has no agent to -invite, so the section has nothing to show; Members and Danger zone are -already trimmed for every 1:1 chat, agent or person). One derivation, no -second signal to keep in sync. - -## How an invited agent is launched - -Inviting an agent (or opening its DM) calls Interchange -`prepareProvisionedDeployment` onto that definition's own `kind:workflow` -asset. Interchange mints the run row, the allocation, and (on first -trigger) the run principal. Chat records the returned run id, deployment -id, and mail address — it does not mint the run, own an `agent_session`, -or render a per-run `sourceRef`. - -Launch is asynchronous. A message sent before the sidecar is ready sits -on the reconciler's ready hook and the `workflowDispatchService` queue -until the instance can take it; the UI shows a starting state, with -presence from allocation/deployment status rather than a run principal. -Wake of a reaped or lost instance is a fresh `prepareProvisionedDeployment` -onto the same asset; chat updates the ids it recorded. - -Turns are one native `sendUserMessage` per addressed agent, carrying the -row's `Message-ID` / `In-Reply-To` / `References`. Inference sources on -the deployment are catalog offering ids (`sourceOfferingIds` / -`defaultSourceOfferingId`). - -## How a reply reaches a human - -An invited agent's reply is not something chat scrapes off -`connector.reply` events. Sidecar outbound mail hits the hub -`persistMail` lookup; a Workbench wrapper dual-writes that frame into -`@corbits/mailbox` so each human gets a copy. Attachments for humans are -read from that mailbox frame, not from Interchange `session_mail`. -Correlation is the RFC 5322 headers on the mail, never a `correlationId`. - -The old per-agent event-stream **reply bridge** is gone. A process-lifetime -chat orchestrator still watches sidecar events for turn projection and -in-chat approval parks; it is not the path that delivers the agent's -words to a person. - -## Bench defaults and per-workbench overrides - -A workbench setting can be a bench-wide default every workbench inherits, or an -explicit per-workbench override — the same "Use bench default" vs. "Override" -shape Discord's server-default settings use. Today this applies to exactly -one setting, `chat/contextWindow` (how many prior messages a mentioned -agent sees as context): - -- **Bench-wide default** — `GET`/`PATCH /bench/settings` reads and writes - the tenant's own `chat_bench_settings` row. A bench default is never - itself an override of anything, so it is always a plain number, never - `null`. -- **Per-workbench override** — a workbench's own `chat/contextWindow` in its - settings is nullable: `null` (or the key's absence) means "inherit the - bench default," any other integer is an explicit override for that - workbench alone. -- **Resolution** — `resolveContextWindow(workbenchSettings, benchDefault)` in - `packages/chat/src/workbench-settings.ts` folds the two into the one - effective value a message send actually uses, returning both the value - and which source it came from (`"inherit"` or `"override"`). `GET`/`PATCH -/workbenches/:id/settings` include this resolved `{ value, source }` shape - on every response, so a caller never has to re-derive it from the bench - default and the raw workbench settings separately. - -In the UI this resolved shape drives a two-state control — "Use bench -default (N)" vs. an explicit numeric field — on the workbench's own settings -panel (opened from its header, or from its sidebar row's ellipsis menu). -The bench-wide settings page only ever edits the default itself; it carries -no per-workbench editor, since a workbench's override belongs to the workbench. - -## The HTTP surface - -`@corbits/chat` mounts one router, under a tenant-scoped prefix, with the -following routes: - -| Method & path | What it does | -| ------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `POST /workbenches` | Mints the workbench's own tenant, writes its initial settings, and — for a chat — joins its one counterpart (an agent or a person; see [Chats and direct messages](#chats-and-direct-messages-dms)) | -| `GET /workbenches` | Lists the tenant's workbenches, optionally filtered by kind | -| `GET /workbenches/:id/messages` | Reads the workbench's timeline, decoded into parts, paginated by cursor | -| `POST /workbenches/:id/messages` | Posts a message, fanning a copy to every @mentioned agent participant. `threadId` or `inReplyToMessageId` route it into a thread instead of the root feed; a reply that would nest past depth 2 is a `409 conflict` | -| `GET /workbenches/:id/threads` | Lists a workbench's threads (root, delivery, replies, and sub-threads) plus its root thread id | -| `GET /workbenches/:id/threads/:threadId/messages` | Reads one thread's own membership, decoded into parts — never the full workbench mailbox | -| `POST /workbenches/:id/threads/fork` | Forks a sub-thread rooted at any message inside a thread (CL-5948); idempotent per origin message, and redirects to a sibling sub-thread rather than nesting past depth 2 (see [Threads](#threads-workbench--thread--sub-thread)) | -| `POST /workbenches/:id/delivery-threads` | Creates (or reuses) the delivery thread for a routine run | -| `GET /workbenches/:id/invitable` | Lists the tenant's deployed definitions that can be invited into a workbench | -| `POST /workbenches/:id/invite` | Launches a definition into the workbench and adds it as a participant | -| `POST /workbenches/:id/move` | Re-parents a workbench's own tenant to a different bench | -| `GET /workbenches/:id/settings` | Reads a workbench's settings, including its resolved context window | -| `PATCH /workbenches/:id/settings` | Updates settings, recording each change as a timeline event | -| `GET /workbenches/:id/read-state` | Reads the calling principal's last-seen cursor for the workbench | -| `PUT /workbenches/:id/read-state` | Advances the calling principal's last-seen cursor | -| `POST /workbenches/:id/typing` | Publishes an ephemeral typing indicator to the workbench's live stream | -| `POST /workbenches/:id/messages/:messageId/reactions/toggle` | Toggles the calling principal's reaction with a curated emoji on a message; publishes `chat.reaction` | -| `POST /workbenches/:id/messages/:messageId/pin` | Pins a message; publishes `chat.pin` | -| `DELETE /workbenches/:id/messages/:messageId/pin` | Unpins a message; publishes `chat.pin` | -| `GET /workbenches/:id/pins` | Lists a workbench's currently-pinned messages, decoded into parts, newest pin first | -| `GET /workbenches/:id/turns` | Lists the workbench's agent turns, newest first — each carrying the child run id its reply is traceable to (CL-6329) | -| `GET /workbenches/:id/turns/:turnId` | Reads one turn: its child run id, the messages it answered, the message it produced, and how it ended | -| `GET /workbenches/:id/stream` | Server-Sent Events stream of live workbench activity, including the who's-here roster (`chat.presence`/`chat.presence.snapshot`, CL-6328) | -| `POST /workbenches/:id/presence` | Refreshes the calling principal's `lastActiveAt` on the who's-here roster; 404s with no open stream connection (CL-6328) — never polled, called on real client-side activity | -| `GET /bench/settings` | Reads the tenant's bench-wide chat defaults | -| `PATCH /bench/settings` | Updates the tenant's bench-wide chat defaults | - -Every route runs behind the hub's tenant-scoped middleware, so the calling -tenant and principal are always resolved before a handler runs; principals -never appear in a path. - -## Mounting into a host - -`@corbits/chat` never talks to the platform's own HTTP API or reimplements -its session, grant, or mail machinery. Instead it depends on `ChatPlatform`: -a narrow port describing exactly what the package needs — launching a -workbench or an invited agent, dispatching mail to an agent's own mailbox, -fetching an attachment's bytes, and subscribing to live events. The timeline -itself is not on that port: it is a chat-owned table behind -`RoomMessageStore`. -A host composes this port from services it already builds (a session -service, an asset service, a sidecar router, and its database), and injects -it — along with a settings store and a grant check — into -`createChatRoutes` to get a mountable router back: - -```ts -import { - createChatRoutes, - createDrizzleChatStore, - createHubChatPlatform, -} from "@corbits/chat"; - -const chatRoutes = createChatRoutes({ - store: createDrizzleChatStore(db), - platform: createHubChatPlatform({ - db, - sessionService, - assetService, - sidecarRouter, - }), - requireGrant, - turnTimeoutMs: 5 * 60 * 1000, -}); - -app.route("/api/tenants/:tenantId/chat", chatRoutes); -``` - -Any host that can build an equivalent `ChatPlatform` can mount chat the same -way — the port, not this hub, is the integration contract. `turnQueue` -(see [One turn in flight per workbench](#one-turn-in-flight-per-workbench-cl-6331)) -defaults to a fresh, router-scoped queue when omitted, same as -`workbenchSubscribers`; a host that also drives sends through another -surface (a workflow-participant router, a Slack mount) constructs one -`WorkbenchTurnQueue` itself and passes it to every one of them. - -## Consuming it from the UI - -`@corbits/chat-ui` renders the whole chat surface — sidebar, timeline, -composer, mention picker, new-workbench and invite-agent dialogs, and the live -event stream — as a single `ChatWorkspace` component. A host supplies which -bench to talk to and the current user, and mirrors the active workbench into -its own routing. Each sidebar row also carries a hover-revealed ellipsis -menu (Rename, Pin/Unpin, Workbench settings). - -Workbenches are tenants, so their settings are never a dialog: the gear icon -in the workbench header routes to a full stage surface, -`WorkbenchSettingsSurface` (`packages/chat-ui/src/workbench-settings/`) — -a breadcrumb back to the workbench, a left nav grouped Shared / Personal / -Danger zone, and the active section's panel on the right. `ChatWorkspace` -takes `settingsOpen` and `onSettingsOpenChange` the same way it takes -`workbenchId` and `onWorkbenchChange`, so the host mirrors the surface into its -own routing (`@workbench/web` mounts it at `/c/:workbenchId/settings`). The -General section still PATCHes name, pinned, and the inherit/override -context-window control; Members and Agents reuse the same invite flow -already in `invite-agent-dialog.tsx` rather than duplicating it. - -```tsx -import { ChatWorkspace } from "@corbits/chat-ui"; -import { listPrincipals } from "@corbits/settings-ui"; - - navigate(`/chat/${workbenchId}`)} - settingsOpen={settingsOpen} - onSettingsOpenChange={(open) => - navigate(open ? `/chat/${workbenchId}/settings` : `/chat/${workbenchId}`) - } - onOpenArtifact={(part) => navigate("/library")} - listMembers={async (tenantId) => { - const principals = await listPrincipals(tenantId); - return principals - .filter((p) => p.kind === "user" && p.status === "active") - .map((p) => ({ id: p.id, displayName: p.displayName })); - }} -/>; -``` - -`ChatWorkspace` talks to `@corbits/chat`'s HTTP surface directly — a host -does not hand it a client or re-derive its API calls, only tell it where to -send them and who is asking. - -`listMembers` is what puts a People tab beside the new-chat dialog's -existing Agents tab: `@corbits/chat-ui` resolves no session or tenancy of -its own (see the module note atop `chat-workspace.tsx`), so the bench's -people come from the host, the same way `tenant` and `currentUser` do — -`@workbench/web` sources it from `@corbits/settings-ui`'s `listPrincipals`, -the same call Settings → People renders from. Omitted entirely, the dialog -falls back to exactly the agent-only picker it has always been — a host -that hasn't wired a member directory yet never gets a tab that silently -fails to load. - -The timeline adds a day divider between messages from different calendar -days, and renders a `file` part as a clickable artifact chip once it -carries a persisted `blobId` — a still-in-flight, `data`-only attachment -renders the same chip inert, since it has no stable id yet to open. A chip -click calls the host-supplied `onOpenArtifact`, mirroring `onOpenThread` -and `onOpenProfile`: `@corbits/chat-ui` owns no router, and today a chat -blob has no stored link back to a specific Library artifact, so the most a -host can do is navigate to the Library at large — a real per-artifact deep -link (and opening in canvas rather than navigating away) is follow-up work. - -A quiet typing pulse occupies the incoming-message slot after the last -timeline message — the same left indent as the next agent reply. It -lights up from two sources: the `chat.typing` event -`POST /workbenches/:id/typing` already publishes to the live stream (see -the HTTP surface table above), and an owed agent reply that has not -streamed tokens yet. `ChatWorkspace` tracks the latest human ping with a -short expiry and resolves it to the typist's participant handle, never a -raw principal id. The who-is-typing copy is announced to assistive tech; -the visual is a small three-dot bubble. The pulse stays up across tool -rounds (`inference.done` does not wipe an empty pending) and is held for -a short floor so a fast first token cannot flash it. The signed-in -reader's own messages sit on the right, like an outgoing iMessage. - -### The read path: stream events apply, they never trigger a refetch (CL-6328) - -`useWorkbenchFeed` (`packages/chat-ui/src/use-workbench-feed.ts`) holds the -active workbench's messages/threads/pins as three React Query caches, and -`useWorkbenchStream` (`use-workbench-stream.ts`) is the one `/stream` -connection that keeps them current. Every event that changes what those -caches hold applies straight into the cache it describes — -`applyStreamMessage`/`applyStreamReaction`/`applyStreamPin` — rather than -invalidating and refetching: `chat.message` already carries the full -rendered row (see [the message model](#the-message-model)), and -`chat.reaction`/`chat.pin` already carry the full changed row for their own -narrow concern, so a subscriber folds the delta into state it already -holds. `applyStreamMessage` also bumps the owning thread's `replyCount`/ -`lastActivityAt` in the threads cache — the one piece of thread metadata a -message row itself doesn't carry. Every apply is deduped by `id`/`clientId`, -which is what lets a reader's own optimistic send (`use-optimistic-sends.ts`) -and that same send's `chat.message` echo off the stream converge on one row -instead of a refetch reconciling them: the confirmed row is written into -the cache once, from the `POST` response, and the stream's later echo of it -is a no-op. - -`refreshFeed`'s coalesced `invalidateQueries` (CL-6313) still exists, but -only as the fallback poll `useWorkbenchStream` runs while the connection -itself is down or just reopening — never as a response to a live event on -an open connection. The bar this leaves is a hard one: a stream event that -can't be applied is a missing or under-specified payload in -`packages/chat/src/stream-events.ts`, fixed there, never patched over with -a refetch in `chat-ui`. - -The who's-here roster follows the same rule: `chat.presence.snapshot` -seeds it the moment the stream opens and `chat.presence` deltas -(`useWorkbenchPresenceRoster`, `workbench-presence.ts`) keep it current — -no second connection, no polled HTTP heartbeat. "Here at all" comes for -free from the open stream connection itself -(`packages/chat/src/workbench-presence.ts`); the client only calls -`POST /workbenches/:id/presence` to refresh `lastActiveAt` on real -activity (a message send, the tab coming back into view), never on an -interval. - -### Reactions and pinned messages (CL-6030) - -Message reactions and per-workbench message pinning (the open question left -by CL-5942) are both in scope and backed by their own tables — `message_reactions` -and `pinned_messages` — in `@corbits/chat`'s own `chat` schema (see -`packages/chat/src/schema.ts`; distinct from the sidebar's whole-workbench -`chat/pinned` setting, which is unrelated). Both are presence-as-truth: -a reaction row's existence _is_ the reaction (`./reactions.ts`'s -`toggleReaction` inserts on a miss and deletes on a hit — true on/off, never -a counter that can drift), and a pin row's existence _is_ the pin. - -`GET /workbenches/:id/messages` (and `GET /workbenches/:id/threads/:threadId/messages`) -attach `reactions` (a per-emoji `{ emoji, count, reactedByMe }[]`) and -`pinned` (boolean) onto every item — extending the wire type the timeline -already consumed rather than a parallel read. Both batch over the whole -page in one query each (`listReactionsForMessages`, `listPins`), and both -fields are simply absent from the wire when the host never injects the -corresponding store, the same "no store, no feature" contract -`blockResponses` already follows. - -Reactions are restricted server-side to a small curated emoji set -(`REACTION_EMOJI` in `packages/chat/src/reaction-emoji.ts`, shared with -`@corbits/chat-ui`'s picker) — an emoji outside it is a `400`, never -silently accepted. Toggling and pinning both publish onto the workbench's -existing SSE subscriber registry (`chat.reaction`, `chat.pin`), live, the -same workbench `bridgeWorkbenchStream` already bridges typing and settings -events through. - -In the UI, `WorkbenchTimeline` renders a reaction chip row under each message -(click a chip to toggle; an "add reaction" trigger opens the curated -picker) and a pin/unpin toggle, both hover/focus-revealed and keyboard -operable — see `ReactionActions`/`PinActions` in `timeline.tsx`. The -pinned strip the shell mock shows above the message list -(`@corbits/chat-ui`'s `PinnedStrip`) renders every currently-pinned message -as a jump-to chip; clicking one scrolls the timeline to that message's own -row (`messageDomId`). - -### `ask_user` posts and ends the turn, not a park (CL-7443) - -`@corbits/interaction-tools`' `ask_user` tool has no gate: it posts a -`question` block card and returns immediately with a tool result that -tells the model to end its turn — a Workbench agent is an unbounded -interactive step where every inbound mail is already its next turn, so -"ask a person" is native as "post and stop," not a structural suspend. No -correlation id, no timeout, no parked call. The person's answer arrives -later as an ordinary reply through the block-response route below, which -becomes the agent's next inbound message and therefore its next turn — -never this call's own result. (Before CL-7443, `ask_user` parked the call -on a vendored `message_response` signal/gate until a correlated reply -resolved it; that machinery, and the correlation-id plumbing it needed -through `sendMail`/`sendUserMessage`, is retired.) The tool result's -instruction to end the turn is just text in the model's context — the -runtime does not enforce it, so a model that calls another tool or keeps -talking after `ask_user` is not stopped from doing so. - -Because there is no park, a warm agent's persisted pending operations can -still carry a retired kind from before this change (a `message_response` -op written by a pre-CL-7443 build). Restore drops any such unclassified -kind rather than throwing, reporting each drop — so an in-flight question -still open at deploy is simply lost: the person's eventual answer arrives -as an ordinary next turn instead of resolving anything. - -### Question answers notify at most once (CL-7192) - -A question block's answer is relayed into the workbench as the -responding user's own message, and the asking agent is turned on it — -but only once per question, ever. `upsertBlockResponse` -(`packages/chat/src/block-responses.ts`) is a plain "second answer -overwrites the first" write; the one-time send-and-dispatch is gated -separately by `claimBlockResponseNotification`, a guarded UPDATE on the -same row (`notifiedAt`/`notificationClaimToken`) that at most one -submission for a given (tenant, workbench, message, block, principal) -can ever win. A changed answer or a double-click that beats the UI's -disable still lands its own row and its own `block.response` timeline -event, but never wins a second claim — re-answering after the first -notification updates the stored payload without ever notifying the -agent again. - -A claim a submission wins but fails to act on (the send into the -workbench throws) is released so a retried submission can still reach -the agent; `POST /workbenches/:id/messages/:id/blocks/:id/responses` -answers that case with an explicit `notify_failed` 500 telling the -caller their answer was saved and to retry, rather than leaving the row -reading "answered" with the agent silently never turned. The -`block.response` event is always posted — from the upsert's own -returned row, never the request's local payload — before this claim is -even attempted, so an agent dispatched off a won claim always finds its -own correlation event already on the timeline. diff --git a/docs/GLOSSARY.md b/docs/GLOSSARY.md index ddb09ac29b..ad8c6b8873 100644 --- a/docs/GLOSSARY.md +++ b/docs/GLOSSARY.md @@ -2,67 +2,24 @@ Workbench's user-facing vocabulary, and how each term maps to the [Interchange](https://github.com/faremeter/interchange) platform concept -underneath. Product surfaces (UI, CLI output, docs) use the left column; -code and API paths keep the platform's own names — except **Workbench** -itself (CL-6260): that row's own package (`@corbits/chat`) is ours, not -Interchange's, so its code identifiers, wire fields, and route segments -were cut over to match the product word directly, with no separate -lower-level name left to list. +underneath. Product surfaces (UI, docs) use the left column; code and API +paths keep the platform's own names. -| Product term | Platform term | What it is | -| ------------------------ | ------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| **Bench** | tenant | A shared space where a team and its agents work — members, definitions, runs, and grants live here | -| **User** | principal | An identity that can act in a bench — human or agent | -| **Agent** | principal (agent) | A named coworker principal, not a template; opening the row reopens that agent's one DM. The sidebar mixes that DM with channels in one recency list (pins first) | -| **Myra** | — | The `assistant` workflow: every bench's default agent and the coordinator of its other agents. When asked for a capability the bench lacks, she checks for an existing teammate that already fits, then — reusing rather than duplicating — authors a new agent, workflow, or skill as code, deploys it, grants it only the access it needs, and adds it to the bench or a DM | -| **DM** | kind: chat | The one 1:1 tenant with that agent — two opens never clone a second DM | -| **Channel** | kind: workbench | A shared room between people and agents (multi-principal tenant). `+` mints an empty one with nobody hosted; named templates mint the same empty channel, then instantiate their Workbench Definition and run its onboarding walkthrough in the room | -| **Definition** | workflow definition | A deployable unit of agent behavior, authored as code — not a template you mint into many conversations | -| **Workbench Definition** | — | The arktype description a named template instantiates: its default agents, routines, tools, required/optional plugins, and its ordered onboarding walkthrough — see `templates/index.ts`'s `WorkbenchDefinitionSchema` | -| **Run** | workflow run | A definition executing in a bench; interactive runs carry conversations | -| **Routine** | — | Product name for a scheduled workflow: an authored definition whose frozen projection carries a native `ScheduleTrigger`. Cadence, pause/resume, and run-now live on `@corbits/workflows`; the hub poller ticks matching minutes. See [workflow-model.md](workflow-model.md) | -| **Approval** | approval | A human decision gating an external side effect | -| **Grant** | grant | Permission for a principal to act on a resource | -| **Hub** | hub | The API and coordination service a bench lives on | -| **Sidecar** | sidecar | The execution host that runs definitions on behalf of a hub | -| **Extension** | — | A route factory mounted on the hub to add product surface | -| **Workbenches** | — | The product name and the mint verb ("New workbench"), not a sidebar heading. Conversation tenants (agent DMs and channels) share one recency list, pins first — not two labeled sections | -| **Workbench** | — | The one conversation surface: an agent conversation (named by its agent) or a multi-party conversation (named by its own title) — durable hub-side data with no run of its own; also its own tenant, parented under the bench it was created in, so its membership and grants are its own — see [CHAT.md](CHAT.md) and [workbench-tenancy.md](workbench-tenancy.md) | -| **Timeline** | — | A workbench's own message rows, read back in order, as the conversation record. Human copies of agent mail land in `@corbits/mailbox`; the timeline is the room's ordered record, not Interchange `session_mail` | -| **Participant** | — | An address (human or agent) a workbench's settings list as able to post or be mentioned. An invited agent's address is the Interchange run address `prepareProvisionedDeployment` returned | -| **Handle** | — | A participant's short, unique-within-workbench mention name (e.g. `echo`), distinct from its address | -| **Mention** | — | `@` plus a participant's handle in message text, triggering fan-out to that participant | -| **Reply bridge** | — | Retired. An invited agent's reply reaches humans through the hub `persistMail` wrapper into `@corbits/mailbox`; dispatch and reply correlate by RFC 5322 `Message-ID` / `In-Reply-To` / `References`. There is no chat-owned event-stream hop and no `correlationId` | -| **Concept** | — | A kind of work an agent asks for a model by — `cheap-loop`, `code-work`, `image-reader` — resolved against the bench into an ordered, priced chain of the models it can actually reach; agents never name a model, see [inference-concepts.md](inference-concepts.md) | +| Product term | Platform term | What it is | +| ------------- | -------------- | ----------- | +| **Bench** | tenant | The shared space a team and its agents work in — members, credentials, and grants live here. Every signed-up user gets one; there is no separate tenancy layer underneath it. | +| **User** | principal | An identity that can act in a bench — human or agent. | +| **Agent** | principal (agent) | A named coworker principal, not a template. | +| **Myra** | — | The `assistant` workflow: every bench's default agent. Given a job the bench can't already do, she authors a new tool, skill, or workflow as code, deploys it with only the access it needs, and uses it. | +| **Workbench** | — | A conversation — a mail thread, or the group of threads with one agent or one topic. Talking to Myra or a teammate happens here. | +| **Thread** | mailbox thread | The ordered messages of one conversation, served by `@corbits/mailbox` (`GET /me/inbox/threads`, `POST /me/inbox/send`). | +| **Grant** | grant | Permission for a principal to act on a resource. | +| **Approval** | approval | A human decision gating an external side effect. | +| **Hub** | hub | The API and coordination service a bench lives on. | +| **Sidecar** | sidecar | The execution host that runs agent turns and workflow runs for a hub. | -A bench and a workbench are both tenants underneath, which can read as -the same thing twice. They are not: a bench is the scope a team -provisions and works in (the switcher's rows); a workbench is a -conversation that happens to be minted as a tenant too (the platform's -"workbench"), so it can carry its own membership and grants independent of -the bench it lives in. Every workbench is a tenant, but a tenant a person -would call a "bench" is one nothing else is parented under as a -workbench — in practice, the one they signed into, not one that showed up -as a conversation in their sidebar. +## Package naming -A **Routine** is never a second name for a **Run**, or for Interchange's -own workflow concept — the three sit at different levels. A workflow -definition is the deployable code; a run is one execution of it; a routine -is the named, recurring (or manual) parent a person sets up over runs of a -definition, holding the trigger, delivery workbench, and run history a bare -run does not carry. "Workflow" on its own always means Interchange's -runtime concept — the definition or its runs — never a stand-in for -routine. - -Naming conventions for this repository's packages: - -- `@corbits/*` is the default scope for every package under `packages/`, - whether it deploys as a portable artifact (e.g. - `@corbits/-agent`, `@corbits/-workflow`), graduates into its - own repository (e.g. - [`@corbits/react-ui`](https://github.com/corbitsdev/react-ui)), or is - plain local domain code — with a kebab-case kind suffix where the - package is one of a family (`-agent`, `-tool`). -- `@workbench/*` is a legacy scope being migrated to `@corbits/*` package - by package; a handful of packages still carry it (`connections`, - `onboarding`). New packages never use it. +`@corbits/*` is the default scope for every package under `packages/`, +`tools/`, `agents/`, `skills/`, and `workflows/` — with a kebab-case kind +suffix where a package is one of a family (`-agent`, `-tools`). diff --git a/docs/TENANCY.md b/docs/TENANCY.md deleted file mode 100644 index e4892dc341..0000000000 --- a/docs/TENANCY.md +++ /dev/null @@ -1,296 +0,0 @@ -# Tenancy — workbench contracts and Interchange gaps - -Workbench tenancy is **not** a parallel product schema. Membership, -principals, grants, roles, and the tenant hierarchy live in Interchange -(`@intx/db`, `@intx/hub-api`). Workbench adds product contracts on top: -workbench icons, DM workbench shape, invite links, and -parent-validation at creation time. - -This document is the authoritative gap list for anything that still -requires an upstream Interchange change. **Do not patch `vendor/intx`.** - -## What already works (consume, do not reimplement) - -| Capability | Where | -| ------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Tenant `parentId` hierarchy | `@intx/db` tenant table; POST `/api/tenants` accepts `parentId` | -| Live ancestor-chain inheritance | `getAncestorChain` in `@intx/db` — catalog, credentials, providers walk ancestors at read time | -| Descendant walk | `getDescendantTenants` in `@intx/db` | -| Roles | Interchange native `owner` / `admin` / `member` — mirror 1:1 in UI; never invent a parallel role table | -| First-signup genesis / join | Stock composition mounts no first-login provisioning hook, so signup mints no tenants — the first tenant comes from an ordinary `POST /api/tenants` with a chosen slug | -| Memberships | Native principal + membership routes | - -Inheritance is **live**. Creating a sub-workbench must **not** copy -catalog rows, credentials, or providers from the parent — resolution -walks the chain on every read. - -### Root tenant slug (caller-chosen) - -The root tenant slug is a caller choice, not hub env. Stock -composition mints no root at boot and reads no slug key: -`WORKBENCH_DEFAULT_TENANT`, its `ORG_SLUG` alias, and the retired -`OPERATOR_TENANT_ID` are all gone. The first tenant comes from an -ordinary `POST /api/tenants` with a caller-chosen slug (see the -stock-signup test in `apps/hub/test/composition.test.ts`), and its -creator is the native `owner`. - -There is no signup fallback to configure: stock composition carries -no signup gate, so a freshly minted root needs no `access_policy` row -for signup to work. This cutover does not migrate policy rows from a -previous operator tenant. - -### The 0→1 contract (no genesis mint) - -On a hub that starts with **zero tenants and zero users** — the -default after boot, which never mints a root — nobody has to pre-seed -an admin, and the first signup mints nothing either: - -- The sign-up/email route registers the very first account ungated — - stock composition carries no signup mode, so there is no closed - default and no empty-hub exception to grant. -- The caller's composition creates the first tenant through an - ordinary `POST /api/tenants` with a caller-chosen slug; the creator - is its native `owner`. -- Every later signup only registers an account — it never mints a - tenant of its own. Signup never seeds the default workflow set; - seeding belongs to the credential step. - -## Workbench-side contracts (this repo) - -### Signup (ungated in stock composition) - -Stock composition carries no signup gate: `WORKBENCH_SIGNUP`, -`WORKBENCH_ALLOWED_EMAIL_DOMAINS`, and `ALLOW_UNVERIFIED_EMAILS` are -all gone, and nothing gates better-auth's `/sign-up/email` route or -requires `user.emailVerified`. Anyone can register (rate-limited -only — see `SIGNUP_RATE_LIMIT_*` in `.env.example`). - -`@workbench/access-policy` (the closed-by-default signup/sub-workbench -policy table and gate) was deleted rather than kept unwired (CL-8130): -it had no native equivalent and nothing in the hub called it. A -deployment that needs closed signup gates it in the composition that -embeds the hub — there is no Workbench-side policy row or env flag to -configure. - -### Workbench icon - -Product metadata per tenant: monogram (1–2 characters) + color token. -Stored in a workbench-owned table (or tenant metadata when Interchange -exposes a stable product-metadata field). Served to the bench switcher -and avatar badges. See `WorkbenchIcon` in `@corbits/bench-ui`. - -### Sub-workbench creation - -**[Intx gap] CL-6041**: `POST /api/tenants` itself is ungated at the -platform level — any authenticated caller can hit it directly with an -arbitrary `parentId` and become owner of a child under any tenant. -Filed upstream; the workbench-side outer guard -(`apps/hub/src/tenant-create-guard.ts`) was removed in the stock- -composition cutover, so until the platform check lands this gap is -open: every `POST /api/tenants` call, whoever originates it, mints a -tenant with the caller as owner — no signup-gate landing zone, no -membership check on `parentId`. - -`@workbench/access-policy`'s `POST /api/tenants/:tenantId/access-policy/ -child-tenants` — a polished UI-facing wrapper that pre-flighted this -same decision with a clean 403 — was deleted (CL-8130): it was never -mounted, so it never closed this gap either. Native `POST /api/tenants` -grant-gating is the only real gate; until the platform check lands -this gap stays open. - -Interchange currently does **not** validate `parentId` on POST and has -**no** cycle constraint — see gaps below. - -### DM contract - -A DM is a **two-member workbench** tenancy: - -- Exactly two human principals (plus optional workbench-host agent). -- Auto-named from the counterparty display name. -- Flagged `dm: true` on the workbench record (workbench wire field). - -Creation helper: `createDmWorkbenchSpec` in `@corbits/bench-ui`. - -### Shared workbenches (Slack-Connect-style projection) - -**Achievable today without Interchange changes (CL-5882):** - -- A workbench owned by one tenant can be **projected** into a sibling - tenant — no dual native membership required — gated by two - workbench-owned facts, both required, neither implied by the other: - 1. **Bilateral federation trust** between the two tenants: both sides - must hold a `direction: 'bilateral'` row naming the other, in - Interchange's native `federation_trust` table (see - `packages/chat/src/federation-trust.ts`, which reads/writes that - table directly — it never forks - `vendor/intx/hub-api/src/routes/tenant-federation.ts`). A single - one-directional trust row is never enough. - 2. **An explicit per-principal share membership** the projected - tenant's own admin maintains (`workbench_share_member`, see - `packages/chat/src/workbench-share.ts`), fully separate from the - owning tenant's own `chat/participants`. Creating a share never - auto-adds anyone; each side's own admin explicitly adds their own - principals. -- `workbench_share` records that a projection exists; `workbench_share_member` - records who, on the projected side, can actually see it. - `packages/chat/src/routes.ts`'s `resolveWorkbenchAccess` is the one - fail-closed gate every message/read-state/typing/stream/blob/block-response - route resolves through: no share row → not found; a share row but the - caller's principal was never added as a member → also not found — a - third tenant with no share, and a projected tenant's principal nobody - added, are both indistinguishable from "workbench doesn't exist" to the - caller. -- The approval boundary is unchanged: every route still evaluates - `requireGrant` against the ACTING tenant's own grants. A share never - widens what a projected-tenant caller may do — it only widens which - workbench that tenant's own rules apply to. -- `GET /workbenches` now sets a real `sharedLabel` for a workbench projected - into the caller's tenant ("shared via parent · " for true - siblings sharing a parent, "shared · " - otherwise), and a message's sender carries an optional - `tenantId`/`tenantName`/`tenantMonogram` when it was sent by a share - member of the "other side" — closing CL-5913 and CL-5881's tracked - gap: the sidebar's per-row "shared" badge - (`apps/web/src/shell/workbench-list.tsx`) and the timeline's - tenant-monogram badge (`packages/chat-ui/src/timeline.tsx`) both - render from this real signal now, never a guess from participant - addresses. -- `GET /workbenches/:id/stream` enforces `resolveWorkbenchAccess` live, not - only at connect time: `bridgeWorkbenchStream` - (`packages/chat/src/workbench-events.ts`) re-runs the same fail-closed - check before every event it writes, from either the local typing/ - settings registry or the platform's own event stream, and the moment - it returns "no access" it unsubscribes from both sources and closes - the connection. Revoking a `workbench_share_member` row or a - `workbench_share` mid-connection stops that subscriber as of the next - event published on the workbench — live relative to the workbench's own - traffic, not an instant kill on a workbench that goes quiet (a truly - instant kill would need a poll/heartbeat independent of traffic, - which is out of scope here). - -**Explicit scope boundary — what this does NOT do:** - -- Settings (rename/pin/participant edits), invite, move, and thread - routes remain owner-tenant-only. A projected tenant's member can read - and post messages and see live events, but cannot administer the - workbench. Widening that is future work, not silently implied here. -- Revoking bilateral trust does not cascade-delete existing shares — - documented, known follow-up, not fixed here. A share created while - trust existed keeps working after trust is revoked; only new share - creation is gated by trust. -- Sibling benches under the same parent can still additionally be given - ordinary native membership on the workbench's own tenant (the older, - dual-membership path) — that path is unaffected by, and independent - of, the projection machinery above. - -### Tenancy kind (bench vs workbench child) - -A native tenant row carries no `kind`/`type` field, so `/api/me/principals` -returns one row per tenant a principal belongs to — workbenches and -workbench child tenancies alike, indistinguishable to the platform. The -web client needs to treat only real workbenches as selectable benches, so -workbench owns the discriminator: - -- `packages/chat`'s `workbench_tenancy` link table was the source of truth - for "this tenant is a workbench". `WorkbenchTenancyStore.listWorkbenchTenantIds` - answered it in bulk; `POST /api/workbench-tenancies/kinds` (mounted - outside the tenant prefix, alongside `/api/onboarding`) exposed it to - the web client for the caller's own tenant ids. The web/`bench-ui` - client usage of that surface was removed in the thread-native - client-driver cutover (this branch cut the client only; the server - table, store, and route remain — their removal lands separately). -- `@corbits/bench-ui`'s `classifyBenchMembership` used to combine that set with - `isRawIdentifier` (a tenant with no human-assigned name never renders, - regardless of kind) to produce a `TenancyKind`: `"bench"`, - `"workbench"`, or `"unknown"`. `filterBenchMemberships` was what callers - used to keep only real benches. Both helpers and `TenancyKind` were - removed; the client-side replacement is `isBenchMembership` in - `apps/web/src/bench-context.tsx` (named, never raw), with child-tenant - exclusion living in the client-held workbench list - (`apps/web/src/needs-list.ts`'s `childTenantStore`). - -That discriminator was the extension point for every other tenancy kind -the product added (sub-workbenches, DMs, shared workbenches): each was -still a tenant underneath, and stayed distinguishable only by adding a -case to `classifyBenchMembership`, never by inventing a parallel field -on the native tenant row — that discriminator no longer exists -client-side. - -## Interchange gaps (upstream only) - -These are real platform holes. File them against Interchange; do not -fork or shim inside `vendor/intx`. - -1. **No child-listing route** — there is no first-class - `GET /api/tenants/:id/children`. Workbench can walk - `getDescendantTenants` only when it has a DB handle (hub mount), not - from a pure HTTP client. Prefer a hub-api children route. - -2. **No `PATCH parentId`** — reparenting a tenant is not exposed. Sub- - workbench moves require an upstream API. - -3. **Unvalidated `parentId` on `POST /api/tenants`** — the platform - accepts any string (or null). Workbench validates at the product - layer; the platform should reject unknown parents and cycles. - -4. **No cycle constraint** — nothing prevents A→B→A if reparent ever - lands without a check. - -5. **No product-metadata / icon field on tenant** — monogram+color is - workbench-owned until Interchange offers stable tenant presentation - metadata. - -6. **No native workbench-projection primitive** — Interchange itself has - no concept of "project this workbench into another tenant"; workbench - built the whole thing product-side on top of the native - `federation_trust` table (see "Shared workbenches" above, CL-5882). - What's still missing upstream: `federation_trust` has no cascade - from trust revocation to anything workbench layers on top of it (by - design — cascading is a product decision, not a platform one), and - there's still no native notion of workbench membership scoped to a - non-owning tenant, which is why `workbench_share`/`workbench_share_member` - have to be workbench-owned tables rather than native ones. - -7. **Signup is always open at the auth layer** — better-auth - email+password enablement is hub-config, not a platform tenancy - primitive, and stock composition adds no gate on top — signup stays - ungated. - -8. **Invite is membership-side, not token-link** — native invite is - "add principal"; copy-link invite tokens are a workbench product - flow until the platform owns invite URLs. - -9. **No tenant kind field** — `/api/me/principals` cannot say whether a - tenant is a workbench, a workbench's own child tenancy, or anything - else; workbench derives it from `workbench_tenancy` plus name shape - (see "Tenancy kind" above) until the platform exposes one. - -10. **[Intx gap] CL-6041 — `POST /api/tenants` has no grant/policy - check of its own** — any authenticated user may call it directly - with an arbitrary `parentId` and become owner of a freshly-minted - child under any tenant. Workbench's outer guard - (`apps/hub/src/tenant-create-guard.ts`) was removed in the stock- - composition cutover, so this gap is currently open rather than - closed workbench-side (see "Sub-workbench creation" above); the - platform should reject the request unless the caller already holds - a create-child grant on `parentId`. - -## Roles (mirror only) - -| Role | Product meaning | -| -------- | ---------------------------------------------------- | -| `owner` | Full control; create sub-workbenches; manage members | -| `admin` | Manage members and settings; not ownership transfer | -| `member` | Participate; no member admin | - -Never invent `viewer`, `guest`, or parallel role tables. If the product -needs a weaker role, that is an Interchange conversation first. - -## Related packages - -- `@corbits/bench-ui` — `isRawIdentifier` raw-id guard, tenancy contracts (tenancy-kind helpers and the workbench-tenancy client were removed in the thread-native client-driver cutover) -- `@workbench/onboarding` — genesis-or-join first-signup provisioning - (retained, unwired in stock composition) -- `apps/hub` — invite routes, icon routes; one of the - explicitly-listed apps/hub mounts pending extraction into a package (see - [ARCHITECTURE.md](../ARCHITECTURE.md), CL-6127) diff --git a/docs/bench-provisioning.md b/docs/bench-provisioning.md deleted file mode 100644 index c53ec28135..0000000000 --- a/docs/bench-provisioning.md +++ /dev/null @@ -1,98 +0,0 @@ -# Bench provisioning - -Connecting a provider deploys nothing. - -That is the whole rule, and it exists because the alternative was -measured: pasting an Anthropic key on onboarding sat on a static -"Connecting…" for over two minutes, because the connect request -deployed five default workflows — around twenty seconds each — before it -answered. Long enough that the owner concluded the app had frozen. - -## The split - -Connecting a credential has a fast half and a slow half, and only the -fast half is allowed on a request. - -- **Fast half** (`testAndPersistCredential`) — persist the credential, - seed its model catalog. Durable, and measured in seconds. This is all - `POST /api/onboarding/complete` does before it answers. -- **Slow half** (`ensureSeeded` → `seedTenant`) — deploy the bench's - default workflows. This runs in the background, in - `createBenchProvisioner` (`packages/onboarding/src/bench-provisioning.ts`). - -No HTTP route runs the slow half. `POST /complete` and `POST -/complete-setup` both answer with where the bench's agents stand and -hand the deploying to the drain; `GET /api/onboarding/provisioning-status` -reports the same thing for anything that needs to poll. - -`completeCredentialSetup` still composes both halves back to back, but -only for harnesses that need a fully deployed bench before a scenario -can begin (`@workbench/evals`' real target). A route that calls it -re-creates the freeze. - -## Why the drain converges - -The drain is not a job queue. It is a pass over -`onboarding.pending_seed`, where the row _is_ the work item and means -"this bench has a credential and does not yet have its agents". Three -properties follow from that framing rather than having to be built: - -- **Idempotent** — every pass re-reads the bench's real asset and - deployment state before acting, and `seedTenant` underneath is - ensure-then-create at every step. A pass over a finished bench deploys - nothing and clears the row. -- **Convergent** — a pass that gets partway leaves the row in place, so - the next pass picks up exactly the workflows still missing. -- **Restart-safe** — nothing about outstanding work lives in the - process. A hub that dies mid-deploy leaves the row; the next boot's - first tick finishes it. In-memory state is only ever a dedupe guard or - a retry backoff, never a fact the system needs to be correct. - -A failing bench is held off with exponential backoff (15s, doubling to a -10-minute ceiling) so a sidecar outage is not hammered, and a bench is -never dropped for failing — the row stays until it converges or its TTL -(24 hours) expires. That backoff bookkeeping is reclaimed once the row -itself is gone — TTL-expiry or otherwise — not only when the bench -converges, so a permanently-failing bench never leaves a stale backoff -behind for a later, unrelated connect to the same user/tenant to -inherit (CL-7233). - -## Sessions - -A background loop has no request to borrow cookies from, and everything -`seedTenant` drives speaks the hub's own HTTP API. The provisioner takes -`sessionFor` as a seam for the composition root to fill by minting the -bench owner's own session in process. The hub currently has no filler -wired for it. - -It has to be the owner's session. The hub resolves a tenant by looking -up a principal for that user, and rights do not flow from a parent org -down to a child bench — an administrator acting on someone else's -personal bench is refused outright. - -## What someone sees - -Connecting hands the person forward immediately, and the wait on the -other side is as short as it can honestly be: the setup agent (Myra) -leads `DEFAULT_WORKFLOWS`, so she is deployed before anything else, and -the land hop drops the person into her room the moment she can answer. -The other catalog workflows (digest, recurring-task, research) are not -seeded at all — they are deployable through the catalog instantiate route -(CL-7073) once someone picks one, not on a critical path nobody is waiting -on. - -While she is still coming up, the shared warm loader is the whole -screen: one honest headline and a rotating tip, never an internal count. -How many workflows a bench seeds is an implementation detail, and "0 of -5 ready" told a waiting person nothing they could act on. A wait that -runs past a sensible interval says so plainly and offers another go, -rather than showing an unchanging number. - -Once the person has landed, anything still deploying is at most a single -dismissible line — never a screen that blocks them. - -Whether a bench can be started is asked of the bench, never read off an -error message: `GET /api/onboarding/provisioning-status` answers with -`setupAgentReady`, which is the only field a waiting surface should -branch on. `deployed`/`pending` stay in the response for operators and -logs. diff --git a/docs/command-palette.md b/docs/command-palette.md deleted file mode 100644 index f7adfff0d1..0000000000 --- a/docs/command-palette.md +++ /dev/null @@ -1,97 +0,0 @@ -# Command palette - -Cmd/Ctrl-K opens a global search-and-jump overlay: pages the app shell -already renders, plus workbenches, agents, workflow runs, routines, skills, -plugins, and Library artifacts, ranked and grouped, with full keyboard -navigation. - -## How it's invoked - -This palette is a separate surface from the stage top bar's per-page filter -magnifier (DESIGN.md → Search, `docs/DECISIONS.md` → Search) — the two are -never merged, and neither opens the other. The palette has exactly two doors: -Cmd/Ctrl-K anywhere, and a context-menu item. Both call -`openCommandPalette()` from `command-palette-open-store.ts`, an external -store rather than component state because those doors and -`CommandPaletteProvider` (which renders the palette itself, mounted once in -`app.tsx`'s Shell above `AppShell`) are siblings, not ancestor/descendant. -That state outlives a remount, so it is scoped explicitly — the provider -closes search on a route change (a Back out of a result never leaves the -overlay standing) and on a bench switch. cmd+K opens and does not toggle: -react-ui's shortcut yields to text fields, and an open palette holds focus in -its own input, so Escape and the overlay are the ways back out. - -The palette renders as react-ui's `CommandPalette`, a centered modal dialog — -its own surface, independent of any page's stage top bar, which is exactly -what makes it reachable from a route that renders no stage top bar of its -own (an unmatched route included). - -## Where it lives - -The overlay itself — the dialog, the keyboard contract (arrows, Enter, -Escape), grouped result rendering, and `groups`/`onQueryChange`/`loading`/ -`error`/`hasMore`/`onLoadMore` as first-class props — is `CommandPalette` in -[corbitsdev/react-ui](https://github.com/corbitsdev/react-ui). It knows -nothing about workbenches, routines, artifacts, or agents; it renders whatever -grouped items a consumer hands it and calls back on selection and on query -change. That is a deliberate boundary: a reusable component cannot carry one -product's vocabulary. - -What workbench actually shows — which pages exist, and which workbenches, -agents, and workflow runs match what someone typed — is -`@corbits/command-palette` (`packages/command-palette`), a UI-free package -with three responsibilities: - -- `buildStaticCommands` turns the app shell's own route table into palette - commands. It never invents a destination — a route only becomes a command - because `apps/web/src/routes.tsx` already renders it. -- `useEntitySearch` (backed by `searchEntities`) debounces a typed query - across a set of per-category `fetch` sources and pages the combined - results, grouped by category. -- `buildCommandPaletteGroups` assembles the final `groups` handed to - react-ui's palette: recents, `#`/`@`/`>`/`/` scope parsing for a bare-scope - view, and the source ordering the mock defines. - -`apps/web/src/command-palette-provider.tsx` is composition only: it wires -`useEntitySearch` to live fetchers for workbenches (`@corbits/chat-ui`'s -`listWorkbenches`), agents (`listAgentDefinitions`), and workflow runs -(`/api/me/workflows/runs`), fetches routines, skills, and Library artifacts -as small per-bench catalogs (filtered client-side, the same way the static -route list already is), lists the bench's connected MCP servers as Plugins, -builds `@corbits/command-palette`'s static commands -from `apps/web/src/routes.tsx`, and renders the assembled groups through -react-ui's data-driven `CommandPalette` itself. No new endpoint beyond the -ones the Routines, Skills, and Library pages already use, no domain logic in -the app. - -## Groups, in display order - -Commands, Workbenches, Pages, Runs, Routines, Skills, Plugins, Files, then -People & agents — the mock's `buildCmdkEntries` ordering, with Plugins (this -bench's connected MCP servers) added where the gallery sits in the rail. -Selecting a workbench, run, agent, routine, skill, or plugin result navigates -to its real route and records a recent entry (`recentsStoreForBench`, -per-bench, local to the browser); selecting a Library result opens the -Library list, since a Library item has no dedicated route of its own yet. - -Agents, skills, and plugins resolve to the slug-addressed detail routes -DESIGN.md's Detail Pages section defines (`/agents/`), through -`detailPath`, which is handed the entity's _own_ minted slug — an agent's -handle, a skill's name, an MCP server's slug. A slug is never derived from a -display title: slugs are immutable and titles are not, so a guess 404s the -moment the two disagree. An entity whose slug is not a slug falls back to -its opaque id, which every roster still resolves as a deep link. Routines and Library results still open their -roster deep links, which carry real content the slug placeholders do not -yet; moving them over belongs with those detail pages. - -## What is not wired yet - -**No scope-chip badge or footer prefix legend.** The mock's input shows a -small badge for the active `#`/`@`/`>`/`/` scope and a footer legend naming -each prefix; react-ui's `CommandPalette` has no slot for either today. The -placeholder text (`Search or jump to… (# workbenches · @ people · > actions · -/ pages)`) carries that information as plain text instead. - -**Plugins covers connected servers only.** The gallery's presets and catalog -entries are not connected things and have no detail route yet, so they are -not searchable — a follow-up on this group, never a second search surface. diff --git a/docs/connect-cards.md b/docs/connect-cards.md deleted file mode 100644 index 533acdbf60..0000000000 --- a/docs/connect-cards.md +++ /dev/null @@ -1,88 +0,0 @@ -# In-room connect cards - -The 0-1 flow (CL-6393): when a request needs a service the workbench -hasn't connected, the agent's reply stays helpful on its own and a -connect card appears in the room. Connecting flips the card and resumes -the agent — no settings page round-trip, no "report back when done". - -## The pieces - -- **Block** — `connect-service` (`packages/chat/src/blocks.ts`, - `ConnectServiceBlockData`). Carries agent-authored framing only: - `connectorId`, `displayName`, and a consumer-language `reason` - ("Connect Gmail so I can send this for you."). It never carries an - auth mode or a connected verdict — those would let an agent spoof - live state next to a live button. -- **Card** — `ConnectServiceBlockView` / - `ConnectServiceBlockContainer` (`packages/chat-ui/src/blocks/`). - The host-supplied `ConnectServiceActions` port resolves the live - state and the connect affordance: one click for hosted OAuth and - keyless MCP presets, an inline key-paste for api-key connectors. - `apps/web/src/connect-service-actions.ts` is the workbench binding. -- **Emission** — `request_connection` - (`tools/connections/src/tool.ts`) posts the block into the - caller's own room through the same `participants/messages` route - `ask_user` uses, and tells the model to keep helping in the meantime - (draft now, finish once connected). A run with no room of its own - falls back to a plain deep link. -- **Pending ledger** — posting the block registers the connector under - the room settings key `connections/pending` - (`packages/chat/src/connect-pending.ts`, applied by - `workflow-participant-routes.ts`), so a later connect can find every - room that is waiting. -- **Settling** — every connect surface in `@corbits/connections` - (OAuth callback, pasted key, MCP OAuth, keyless MCP preset) fires the - optional `onConnected` hook (`src/connected-hook.ts`) once the - credential is durably stored. The hub wires it to - `settleConnectedService` (`packages/chat`): the pending entry clears, - `chat.settings` publishes so the open card flips, and a - `connection.connected` event is posted — never as the connecting - person. A room waiting only under `template/pendingConnections` does - not `dispatchTurn`. A room whose own `connections/pending` named the - connector still wakes the asking agent via `dispatchTurn` / - `sendMail`. - -## GitHub (Code review) - -GitHub for Code review is **PAT-first** today (`connect-github` block + -`packages/chat-ui` card, CL-6345). The card is posted by -`POST /workbenches/:id/onboarding` (`packages/chat/src/routes.ts`) from -a `system@` sender when the Code review Workbench -Definition's onboarding steps ask to connect GitHub — never as a side -effect of hosting an agent, so an empty, hostless room can still run -the walkthrough. Connect opens a guided personal-access-token paste -(create a token with the `repo` scope, paste it, store encrypted). -After Connect succeeds — or when GitHub is already connected — the -same in-room card flips in place to pick repositories; there is no -`/new` already-connected dialog. Code review needs a repo pick before -reviewers are watching, then Start reviewing. Settling a credential a -template room is waiting on posts the connected notice from the system -address and never wakes an agent (`packages/chat/src/connect-pending.ts`) -— a reviewer roster has no host to answer. Once reviewing starts, each -reviewer posts its own canned introduction under its own address, in -roster order (`packages/code-review/src/introductions.ts`). - -A GitHub App / hosted OAuth Connect as the welcome mat is CL-6343, out of -scope for the shipped card — do not document OAuth-first GitHub connect -as current. A stale Connect after success, or an agent 401 after GitHub -already connected, is still a broken first minute (PRODUCT.md). - -The generic `connect-service` card (above) still offers one-click hosted -OAuth for connectors that are actually OAuth-only (e.g. Gmail); GitHub's -own template card does not ride that path. - -## Gmail - -`gmail` is a pure hosted-OAuth connector -(`packages/connections/src/gmail-connect.ts` + its descriptor in -`templates/connectors.ts`): -Google's code flow with PKCE, the `gmail.modify` scope, and -`access_type=offline` so the exchange captures a refresh token — stored -as the credential row's `refreshSecret` next to the one-hour access -token. Configure `GMAIL_CLIENT_ID` / `GMAIL_CLIENT_SECRET` (a separate -OAuth client from the sign-in `GOOGLE_CLIENT_ID`); see `.env.example` -for the exact-match redirect-URI requirement and the Google app -verification lead time. Unconfigured deployments render the card as -not-configured — never a dead key-paste form. Automatic refresh of -registry-connector tokens and Gmail read/draft/send tools are not built -yet; the connector stores proven material for both. diff --git a/docs/credential-wiring.md b/docs/credential-wiring.md deleted file mode 100644 index 6cda553ebf..0000000000 --- a/docs/credential-wiring.md +++ /dev/null @@ -1,117 +0,0 @@ -# Credential wiring: from connect to tool call (CL-6032) - -How a tenant's connected credential (Settings · Connections) reaches the -tool call a workflow step makes, and the `CredentialProvider` plugins -that shape it. - -## The chain - -1. A tool package (e.g. `@corbits/granola-tools`) declares a credential - handle in its `package.json`'s `interchange.credentials` (CL-6028). -2. A workflow definition binds that handle via `credentialBindings`. -3. At launch, `buildCredentialDelivery` (`vendor/intx/db/src/ -credential-resolution.ts`) resolves the binding against a tenant-owned - credential, decrypts its secret once, and stamps a `credential:{id}` / - `use` grant scoped to the tool package's consumer identity - (`toolConsumer(packageName)`). -4. The resolved `CredentialDelivery` and the run's grants ride the deploy - frame and every `credentials-updated` control frame into the - workflow-process child (`vendor/intx/workflow-host/src/child/ -run-child.ts`), which threads them to every step invocation as - `CredentialWiring` -- `ChildStepInvoker`'s 6th argument. -5. The sidecar's `invokeStep` binding - (`apps/sidecar/src/workflow-substrate-factory/index.ts`) threads that - wiring into the per-step env (`step-env.ts`'s `attachStepCredentials`). -6. `createToolBearingAgentFactory` (`apps/sidecar/src/ -step-agent-tools.ts`) derives each tool factory's consumer identity - from its own declared `id`, and shapes a consumer-scoped `credentials` - capability via the platform's `createCredentialCapability` - (`@intx/harness/src/credential-capability.ts`) -- the same - fail-closed, per-`{tool}`-condition gate the launch-time grant sets up. -7. The tool bundle calls `env.credentials.resolve(handle)` and gets a - mediated `fetch`, never the raw secret. An unbound handle or a denied - grant makes `resolve()` throw; the tool catches it and reports "not - connected" rather than failing the step. - -## Provider plugins - -A credential's `provider` row names a `plugin` -- the `CredentialProvider` -key its bindings shape through. The sidecar's `CredentialProviderRegistry` -registers: - -- **`http`** -- `@intx/harness`'s vendored `createHttpCredentialProvider`. - Sends `authorization: Bearer `. The default for any bearer-token - API (Granola's, for example). -- **`http-raw-authorization`** -- `@corbits/credential-providers`'s - `createHttpRawAuthorizationCredentialProvider`. Sends the secret - verbatim in `authorization`, no `Bearer ` prefix. Linear's API expects - this raw-key convention, not a bearer token, so a Linear provider row - MUST set `plugin: "http-raw-authorization"` -- seeding it as `"http"` - sends the wrong header shape and Linear rejects the call. -- **`http-x-api-key`** -- `@corbits/credential-providers`'s - `createHttpXApiKeyCredentialProvider`. Sends the secret in `x-api-key`. - Exa and ScrapeCreators expect this header, not `authorization`. -- **`http-x-manus-api-key`** -- `@corbits/credential-providers`'s - `createHttpXManusApiKeyCredentialProvider`. Sends the secret in - `x-manus-api-key`. A Manus provider row MUST set this plugin -- seeding - it as `"http"` or `"http-x-api-key"` sends the wrong header name and - Manus rejects the call. -- **`mcp-streamable-http`** -- `@corbits/credential-providers`'s - `createMcpStreamableHttpCredentialProvider`. For a token-bearing MCP - server it sends `authorization: Bearer `; for a keyless one - (`MCP_NO_TOKEN_SENTINEL`) it sends no authorization header. The vendored - `http` plugin cannot express the keyless case. - -All five plugins the sidecar registers mirror the same origin-pinning and -`redirect: "manual"` protections; only the injected header (or its -absence, for keyless MCP) differs. - -`templates/connectors.ts` (the concrete connector set a build passes into -`@corbits/connections`) is the other place this plugin id needs to be -correct: a Linear connector descriptor there must set -`plugin: "http-raw-authorization"`, and a Manus one must set -`plugin: "http-x-manus-api-key"`, matching this doc, not the `"http"` -default other connectors use. - -## Trust boundary - -`createToolBearingAgentFactory` derives a tool factory's consumer -identity from the factory's own self-declared `id` -(`defineTool({ id, ... })`) -- there is no loader-side check binding that -`id` to the package the factory actually shipped in. This is acceptable -only because tool packages are operator-installed, root-bucket-trusted -code (AGENTS.md: root-bucket modules "may declare routes, migrations, -credentials, and grants"; sandboxed installables never reach this path). -It is not a boundary that would hold against an untrusted or sandboxed -tool package. See the comment on `packageFromToolId` -(`apps/sidecar/src/step-agent-tools.ts`) for the same note in code; the -loader-side provenance check this implies for a future untrusted-package -story is tracked as its own follow-up, not solved here. - -## Rotation reaches live agents (CL-6687) - -Inference sources -- the decrypted provider key included -- are resolved -at deploy time and rendered into the run's bytes; nothing re-reads them -while the run is resident. So a rotated key (Settings > AI providers, -disconnect + reconnect after a 401) used to reach only the _next_ deploy, -and an already-open workbench kept sending the dead key until the stack -restarted. - -Every deploy now records `inferenceSourcesDigest` (a SHA-256 over the -resolved chain, secret included; the secret itself is never stored) on -the run's `workbench_launch` row. Two paths compare it against today's -resolution and relaunch the run on a mismatch, the same relaunch a -definition-content drift triggers (CL-6588): - -- `reconcileDriftedRun`, ahead of every send, so the next message in an - open room uses the new key even if nothing else fired. -- `reconcileInferenceSources(tenantId)`, kicked by the hub the moment an - inference provider's `/complete` stores a credential, so the fix an - operator just applied lands without waiting for a message. - -Re-saving the same key produces the same digest and relaunches nothing. -A run whose row predates the column (`sources_digest IS NULL`) gets -today's chain recorded as its baseline on its first check and is left -alone that once. The per-send check is throttled to one resolution per -participant per 30 seconds; the provider-connect hook is the path that -reaches live rooms immediately. diff --git a/docs/folded-mail-vs-intx-mailbox.md b/docs/folded-mail-vs-intx-mailbox.md deleted file mode 100644 index cce70ef8f3..0000000000 --- a/docs/folded-mail-vs-intx-mailbox.md +++ /dev/null @@ -1,79 +0,0 @@ -# `folded-runs/src/mail.ts` vs `@intx/mailbox` - -Analysis for CL-7276. Read against the vendored pin `a8bc06ae`. - -## The claim under test - -CL-7276 was filed on the claim that `folded-runs`' mail layer duplicates published -`@intx/mailbox` surface — noting that `executeSearch` and `executeThread` have zero -callers in this repo — and that `listFoldedMail`'s hand-rolled keyset pagination -should be routed through the native package. - -**That claim is wrong, and the zero-caller signal was misleading.** The two operate -on different storage planes. There is nothing to route. - -## The two planes - -| | `folded-runs/src/mail.ts` | `@intx/mailbox` | -| ------------------- | ----------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------- | -| Backing store | `session_mail`, a **Postgres** table (`@intx/db/schema/messages.ts:71`) | UID/MODSEQ message store, IMAP-shaped | -| Backings that exist | — | `createInMemoryMailboxStore`; `createSubstrateMailboxStore` (`@intx/workflow-host`), writing into the workflow-run **git repo** | -| Postgres dependency | drizzle + `@intx/db` | **none** — deps are `@intx/crypto`, `@intx/mime`, `@intx/types`, `arktype` | -| Plane | Hub | Sidecar / run substrate | -| Addressing | `(sessionId, createdAt, id)` keyset | `uid`, `modseq`, `uidValidity` | - -`@intx/mailbox` has no Postgres backing and no way to acquire one without a new -adapter. `executeSearch`/`executeThread` take a `MailboxStore`, which -`session_mail` is not and cannot cheaply become — `MailboxStore` is a synchronous -in-memory interface exposing `messages`, `uidNext` and `highestModSeq` as readonly -properties. - -So the hand-rolled keyset walk in `listFoldedMail` is not a reimplementation of -`executeSearch`. It is a Postgres query against a Postgres table, and the native -functions could not serve it. - -## Why the zero-caller signal was misleading - -`executeSearch` and `executeThread` having no callers in this repo is real, but it -means the _sidecar-side_ mailbox surface is unused here — not that we reimplemented -it. Those functions are consumed inside `@intx/workflow-host`'s substrate mailbox, -which the vendored sidecar runs. The hub never holds a `MailboxStore`. - -This is worth recording as a caution for the parent audit (CL-7257): "native export -with zero callers" is a lead, not a finding. Two of the five sub-issues filed off -that signal have now been disproved by reading the backing store. - -## What the send path actually shares - -`sendFoldedMail` already composes native primitives rather than reimplementing -them — `@intx/mime`'s `parseMailToEmail` here, and `assembleMessage` / -`assembleSignedContent` / `createDetachedSignatureFromProvider` in the sibling -delivery paths. MIME assembly and signing are not duplicated. - -The retry wrapper `sendFoldedMailWithRetry` has no native counterpart and should -stay. Its hardening is load-bearing: see the module header of -`packages/webhook-triggers/src/launch.ts` on why a delivery-failed mail must not -throw past an already-202'd webhook route, or a sender retry mints a second run for -one event. - -## `POST /workflows/:runId/mail` - -Native, and real (`vendor/intx/hub-api/src/routes/workflows.ts:846`), but it is an -HTTP route requiring a `workflow-run:` `manage` grant. The hub calling its own -route in-process to write a row it already owns would add an authz round-trip and a -serialization hop to a direct write. Not recommended. - -## Conclusion - -No change recommended. CL-7276 should be closed as "different storage planes". - -The genuine version of this work is **CL-7103** (adopt Interchange per-run mailboxes -and threaded mail in workbench chat). That is a real migration — moving hub chat -onto the run-substrate mailbox — with a product decision and a data migration -attached. It is not a deduplication, and this ticket should not be confused with it. - -## Not verified - -- Whether `session_mail` could be exposed _as_ a `MailboxStore` adapter cheaply - enough to be worth it under CL-7103. The interface's synchronous readonly - `messages` array suggests not, but that is CL-7103's question to answer. diff --git a/docs/images/composer-after.jpg b/docs/images/composer-after.jpg deleted file mode 100644 index 6b535f49fc..0000000000 Binary files a/docs/images/composer-after.jpg and /dev/null differ diff --git a/docs/images/composer-before.png b/docs/images/composer-before.png deleted file mode 100644 index 22cec3146f..0000000000 Binary files a/docs/images/composer-before.png and /dev/null differ diff --git a/docs/inference-concepts.md b/docs/inference-concepts.md deleted file mode 100644 index c369765847..0000000000 --- a/docs/inference-concepts.md +++ /dev/null @@ -1,110 +0,0 @@ -# Choosing models by concept - -An agent asks for a model by what the work needs. It never names one. - -A model named from memory is a guess about a bench the agent cannot see — -the bench may not have that model, may not be able to reach it, or may not -be willing to pay for it. A **concept** names a kind of work instead, which -is a question this bench can answer from its own connected providers, its -own capability data, and its own prices. - -The answer is always a **chain**: the models this bench can reach for that -work, cheapest first, with fallbacks behind the head. Never a single model — -one model is not a fallback plan. - -## The vocabulary - -Ceilings are USD per million tokens, input and output separate. The mix is -the reference workload that defines "cheapest" for that concept: an -input-heavy concept ranks a model that is cheap on input first. - -| id | when to use | requires | ceiling in / out | mix in / out | -| -------------------- | ------------------------------------------------------------- | --------------------------------------------------------------- | ---------------- | ------------ | -| `cheap-loop` | A step that runs hundreds of times: classify, tag, route. | `plain-text` | 0.30 / 1.00 | 1 / 0.05 | -| `bulk-extraction` | Pull fields out of a pile of documents into a fixed shape. | `plain-text`, `structured-output` | 0.60 / 2.50 | 4 / 0.10 | -| `long-document` | Read something very large end to end and answer about it. | `plain-text` | 3.00 / 12.00 | 10 / 0.25 | -| `judgment-heavy` | Hard calls where being right matters more than being cheap. | `plain-text`, `reasoning-content` | 6.00 / 30.00 | 1 / 1 | -| `everyday-assistant` | Live chat with a person, using tools as it goes. | `plain-text-streaming`, `function-calling-multi-turn-streaming` | 3.00 / 15.00 | 2 / 0.5 | -| `agentic-workhorse` | Multi-step tool work run unattended to completion. | `function-calling-multi-turn`, `structured-output` | 3.00 / 15.00 | 3 / 1 | -| `deep-tool-reasoner` | Long autonomous work where a wrong tool call is expensive. | `function-calling-with-thinking`, `reasoning-content` | 8.00 / 40.00 | 3 / 1 | -| `code-work` | Read, write and run code against a repository. | `function-calling-multi-turn`, `structured-output` | 4.00 / 20.00 | 5 / 1 | -| `image-reader` | Screenshots, scans, diagrams — describe or extract from them. | `vision-input`, `plain-text` | 3.00 / 12.00 | 2 / 0.25 | -| `image-maker` | Produce a picture rather than words. | `image-output` | 2.00 / 10.00 | 1 / 0.5 | -| `voice-in` | Take spoken input — a recording or a live call. | `audio-input` | 2.00 / 10.00 | 2 / 0.5 | -| `fresh-facts` | Answers that must reflect the world right now, with sources. | `grounding`, `plain-text` | 4.00 / 16.00 | 2 / 0.5 | - -`code-work` prefers `code-execution` rather than requiring it: requiring it -would exclude most relay deployments, which can still read and write code -perfectly well. - -`long-document` requires only `plain-text` because the capability that -expressed "handles a huge input" — `long-context` — is baked onto catalog -offerings but is not in `@intx/types`' storable capability vocabulary, so -nothing can filter on it. Its 10M-token reference mix carries that meaning -instead: it ranks the model that is cheap on enormous inputs first, which is -the decision the capability was standing in for. When the vocabularies -converge, `long-context` moves into `requires` and this paragraph goes away. - -## How a chain is resolved - -1. **The need.** A concept id, or a bare capability set. An unknown concept - is an error naming the real ones — never a silent fall back to "anything". -2. **What this bench can reach.** The tenant-visible offerings (inheritance, - shadowing and disable-cascade already applied by the platform) whose - provider has a credential. -3. **Capability filter.** The offering must advertise every capability the - concept requires — the same predicate the platform's own source - resolution uses, so a chain resolved here is never rejected downstream. -4. **Bench policy.** Deny first, then allow (which applies only when - non-empty). Entries match an exact model name, `provider:`, or - `/`. -5. **Price.** The active `model_pricing` row at the resolving instant, - normalized to USD per million tokens. A missing or partial row is - unknown, never zero. -6. **Ceilings.** Soft by default: an over-ceiling model is flagged and sorted - last, not dropped, so a bench whose only provider is expensive still gets - an answer. A bench can set its own ceiling hard, which excludes instead. -7. **Order.** Within each bucket — priced-and-within, priced-and-over, - unpriced — cheapest first for the concept's mix, then catalog priority, - then model name, provider name, and offering id, the same tiebreakers the - settings UI and the workbench-creation default already use. -8. **Provider preference, then diversify.** A bench that pins providers keeps - only those; a bench that prefers them fronts them. If every entry that - survives shares one provider and another provider qualifies, the last slot - goes to it — a one-provider chain is not a fallback chain. - -An empty chain comes back empty, with the reason each candidate was -excluded. Nothing is substituted in to avoid answering "nothing here can do -that". - -## What a bench can set - -`inference_catalog.bench_model_policy`, one row per bench, all of it -optional: - -- `allow` / `deny` — selector lists. -- `maxInputUsdPerMTok` / `maxOutputUsdPerMTok` — the bench-wide ceiling. -- `ceilingIsHard` — whether that ceiling excludes or merely demotes. -- `conceptCeilings` — raise or lower one concept's ceiling without forking - the vocabulary. -- `providerPreference` — `pin` or `prefer`, the platform's own shape. - -A bench with no row is unconstrained. That is deliberate: a freshly -connected bench with one provider gets a working chain with no configuration -at all. - -## The tools - -`@corbits/catalog-tools` publishes three read-only tools onto the agent: -`list_model_concepts`, `pick_models`, and `estimate_run_cost`. None of them -accepts a model name. They reach `@corbits/inference-catalog`'s -run-authenticated surface at `/api/workflow-inference-catalog`. - -## Where capability data comes from - -Every offering workbench seeds now carries the capabilities the pinned -catalog observed for that deployment — see -[model-seeding.md](model-seeding.md). A deployment the catalog has never -probed (a local Ollama model, an open-weight relay) is seeded with an empty -list and is therefore invisible to concept resolution, which is the honest -state: a guessed capability routes real work to a model that cannot do it. diff --git a/docs/local-dev.md b/docs/local-dev.md index 62a3065feb..995d7c7f64 100644 --- a/docs/local-dev.md +++ b/docs/local-dev.md @@ -1,160 +1,50 @@ # Running fully local (Ollama) Workbench can run with no cloud LLM key at all, using a local -[Ollama](https://ollama.com) instance as the inference provider. - -Set `OLLAMA_BASE_URL` in `.env` to the origin your Ollama instance listens -on (e.g. `http://localhost:11434`, or a tailscale-tunneled origin) — -`bun run dev`'s memory setup still reads it (it mounts the `@corbits/memory` -plane against the same origin when `EMBED_BASE_URL` is unset). The hub -itself no longer reads provider env vars at all: connect the Ollama -provider in the UI instead — its onboarding card is the one that asks for -a base URL instead of a token, and it needs no key. +[Ollama](https://ollama.com) instance as the inference provider. Connect +the Ollama provider from the UI — its connect card asks for a base URL +instead of a token and needs no key. Set `OLLAMA_BASE_URL` in `.env` too +(e.g. `http://localhost:11434`, or a tailscale-tunneled origin): `bun run +dev` reads it to also mount the memory plane against the same origin when +`EMBED_BASE_URL` is unset. Tool-heavy turns (anything that calls `mcp_list_tools`, dispatches a task, -or chains several tool calls) take noticeably longer — minutes, not -seconds — on small local models. This is a model-capability limit, not a -platform bug; expect it when testing against a small Ollama model. +or chains several tool calls) take noticeably longer on small local +models. That is a model-capability limit, not a platform bug. ## Migrations after pulling `bun run dev` applies both the platform's own migrations and every -installed package's migrations at startup (`scripts/db-setup.ts`), reporting -what it applied. After pulling changes that add a migration, just restart: +installed package's migrations at startup, reporting what it applied. +After pulling changes that add a migration, just restart: ```sh bun run dev ``` -No separate migrate command is needed — `dev` is safe to re-run and only -applies what hasn't already run. - -## Republishing a tool package - -A workflow that pins a `@corbits/*` tool package (e.g. **assistant** pinning -`@corbits/memory-tools`) resolves that pin from a `package-registry` asset -(`CORBITS_TOOLS_REGISTRY`) carrying the package's tarball, built by -`@corbits/tool-registry-publish`. Hub boot does not publish that registry -(or any other product state). `publishCorbitsToolsRegistry` packs onto a -tenant when onboarding or an explicit `@corbits/seeding` caller asks; -descendants inherit it, and `seedTenant` does not pack. After changing a -tool package's source, bump its version, then publish onto the tenant that -owns the registry — restarting the hub does not republish. The operator -path is `bun run publish-tools` (with `HUB_ADMIN_EMAIL`/`HUB_ADMIN_PASSWORD` -set, and `--tenant ` when the admin belongs to more than one -tenant): it signs in, resolves the target tenant, and installs the -registry onto that already-existing tenant over the hub's native asset -routes. Resolution and the sidecar's materialized store key on -`name@version`, not on content, so republishing unchanged-version bytes -never reaches a running or freshly-launched agent; `tool-registry-publish` -refuses to overwrite an existing `name@version` with different content for -exactly this reason. - -## Per-tenant desired-state reconciliation (CL-7584) - -Every real tenant converges onto the tenant desired-state document -(`TENANT_DESIRED_STATE`, `packages/onboarding/src/desired-state.ts`) — a -plain client-side const composed by reference over `DEFAULT_WORKFLOWS`, -`REQUIRED_SEED_TOOL_PACKAGES`, and `DEFAULT_SKILLS`. It is not a hub -table and there is no migration; growing the core workflow set later is -an edit to the upstream constants, never a schema change. - -`reconcileTenantDesiredState` is the one installer: it reads the -tenant's real state with native GETs only and installs ONLY the absent -pins — tool packages first (workspace-pack publish, or a verified -`tarball-url` fetch through `installRegistryTarball`; no external -artifacts exist yet, so the doc pins only workspace-pack this build), -then skills, grants, and workflows together through `seedTenant` with -`confirmDeployments: false`. Sidecar-unavailable (502-class) pins -report `blocked` without throwing; anything else reports `failed` and -is safe to re-run. With every pin present a reconcile pass is reads -only — `seedTenant` is never entered. - -Convergence has three triggers, all driving the same reconciler: - -1. A tenant-create observation: a 201 from `POST /api/tenants` fires a - fire-and-forget reconcile under the creator's minted session - (`apps/hub/src/tenant-create-onboard.ts`). -2. The pending-seed drain: `runOnce` delegates to the same reconcile; - the `pending_seed` row stays the only durable work item (ready - clears it, blocked keeps it, failed keeps it and backs off). -3. The revisit kick: `POST /api/onboarding/provision` fires a kick when - the caller's tenant still has pending pins — how a joined member's - bench converges. - -`GET /api/onboarding/provisioning-status` carries the doc-derived -`steps` list the onboarding page renders; a ready answer collapses it. +No separate migrate command is needed — it is safe to re-run and only +applies what has not already run. ## Memory plane -The memory plane (embeddings-backed recall) is `@corbits/memory`, mounted -by `apps/hub/src/memory-mount.ts`. An explicit `EMBED_BASE_URL` wins; -otherwise `OLLAMA_BASE_URL` is a local embed path, and `bun run dev` -injects the native-Ollama embed env when Ollama is on PATH and neither -variable is set. Without any of those, the hub skips mounting the plane -and logs that it did, rather than failing hub startup — and -`memory_search`/`memory_add`/`memory_list` answer with a plain "memory -isn't set up on this server yet" note instead of erroring. - -Run `bun run scripts/setup-memory.ts` (or `bun run setup:memory`) for a -recommendation tailored to this machine — it checks for native Ollama and -Docker, prints the exact env lines and commands, and writes missing -`EMBED_*` keys into `.env` when a local embed path exists: - -1. **Native first.** A local `ollama pull nomic-embed-text` needs no - container and is the preferred embedding path. -2. **Docker** for the pieces with no good native story — the reranker - (`ghcr.io/huggingface/text-embeddings-inference:cpu-latest`, serving - `BAAI/bge-reranker-base`) and Gotenberg PDF rendering - (`gotenberg/gotenberg:8`) — and as a fallback for embedding when native - Ollama isn't installed. -3. **A remote endpoint**, always available as a third option — including - an existing Ollama, TEI, or Gotenberg instance running elsewhere (the - owner's own Tailscale-tunneled Ollama box is a first-class example, not - a fallback of last resort). - -Two things degrade on purpose rather than failing loudly, and both are -worth knowing before you rely on either: - -- **No embedding configured (`EMBED_BASE_URL` and `OLLAMA_BASE_URL` - unset, and no native Ollama for `bun run dev` to inject):** memory - tools reply with a "not set up" note; search finds nothing. Setting - `EMBED_BASE_URL` later does **not** retroactively embed anything written - while it was unset — migrations create the memory plane's tables either - way, but there is no automatic backfill. -- **No reranker configured (`RERANK_BASE_URL`/`RERANK_MODEL` unset, or a - configured reranker failing at request time):** search still works, - just ordered by vector/full-text fusion alone rather than a - cross-encoder pass — a reranker outage degrades search quietly rather - than breaking it. Setting only one of `RERANK_BASE_URL`/`RERANK_MODEL` - is a boot-time error, not a silently half-enabled reranker. - -## Isolated capacity (exclusive per-workbench sidecars) - -Set `SIDECAR_PROVISIONER=docker` and `DOCKER_PROVISIONER_IMAGE` (see -`.env.example`) to register `@corbits/docker-provisioner` at hub start. This -flips the Workbench Settings › Capacity toggle from "not available on this -server" to available, and lets a tenant's "run this workbench on its own -sidecar" setting provision a real Docker container per exclusive allocation -via the vendored sidecar-allocation subsystem — no additional orchestration -needed on top of what's already wired in `apps/hub/src/index.ts`. - -`DOCKER_PROVISIONER_IMAGE` must point at a built image of `apps/sidecar` -that the local `docker` CLI can run. Build one from the repo root (the -build context has to be the repo root, not `apps/sidecar`, because the -workspace's `@intx/*`/`@corbits/*` deps are `workspace:*` and resolve from -source): - -```sh -bun run build:sidecar-image -``` - -Then point the provisioner at it and flip the toggle: - -```sh -DOCKER_PROVISIONER_IMAGE=corbits-sidecar:dev -SIDECAR_PROVISIONER=docker -``` - -Restart `bun run dev` after setting these, then enable the Workbench -Settings › Capacity toggle to provision an exclusive per-workbench sidecar. +The memory plane (embeddings-backed recall, `@corbits/memory`) needs an +embedding endpoint: an explicit `EMBED_BASE_URL` wins, otherwise +`OLLAMA_BASE_URL` is used, and `bun run dev` injects the native-Ollama +embed env when Ollama is on `PATH` and neither is set. Without any of +these the hub skips mounting the plane and logs that it did, rather than +failing startup; `memory_search`/`memory_add`/`memory_list` then answer +with a plain "memory isn't set up on this server yet" note. + +Run `bun run setup:memory` for a machine-specific recommendation — it +checks for native Ollama and Docker, prints the exact env lines, and +writes missing `EMBED_*` keys into `.env` when a local embed path exists. +Setting `EMBED_BASE_URL` later does not retroactively embed anything +written while it was unset: migrations create the tables either way, but +there is no backfill. + +An optional reranking pass (`RERANK_BASE_URL`/`RERANK_MODEL`) needs both +values set or neither — a half-configured reranker fails boot, but once +both are set a reranker outage degrades search quietly instead of +breaking it. + +See `.env.example` for every memory-plane variable. diff --git a/docs/local-rip.md b/docs/local-rip.md deleted file mode 100644 index 6ac97353a3..0000000000 --- a/docs/local-rip.md +++ /dev/null @@ -1,137 +0,0 @@ -# The local rip - -A "local rip" is the whole platform proof, run by hand on your own -machine, with your own real key: sign up, get a personal bench, connect a -real inference provider, and watch a working workbench come up in front of -you — nothing mocked, nothing skipped. `scripts/e2e/browser/walkthrough.ts` proves the same path end to end -against a real browser with a stubbed provider probe and a scratch -database; this doc is the honest, no-shortcuts version for a person, not -CI. - -This covers the whole path: sign-up through a fully connected bench, with -a real inference provider wired up end to end. - -## Prerequisites - -- [Bun](https://bun.sh) >= 1.2 -- A local [Postgres](https://www.postgresql.org) 17 with pgvector — see - [README.md](../README.md#running-locally) for the `brew install` line -- `git` on your `PATH` — the onboarding connect flow pushes the default - workflow definitions into the hub over git smart-HTTP - (`packages/seeding/src/workflow-push.ts`'s `createGitWorkflowPusher` - shells out to the system `git` binary; it fails loud with an install - hint if `git` isn't found) -- A real API key for the provider you want to connect (this walkthrough - uses OpenRouter's OAuth connect, which needs no key of your own to - paste — see below — but any of the providers in - [`packages/seeding/src/catalog-seed-data.ts`](../packages/seeding/src/catalog-seed-data.ts) - works the same way with a pasted key) - -## 1. Bring up a clean stack - -```sh -cp .env.example .env -bun run reset # only if you have a previous local-rip's state lying around -bun run dev -``` - -`bun run reset` drops the schema and every on-disk asset directory -`bun run dev` created — skip it on a -genuinely fresh checkout. `bun run dev` validates `.env`, confirms -`DATABASE_URL` is reachable, applies pending migrations, builds the web -UI, seeds the administrator account and root tenant, and starts the hub -and the web build. An empty database is a valid hub: boot does not insert -agents, tools, workflows, or skills (see -[README.md](../README.md#running-locally) for exactly what it checks). -No provider API key is needed in -`.env` for this walkthrough — the hub never reads provider env vars — -the point is proving a bench with no hub-owned seed -model gets fully seeded through a person's own connected credential, not -through an operator's key. - -Once `bun run dev` reports the hub and web server up, open -`http://localhost:3000` (or whatever `BASE_URL`/`PORT` you set in `.env`). - -## 2. Sign up - -The app root renders the sign-in screen when you're not authenticated -(`apps/web/src/app.tsx`'s ``); toggle it to sign-up and create -an account with any email/password. Self-serve sign-up is ungated in -stock composition — there is no signup mode left to open, so sign-up -works out of the box here, against `bun run dev` and any deployed hub -alike. `scripts/e2e/browser/walkthrough.ts`'s "signup stays ungated once the -hub is occupied" hop proves exactly that response. - -## 3. Name your bench - -Signing up lands you on `/onboarding`. Submitting the "Create your -workbench" name form calls `POST /api/onboarding/provision` with that name, -which mints your personal bench through the platform's native -tenant-creation route. (An occupied hub joins you to its root as a plain -member instead — `kind: "existing-member"` naming the tenant you joined, -no wizard.) With no hub-owned credential, a minted bench is -unseeded (`seeded: false`, with a -`seedSkipReason` naming why) — the UI keeps you on the credential step -rather than pretending you're done. - -## 4. Connect a provider - -The credential step offers a "Connect with OpenRouter" button (PKCE OAuth, -no client id or secret needed — OpenRouter's connect works with zero extra -config) alongside a paste-a-key form for any provider in -`CREDENTIAL_PROVIDERS` (`apps/web/src/onboarding.ts`), Anthropic included. -Either path: - -1. proves your key or exchanged token with a real, free call against the - provider's own auth-gated endpoint (`testProviderCredential` — - `packages/connections/src/credential-test.ts`) before storing anything; -2. plants it as a credential on your bench alongside that provider's - curated model catalog; -3. deploys and (unlike the OAuth callback's own fast half) confirms the - one default workflow every real signup gets: **assistant** - (`packages/seeding/src/seed.ts`'s `DEFAULT_WORKFLOWS`, CL-7074). - **echo** and **workbench-digest** are on-demand `CATALOG_WORKFLOWS` - entries now — deployed only when something asks for them by name, - never automatically at signup. - -Expect the page to show a short "setting up your workbench" wait while -`/complete-setup` polls, then land on a "Your first routines are running" -screen listing each routine as confirmed running with your credential, and -a "Meet Myra" button into the bench itself. - -### Publishing the corbits-tools registry - -The **assistant** default workflow pins the `@corbits/memory-tools` tool -package (`agents/assistant/src/index.ts`), and that pin only resolves -once a `package-registry`-kind asset named `corbits-tools` carries its -tarball (see `apps/hub/src/index.ts`'s `CORBITS_TOOLS_REGISTRY` comment). -Hub boot does not publish that asset, and neither does anything -hub-triggered (CL-8190): packing a package needs `fs` and `bun build`, -so `reconcileTenantDesiredState` reports an absent `workspace-pack` pin -`blocked` instead of packing it. The only publisher is the operator-run -`bun run publish-tools` (`scripts/publish-tools.ts`), an out-of-band API -client over `@corbits/tool-registry-publish` (bundles -`@corbits/memory-tools` into a self-contained tarball and pushes it -through the hub's native asset REST routes). Descendants inherit it; -`seedTenant` does not pack. Isolated -tests run with no explicit tenant config so the walkthrough's personal -bench is itself the root — then the same publish happens once onto that -bench, and **assistant** comes up live by default; `scripts/e2e/browser/walkthrough.ts` -also deploys **echo** and **workbench-digest** from `CATALOG_WORKFLOWS` -onto that same bench (the on-demand path, exercised directly rather -than assumed) to prove they still come up live too. The default -self-serve story is the other way: the hub ensures no root tenant at -boot — the walkthrough's personal bench is itself the root. - -## 5. Check the Connections surface - -Back in Settings → Connections, the provider you connected shows as -`connected`, cross-referenced from your tenant's own credentials list -(`GET /api/tenants/:id/credentials`) the same way -`packages/settings-ui/src/connections-status.ts`'s `connectorStatus` -does — the credential named `-default` -(`inferenceCredentialName`, `packages/seeding/src/seed.ts`), `status: -"active"`. - -That's the onboard → connect leg, proven with your own real key end to -end. diff --git a/docs/model-seeding.md b/docs/model-seeding.md deleted file mode 100644 index 9a0579932a..0000000000 --- a/docs/model-seeding.md +++ /dev/null @@ -1,135 +0,0 @@ -# Model seeding: current state and upstream inference-discovery - -Written in answer to "are we using Interchange's new model discovery to -handle model seeding?" — we are not, and after reviewing what upstream -actually ships under that name, adopting it would not replace what -workbench's seeding does today. This documents the current path, what -upstream's `inference-discovery`/`inference-testing` packages are for, and -why this is a plan rather than a cutover. - -## How workbench seeds models and providers today - -Everything about "which providers and models a bench knows about" is -curated, hand-authored data, planted through the hub's native catalog HTTP -API — never discovered at runtime: - -- **`packages/seeding/src/catalog-seed-data.ts` (`CATALOG_SEEDS`)** — one - curated seed per supported credential provider: a provider row (its - adapter plugin and base URL) and a small hand-picked model set. This is - what onboarding plants for whichever provider a person connects — including - the OpenRouter PKCE connect - (see [onboarding-openrouter-connect.md](onboarding-openrouter-connect.md)). -- **`packages/seeding/src/seed.ts` (`seedCatalog`)** — walks one - provider's seed - through `POST /api/tenants/:id/catalog/{model,providers,credentials,offerings}`, - idempotently. The offering's `capabilities` field (see - `@intx/types/src/catalog.ts` and - `vendor/intx/db/src/schema/catalog.ts`) is set from - `@corbits/inference-catalog`'s `capabilitiesForDeployment`, which reads - the pinned catalog's probe results: what that exact - `(baseURL, canonicalName)` deployment demonstrated, or the intersection - across other deployments of the same model on the same wire. A - deployment the catalog has never probed — a local Ollama model, an - open-weight relay — is seeded with an empty list, said out loud in the - seed log, because a guessed capability routes real work to a model that - cannot do it. -- **`packages/connections/src/credential-test.ts` (`PROVIDER_TEST_CONFIG`)** — - a second, independent hardcoded table: each provider's free auth-gated - probe endpoint, used to prove a freshly-entered key works before it is - ever stored. Its own `probeModel`/`baseURL` fields exist only to build - that probe request — nothing downstream of the probe reads them. -- **`packages/onboarding/src/complete-credential.ts`** — plants the - browsable catalog for whichever provider was connected, key paste and - OpenRouter/Hugging Face connect alike, via that provider's - `CATALOG_SEEDS` entry. The default workflow set is deployed against - that same entry's first model (`CATALOG_SEEDS[provider].models[0]`) — - the one place a provider's default model is named, so the deploy target - and the catalog it is chosen from can never drift apart — but that - deploy happens in the background, never in the connect request itself - (CL-6457; see [bench-provisioning.md](bench-provisioning.md)). This is also how a - bench the hub's own sign-in hook could only mark `bench_unseeded` (no - hub-owned `ANTHROPIC_API_KEY` configured) finishes seeding: the first - working credential a user connects, through whichever onboarding path - reaches this function, seeds the bench with that credential's own - provider. - -None of this reads any kind of external or upstream model registry. Every -model/provider name in the codebase is a literal string someone typed in. - -## What upstream's inference-discovery/inference-testing packages are - -Read from the upstream Interchange checkout's `packages/` directory (upstream, -not vendored into this repo): `inference-discovery`, -`inference-discovery-{anthropic,google-genai,openai}`, and -`inference-testing`. These are separate from `@intx/inference-catalog`, -which does exist and already ships in this repo: pinned in `package.json` -at `0.3.0` and imported directly (not vendored — see `VENDORED.md`) by -`packages/inference-catalog/src/offering-capabilities.ts` for -`catalogProviders`, the pinned catalog data that `seedCatalog` reads -through today (see "How workbench seeds models and providers today", -above). It is a baked catalog of prior probe results, not a discovery -service — it has no relation to `inference-discovery`/`inference-testing` -beyond the name. - -`@intx/inference-discovery` is **not** a live model/provider discovery -service. Per its own README, it is the runtime for Interchange's internal -"discovery rig": a CLI (`bin/discover`) that makes real, paid calls against -upstream providers to capture fixture bundles proving which -`(provider, model, capability)` tuples actually behave as documented. It -explicitly refuses to run in CI (`assertNotCI`). `@intx/inference-testing` -then replays those captured fixtures in Interchange's own adapter test -suite — it is a deterministic `fetch`-replacement test harness, not -anything workbench would deploy. - -The one piece with any seeding-shaped surface is -`@intx/inference-discovery/catalog`, specifically -`catalogCapabilitiesFor(provider, model)`: given a `(provider, model)` -pair, it returns the capability tags (vision, tools, structured output, -etc.) that pair has a _fixture-bearing, captured_ row for in the -`SUPPORT_MATRIX`. That is a natural fit for workbench's currently-empty -offering `capabilities` field — but it answers "what can this known model -do," not "what models exist." It supplies no provider or model catalog of -its own; `SUPPORT_MATRIX` only has entries for whatever Interchange's own -discovery runs have already captured, and callers still name the -`(provider, model)` pair up front. - -## Whether workbench should adopt it - -Not as a cutover, for three independent reasons: - -1. **Wrong problem.** The owner's question was about "model seeding" — - which providers/models a bench's catalog knows about. Discovery answers - a narrower, different question (capability tags for a model workbench - already named), and does so via paid, non-CI, human-triggered capture - runs against real provider accounts, not a runtime service boot-time - seeding could call. -2. **Not consumable without vendoring.** `@intx/inference-discovery` and - `@intx/inference-testing` are published on npm, but only at `0.2.2` — - the same pre-folded-model version that is the stated reason every other - `@intx/*` package in this repo is vendored under `vendor/intx/` rather - than depended on directly (see `VENDORED.md`). Consuming either package - today would mean adding two more vendored packages (plus a provider - probe plugin per provider used) with their own ledger rows and kill - dates, not a plain npm dependency bump. -3. **No consumer for what it would provide** — retired by CL-6341. - `capabilities` has a consumer now: `@corbits/inference-catalog` filters - on it to answer "which models here can do this kind of work", and the - platform's own source resolution filters on it too. `seedCatalog` - populates it from the pinned catalog's baked probe results, so reasons - 1 and 2 still stand — this reads catalog literals, it does not run a - discovery rig. - -## If this becomes worth doing later - -Live discovery would still be worth revisiting once -`@intx/inference-discovery` has a real npm publish past the folded-model -line (or workbench is ready to vendor it with a ledger row and kill date -like everything else in `vendor/intx/`). What it would add over the baked -literals is coverage of the deployments the pinned catalog has never -probed — local Ollama models and open-weight relays, which seed with an -empty capability list today and are therefore invisible to anything that -picks a model by what it can do. It does not change what models or -providers workbench seeds — that remains curated data, same as -`PROVIDER_TEST_CONFIG` and `catalog-seed-data.ts` are today, since discovery -has no opinion on which models exist, only on what a named model can prove -it does. diff --git a/docs/needs-you.md b/docs/needs-you.md deleted file mode 100644 index f803af6ba2..0000000000 --- a/docs/needs-you.md +++ /dev/null @@ -1,88 +0,0 @@ -# Needs you - -Approval is "needs you." When a running workflow parks on a human decision, -that pause is Interchange's own native state — an `approval` row backed by a -`signal_correlation` row, produced by `AwaitSignalPrimitive` — not a second -concept this repo invents alongside it. This document describes how a -pending approval becomes something a person can read and act on, entirely -through the platform's own routes. - -## What's native, unchanged - -Everything about reading and deciding an approval is Interchange's, -consumed as-is: - -- `GET /api/tenants/:tenantId/approvals` lists a tenant's pending - approvals; each row carries the tool snapshot, its arguments, the run it - came from, and a status. -- `GET /api/tenants/:tenantId/approvals/:approvalId` reads one, in any - status, so a card can render a terminal state after the fact. It is - gated on the same grant approve and reject require, so a refusal here is - proof the viewer cannot resolve it; a foreign id reads as `404`, never a - `403` that would confirm it exists. -- `POST /api/tenants/:tenantId/approvals/:approvalId/approve` and - `.../reject` resolve one. Both run inside a single transaction that - claims the approval's `signal_correlation` row under a `resolved_at IS -NULL` guard and only then flips the approval's status — so two - concurrent decisions (two clicks, two people) can never both land: the - second finds nothing left to claim and is told the approval is already - resolved. -- `GET /api/tenants/:tenantId/runs/:runId` returns the run view, whose - `definitionName` is the display name of the agent that is asking. - -There is no sibling read on the hub. Nothing in this repo creates, claims, -or resolves an approval. - -## Names, not ids - -An `approval` row carries only what it needs to authorize and resume: a -`runId`, an `agentAddress` with an instance id baked into it. None of that -is something a person should have to read. `apps/web/src/pending-approvals.ts` -is the one composer that turns those rows into the display model every -approval surface renders: - -- the run view's `definitionName`, read once per distinct run and cached, is - which agent is asking; -- the account's own membership (`GET /api/me/principals` → `tenantName`) is - which bench the ask is in; -- `headlineFor` (`@corbits/approvals/headline`) turns the tool snapshot into - the line a person reads — the tool's own description, plus the call's - `title` argument when it carries one. - -Not one field on that model holds an identifier. When the naming read is -refused — a per-deployment approver can resolve an approval without holding -the read grant on its run — the approval is still shown, with the asker -unnamed: a blocking decision must never be hidden because its label was -unavailable. - -## Where it shows up - -There is no "Approvals" nav row and no dedicated Approvals page. Pending -approvals surface in three places, all reading the composer above: - -- the shell's top bar chip ("N waiting on you", or "All caught up") and the - second column's signal above the conversation list; -- Mission Control's "Needs you" panel, which approves or denies inline and - invalidates the same query key the chat card does, so both update at once; -- a workbench's Timeline, where an approval takes its place on the - wall-clock spine. - -The in-chat approve card (`@corbits/chat-ui`) reads one approval's live -status through the same composer and posts approve/reject straight to the -native routes. Approve only offers scope "once" (the hub rejects "always" -with a 400); reject collects an optional message through a confirm dialog. - -A refused list read is shown as a failure to load, never as an empty -"nothing pending" — a person who cannot see the queue is told so. - -## What this deliberately does not add - -No new "gate" table, no parallel resolution path, no reimplementation of -`claimTerminal` or the approve/reject transaction, and no read route -mirroring one the platform already serves. `@corbits/approvals` keeps only -what the platform has no reason to own: the grant-allowance gate that -auto-approves a call an existing grant already covers, and the pure -headline builder. Mail-based delivery (notifying a human's inbox the moment -an approval is created, rather than a page polling for it) and -reply-to-resume are real, valuable follow-ups that build on this same -`approval` row — neither is part of this change. diff --git a/docs/notifications.md b/docs/notifications.md deleted file mode 100644 index cf2383af36..0000000000 --- a/docs/notifications.md +++ /dev/null @@ -1,90 +0,0 @@ -# Notifications - -A notification is mail. There is no notification bus, no notification feed, -and no second copy of "things that need you" — the durable record is always a -message in somebody's mailbox, and everything else (an unread count, a push to -Slack, an end-of-day digest) is a read of that mailbox or a fan-out from it. - -## Approval is "needs you" - -The platform already has the concept. A workflow run that parks on a signal is -represented by a `signal_correlation` row and an `approval` row, written -together by Interchange's own register co-write. That pair **is** the needs-you -state. `@corbits/notify` invents no sibling concept, widens no approval kind, -and registers no correlation of its own. It adds exactly one step the platform -was missing: an approval exists, therefore the people who can resolve it have -mail. - -Two other things reach a person the same way and are simpler, because they -have nothing to resolve: a run that failed, and a mention in a thread. - -## The shape - -``` -approval / run failure / mention - │ - ▼ parse (arktype) → render → one message per recipient - mailbox (the durable record, per human principal) - │ - ▼ post-commit, one row per (message, enabled sink) - notify_dispatch → dispatch worker → a sink -``` - -`deliverNotification` parses its input with `NotificationEvent` before anything -is written, so an unvalidated shape can never reach a mailbox. It writes one -message per recipient, keyed on a stable external id — an approval keys off the -approval itself, so a redelivered register frame mails once and only once. - -Sink fan-out is queued strictly after the mail commits, never called inline. -A sink going down cannot cost anybody a notification: the message is already in -the mailbox, and `notify_dispatch` remembers what still owes a copy. - -## `notify_dispatch` - -The one table this package owns, and the only new table in the design. A row is -one attempt stream for one (message, sink) pair: `pending` → `delivered`, or -`failed` with an exponential backoff, or `dead` once the attempt ceiling is hit, -the failure is not retryable, or the sink is no longer registered at all. It is -bookkeeping, not an event log — the fact lives in the mail row it points at. - -With no sink registered, `deliverNotification` queues zero rows and the worker -finds nothing due. That is the correct steady state of a fresh install. - -## Adding a sink - -A sink is a package, not a case in a switch. It exports a -`NotificationSinkPlugin` — a name, `isEnabledFor(scope)`, and `deliver(ctx)` — -and the hub's composition root registers it: - -```ts -import { createSlackNotificationSink } from "@corbits/notify-sink-slack"; - -sinks.register(createSlackNotificationSink(deps)); -``` - -That one line is also the approval gate. Installing a sink is a reviewed change -to the composition root, which is why there is no per-send approval prompt: a -notification about needing a human cannot itself wait on a human. - -Nothing inside `@corbits/notify` changes when a sink is added. The registry -holds plugins by name, refuses two sinks with the same name, and the worker -resolves a queued row's sink by that name. - -## Authorization - -Reading is structural. A mailbox is scoped to a single principal by -construction, so there is no cross-principal read path to guard. - -Emitting is checked. `resolveNotifyContext` authorizes the principal against -the platform's own grants — resource `notify:`, action `deliver`, -evaluated by `@intx/authz` exactly like `approval:` is — and then -resolves the sink's credential. Each of the three ways this can fail has its -own named error: `NotifyGrantMissingError`, -`NotifySinkNotConfiguredError`, `NotifySinkCredentialInvalidError`. - -## What a person sees - -Subjects and bodies are written for a reader: `Approve "send_invoice"?`, -`"Nightly digest" failed`, `Noor mentioned you in "Launch plan"`. Identifiers -never appear in what is displayed — they travel in the message's `refs`, where -the interface uses them to navigate and nothing else. diff --git a/docs/onboarding-huggingface-connect.md b/docs/onboarding-huggingface-connect.md deleted file mode 100644 index 60375cd77f..0000000000 --- a/docs/onboarding-huggingface-connect.md +++ /dev/null @@ -1,178 +0,0 @@ -# Hugging Face connect: notify-to-reconnect - -The onboarding wizard's OAuth tier offers Hugging Face below OpenRouter: -"Sign in with Hugging Face — pay-as-you-go across Groq, Together, -Fireworks & more, billed to your HF account." Unlike OpenRouter's -registration-free flow, Hugging Face requires a self-serve-registered -OAuth app (see "Registering the app" below), and unlike OpenRouter's -durable minted key, HF's token **expires** — this doc also covers the -notify-to-reconnect design that fallback requires. - -## Registering the app - -1. Sign in to a Hugging Face account with access to the client id this - deployment will use, then go to - [huggingface.co/settings/applications/new](https://huggingface.co/settings/applications/new). -2. Create the app **without a client secret** ("Public OAuth apps" — - HF's own term for this). A public app authenticates with PKCE and a - client id alone; there is nothing else to keep secret or rotate. -3. Set the redirect URI to - `/api/onboarding/oauth/huggingface/callback`, where - `BASE_URL` is this hub's configured public origin (the same value - `BASE_URL` in `.env` already names). Register one redirect URI per - deployment (dev, staging, prod each need their own). -4. Copy the app's client id into `HUGGINGFACE_OAUTH_CLIENT_ID` in - `.env` (see `.env.example`). Leaving it unset disables the connect - card's routes; Hugging Face still works as a paste-a-token provider - card either way, since `PROVIDER_TEST_CONFIG` and `CATALOG_SEEDS` - don't depend on the OAuth app existing. - -No other HF-side configuration is needed — scopes are requested at -authorize time, not configured on the app. - -## The round trip - -All server pieces live in `@workbench/onboarding` (mounted by the hub -at `/api/onboarding`); the wizard only navigates. The shape mirrors -OpenRouter's connect (`docs/onboarding-openrouter-connect.md`) with two -differences the provider forces: a registered client id, and an -expiring token. - -1. **`GET /oauth/huggingface/start`** — requires a signed-in session - and a configured `HUGGINGFACE_OAUTH_CLIENT_ID`; without one, it - redirects with `outcome=error&code=not_configured` rather than - crash. Generates a fresh PKCE verifier and its S256 challenge - (`huggingface-connect.ts`, sharing the same PKCE primitives - OpenRouter's flow uses — see `pkce.ts`), seals the verifier into a - single-use state with a ten-minute TTL — encrypted through the same - `CredentialCipher` seam `CREDENTIAL_ENCRYPTION_KEY` backs everywhere - else a secret is encrypted at rest, so the verifier never leaves the - server in the clear — sets that state in an HttpOnly `SameSite=Lax` - cookie, and 302s the - browser to - `https://huggingface.co/oauth/authorize?client_id=...&redirect_uri=...&scope=openid+inference-api&state=...&code_challenge=...&code_challenge_method=S256`. - The `redirect_uri`'s origin comes from the hub's configured - `BASE_URL` (never the request's host header) and its path from the - request, exactly as OpenRouter's callback URL is built. -2. **User approves on Hugging Face.** HF's consent screen names the - registered app and the requested scopes. -3. **`GET /oauth/huggingface/callback?code=...&state=...`** — Hugging - Face, unlike OpenRouter, echoes `state` on the callback; this route - checks the query `state` against the cookie's before even touching - the state store, on top of the store's own single-use/cross-user - checks. It then POSTs a standard `authorization_code` + PKCE form - (`grant_type`, `code`, `redirect_uri`, `client_id`, `code_verifier`) - to `https://huggingface.co/oauth/token`. The response is parsed with - arktype at the trust boundary; a 200 without an `access_token` is a - failure, never a crash or a fabricated success. When the response - carries `expires_in`, it is converted to an ISO `expiresAt` instant. -4. **Only the fast half runs inline, plus expiry metadata.** The minted - token goes through `testAndPersistCredential` with provider - `huggingface`: the free `testProviderCredential` probe - (`GET https://huggingface.co/api/whoami-v2`, HF's own documented - account endpoint — the router's model list is a public catalog and - proves nothing) is the only proof the token gets — a rejected probe - is the sole way this ends in `key_rejected`. Once it passes, the - curated Hugging Face seed from `CATALOG_SEEDS` (three router models - on one provider row, plugin `openai-compatible`, base URL - `https://router.huggingface.co/v1`) is planted and the token is - stored, typed `oauth_token` rather than `api_key`, with the reported - expiry (when present) on the credential's `metadata.expiresAt` field - — a plain HTTP field on the credential row, never baked into a URL or - a log line, alongside the token itself. **Deploying the default - routines against that token never happens here** — see - `docs/onboarding-openrouter-connect.md`'s step 4 for why (the same - duplicate-callback defect and fast/slow split apply here unchanged). -5. **The plaintext token rides forward server-side**, the same - `@workbench/onboarding` pending-seed store OpenRouter's flow uses - (step 5 in `docs/onboarding-openrouter-connect.md`) — a row keyed by - `(userId, tenantId)` carrying `provider: "huggingface"` and the - minted access token, never the expiry metadata (that's already - stored on the credential row by this point) and never anything the - browser sees. -6. **Back to the wizard.** Every ending 302s to - `/onboarding?connect=huggingface&...`, parsed by - `readHuggingFaceConnectReturn` exactly as `readOpenRouterConnectReturn` - parses OpenRouter's: `outcome=connected` with the bench slug, or - `outcome=error` with a short machine code (`state_expired`, - `exchange_failed`, `key_rejected`, `no_bench`, `setup_failed`, - `signed_out`, `rate_limited`, plus HF's own `not_configured`). The - same duplicate-callback recovery OpenRouter's flow runs (step 6 - there) applies here too, keyed on an active `huggingface` credential - instead. -7. **The wizard finishes the job** exactly as it does for OpenRouter — - see step 7 in `docs/onboarding-openrouter-connect.md`. - -## Why the token expires, and why that's fine (notify-to-reconnect) - -Hugging Face's consumer PKCE flow documents no silent-renewal -(`prompt=none`) path and no usable `refresh_token` grant — the only -place those terms appear in HF's docs is an Enterprise-only, -org-bound Token Exchange feature, architecturally unrelated to this -flow. Building a keep-alive loop on either would be building on -documentation that doesn't exist for this shape. So this connect ships -**notify-to-reconnect** instead of silent renewal: - -- **The router fails deterministically.** An expired or invalid token - against `https://router.huggingface.co/v1` (or HF's own - `whoami-v2`) returns a clean `401` — confirmed live, not - documentation-only. That is a reliable trigger. -- **Interchange already degrades gracefully.** `@intx/inference`'s - reactor classifies a `401` as a credential-category error, which is - source-specific and already fails over to the next configured - inference source with **no changes needed here** — this is existing, - tested behavior. The one caveat: failover is per-cycle, not sticky, - so a permanently-expired Hugging Face credential is retried (and - fails over again) every cycle until it's fixed or removed — a - standing one-cycle tax, not a one-time skip. A bench that wants this - safety net needs at least one other configured source for the - reactor to land on. -- **A human is told.** `@corbits/notify`'s `credential-expired` - `NotificationEvent` kind (`packages/notify/src/events.ts`) carries - `tenantId`, `credentialId`, `providerId`, `providerLabel`, and - recipients; `deliverCredentialMail` (`deliver.ts`) sends it through - the same mailbox/inbox/bell pipeline the other three notification - kinds use. The rendered copy: _"Reconnect Hugging Face — your token - expired"_, explaining that reconnecting is the same connect card in - Settings/onboarding, and naming the fine-grained personal access - token (permission: "Make calls to Inference Providers") as the - durable alternative for anyone who'd rather not re-click periodically. - Dedupe is keyed on the credential id alone, not the sweep tick, so a - still-unfixed credential is mailed once, not every poll. -- **A light periodic sweep finds the expiry.** `apps/hub`'s - `credential-expiry-sweep.ts` mirrors `routine-scheduler.ts`'s own - shape: every five minutes it loads every `active` credential on a - swept provider (today: `huggingface`), asks - `@corbits/notify`'s pure `findDueCredentialExpiries` which of them - just crossed their stored `expiresAt`, conditionally claims each due - expiry (`active` → `expired`, guarded so a racing hub replica never - double-mails), and mails the winners. The claim/decision split keeps - the "which credentials are due" logic pure and unit-tested in - `@corbits/notify`, and the store-behind-an-interface seam - (`CredentialExpirySweepStore`) makes the loop itself testable against - an in-memory store, the same store-behind-an-interface seam the hub's - native schedule poller uses. - -This sweep's live delivery — writing an actual mailbox row — depends on -`@corbits/mailbox`'s and `@corbits/notify`'s own migrations being -applied against `DATABASE_URL`; this is the first live caller of the -mailbox delivery adapter the hub's composition root already -constructed, but approval/run-failure/mention notifications still have -no writer wired to it. That gap predates this connect and is unrelated -to it. - -## Dev notes - -- In `vite dev`, the start navigation goes through the `/api` proxy to - the hub; Hugging Face then calls back on the hub origin (`BASE_URL`), - so the wizard resumes there rather than on the vite origin — the same - behavior OpenRouter's flow has. -- The pending-connect state is a signed/encrypted token, not a - server-side lookup, so it survives a hub restart between `/start` and - `/callback` (a dev watch reload, a deploy) as long as - `CREDENTIAL_ENCRYPTION_KEY` is stable — the callback no longer has to - land on the exact process that issued it. See `pkce.ts`'s - `createConnectStateStore`. -- A second OAuth-connected provider whose tokens expire should extend - `SWEPT_PROVIDER_LABELS` in `credential-expiry-sweep.ts`, not add a - second sweep loop. diff --git a/docs/onboarding-openrouter-connect.md b/docs/onboarding-openrouter-connect.md deleted file mode 100644 index 830d2dae1f..0000000000 --- a/docs/onboarding-openrouter-connect.md +++ /dev/null @@ -1,152 +0,0 @@ -# OpenRouter connect: one-click credentials in onboarding - -The onboarding wizard's credential step offers two paths: paste a -provider API key, or — the path the card at the top of the step -suggests first — connect OpenRouter with one click. The connect path is -OpenRouter's registration-free OAuth PKCE flow -(openrouter.ai/docs, "OAuth PKCE"): no client id or secret to -provision, and the exchange at the end returns a **durable user-scoped -API key**, not an expiring access token. That property is why the flow -fits the platform's credential model with no new machinery — the minted -key stores as an ordinary `api_key` credential through the hub's native -credential route, embeds in workflow deployments the same way a pasted -key does, and needs no refresh loop. - -## The round trip - -All server pieces live in `@workbench/onboarding` (mounted by the hub -at `/api/onboarding`); the wizard only navigates. - -1. **`GET /oauth/openrouter/start`** — requires a signed-in session. - Generates a fresh PKCE verifier and its S256 challenge - (`openrouter-connect.ts`), seals the verifier into a single-use - state with a ten-minute TTL (matching OpenRouter's own - authorization-code expiry) — encrypted through the same - `CredentialCipher` seam `CREDENTIAL_ENCRYPTION_KEY` backs everywhere - else a secret is encrypted at rest, so the verifier never leaves the - server in the clear — sets that state in an HttpOnly - `SameSite=Lax` cookie, and 302s the browser to - `https://openrouter.ai/auth?callback_url=...&code_challenge=...&code_challenge_method=S256`. - The callback URL's **origin comes from the hub's configured - `BASE_URL`** (never the request's host header) and its **path from - the request** (so the mount prefix is not guessed at) — this is what - makes the same code correct for `http://localhost:3000` dev and a - deployed hub behind a proxy. -2. **User approves on OpenRouter.** Localhost callback URLs are allowed - by OpenRouter for development; a public callback URL is what names - the app in the user's key list. -3. **`GET /oauth/openrouter/callback?code=...`** — reads the state from - the cookie and consumes it (single use: expired, replayed, unknown, - or another user's state all fail the same way, before any network - call), then POSTs `{ code, code_verifier, code_challenge_method }` - to `https://openrouter.ai/api/v1/auth/keys`. The response is parsed - with arktype at the trust boundary; a 200 without a key is a - failure, never a crash or a fabricated success. -4. **Only the fast half runs inline.** The minted key goes through - `testAndPersistCredential` with provider `openrouter` — the same free - `testProviderCredential` probe (`GET /api/v1/key`, the auth-gated - key-status endpoint; OpenRouter's `/api/v1/models` is public and - proves nothing) is the only proof the key gets, and a rejected probe - is the sole way this ends in `key_rejected`. Once it passes, the - curated OpenRouter seed from `CATALOG_SEEDS` (`catalog-seed-data.ts`: - a small hand-picked model set on one provider row with plugin - `openai-compatible`) is planted and the key is stored through the - hub's native `POST /api/tenants/:id/credentials` — the onboarding - package never stores a secret itself. **Deploying the default - routines against that key never happens here.** That used to run - inline too (`seedTenant`, several seconds of deploy calls per - workflow), which made this request the slow, non-idempotent one: a - browser that fired the callback twice with the same code burned the - single-use state on its first arrival and saw `state_expired` on the - second, for a connection that had actually succeeded. The callback - now returns as soon as the key is proven and stored — see - `complete-credential.ts`'s module comment for the fast/slow split, - and `ensureSeeded` for the deploy step's new home. -5. **The plaintext key rides forward server-side, never to the - browser.** Credential secrets are write-only through the hub's own - API, so nothing can re-fetch the key once this request ends. It is - instead sealed (AEAD, the same `CredentialCipher` the state above - uses) into a row in `@workbench/onboarding`'s own `onboarding` - Postgres schema (`pending-seed.ts`, `createDrizzlePendingSeedStore`) - keyed by `(userId, tenantId)`, with a ten-minute TTL — long enough - for the wizard's own follow-up call. The browser gets nothing from - this step: no cookie, no ciphertext, not even a token — its ordinary - session cookie is what scopes `/complete-setup`'s read to exactly - this row. (Before CL-6031 this rode forward as an HttpOnly - `workbench_pending_seed` cookie; moved server-side because the - browser had no business custodying a sealed copy of the key even - though it could never read it. See `pending-seed.ts`'s module - comment for the full rationale, including the future - `InferenceSource` credential-by-reference primitive that would - remove this store entirely.) -6. **Back to the wizard.** Every ending 302s to - `/onboarding?connect=openrouter&...`: `outcome=connected` with the - bench slug (no routine list yet — that comes from step 7), or - `outcome=error` with a short machine code (`state_expired`, - `exchange_failed`, `key_rejected`, `no_bench`, `setup_failed`, - `signed_out`, `rate_limited`). Before reporting `state_expired` for a - single-use state that came back already consumed, the callback checks - whether this exact session's user already has an active OpenRouter - credential created within the state's own TTL — the twin of a - duplicate callback that already succeeded on its first arrival. Only - a genuinely expired or wrong-session state still errors. The wizard - parses these as untrusted input, maps codes to copy, and strips the - parameters from the URL. The key itself never appears in a URL, a - redirect, or a log line. -7. **The wizard finishes the job.** Landing on `outcome=connected`, the - wizard shows a brief "setting up your workbench" state and calls - `POST /api/onboarding/complete-setup`. That route reads the pending - row for the caller's own `(userId, tenantId)`, runs `ensureSeeded` - (`seedTenant` with - `confirmDeployments: false`, same as before), and answers `seeded` - with the deployed routine names once done. It answers `unseeded` - (never an error) if there is nothing to seed with yet, and two - overlapping calls never double-deploy — every step `ensureSeeded` - drives is itself ensure-then-create (a 409 falls back to a list), - the same tolerance `seedTenant` has always had. - -## CSRF model - -OpenRouter's flow has no OAuth `state` parameter — nothing we send to -`openrouter.ai/auth` is echoed back on the callback. The binding is -therefore deliberately two-sided, and spec-valid per RFC 7636: - -- **Session binding is ours.** The single-use state travels only in our - own HttpOnly `SameSite=Lax` cookie, and consuming it requires the - same signed-in user who started the flow — a callback presented under - a different session (login CSRF, replayed cookie) fails before any - network call, and the exchange never runs. -- **Code binding is OpenRouter's.** The authorization code is bound to - the S256 challenge presented at `/auth`; the exchange only succeeds - with the matching verifier, which never left this process. An - attacker-injected foreign code cannot be redeemed with our verifier. - -This is a real trust dependency: the second half rests on OpenRouter -enforcing the challenge/verifier check server-side, as its docs state. -There is no way to strengthen it client-side without a `state` -round-trip OpenRouter does not offer. - -## Why OpenRouter, and why not the others - -OpenRouter's PKCE flow is explicitly designed for third-party apps — -no client registration, S256, key output. Anthropic and Google both -prohibit third parties from riding their first-party OAuth clients -(their cards stay key-paste), and OpenAI's sign-in is partner-gated. -The one-click card is therefore OpenRouter alone today. - -## Dev notes - -- In `vite dev`, the start navigation goes through the `/api` proxy to - the hub; OpenRouter then calls back on the hub origin (`BASE_URL`, - usually `http://localhost:3000`), so the wizard resumes on the hub - origin rather than the vite one. Cookies are host-scoped (ports are - ignored), so both the session and the state cookie survive the hop. -- The pending-connect state is a signed/encrypted token, not a - server-side lookup, so it survives a hub restart between `/start` and - `/callback` (a dev watch reload, a deploy) as long as - `CREDENTIAL_ENCRYPTION_KEY` is stable — the callback no longer has to - land on the exact process that issued it. See `pkce.ts`'s - `createConnectStateStore`. (The per-user rate limiters are still - plain in-process maps, so a restart does reset those — a client can - immediately retry a start it otherwise would have been briefly - rate-limited on. Low-stakes: worst case is one extra pending state.) diff --git a/docs/package-migrations.md b/docs/package-migrations.md index ac8aaa320b..d188b9a734 100644 --- a/docs/package-migrations.md +++ b/docs/package-migrations.md @@ -1,132 +1,57 @@ # Package migrations Convention for any package under `packages/` that owns its own product -tables — how its migrations are written, tracked, and applied. +tables. ## The rules - **Literal SQL, not codegen.** Every migration is a plain string in a - `{ name, sql }[]` array, hand-written and reviewed as-is — never a schema - diff generated by `drizzle-kit` at apply time. The SQL a reviewer reads in - the PR is exactly the SQL that runs against the database; there is no - codegen step in between where the two could drift. -- **A ledger table the package owns.** Each package tracks which of its own - migrations have run in a table named for itself (`chat_migrations`, - `routine_migrations`, `notify_migrations`, `webhook_triggers_migrations`, - `insights_migrations`, `skills_migrations`, …) — never the platform's own `@intx/db` drizzle - journal, and never `scripts/db-setup.ts`'s - `workbench_setup_migration` table, which only tracks the platform's own - migrations. A package's ledger stays inside the package: if it is ever - extracted into its own repo, its migration history comes with it, with - nothing to disentangle from the platform's. -- **Every package owns a schema named for itself.** No package table ever - lives in `public` — `public` belongs to the platform's own `@intx/db` - schema alone. Every package creates and owns a Postgres schema named for - itself (`mailbox`, `artifacts`, `memory`, `chat`, `routines`, `notify`, - `webhook_triggers`, `insights`, `skills`, …) and puts every table it - owns there, - including its own ledger table. This holds regardless of whether a - package's rows carry a `tenant_id` / `principal_id` column: those are - plain text identifiers, not foreign keys, so a row in - `chat.workbench_settings` refers to a platform tenant by id exactly the - same way whether it sits in `public` or in `chat` — the schema a table - lives in is never load-bearing for that reference. A package landing in - its own schema, never `public`, is what makes it possible to lift the - whole package into another Interchange application without a name - collision or a manual carve-out of "which public tables were actually - ours." + `{ name, sql }[]` array, hand-written and reviewed as-is — never a + schema diff generated by `drizzle-kit` at apply time. What a reviewer + reads in the PR is exactly what runs against the database. +- **A ledger table the package owns.** Each package tracks which of its + own migrations have run in a table named for itself (e.g. + `webhook_triggers_migrations`) — never the platform's own `@intx/db` + drizzle journal, and never `scripts/db-setup.ts`'s own migration + tracking, which only covers the platform. If the package is ever + extracted into its own repo, its migration history goes with it. +- **Every package owns a schema named for itself.** No package table + lives in `public` — that belongs to `@intx/db` alone. A package's + tenant/principal columns are plain text ids, not foreign keys, so + which schema a table lives in is never load-bearing for that + reference; it only matters for keeping the package extractable. - **Every migration is transactional.** A migration's SQL and its ledger - `INSERT` run inside one `sql.begin(...)` transaction. If the SQL fails - partway, nothing about that migration — not a table, not a column, not the - ledger row — is left behind half-applied; the next run retries the same - migration from a clean slate instead of getting stuck on a schema that - looks partially migrated but has no ledger row to explain why. -- **A session-level advisory lock guards the whole run.** Two hub replicas - can boot at the same time and both call a package's migration runner - concurrently; without a lock, both can see a migration as unapplied and - race the same ledger `INSERT`, crashing the loser on a duplicate key. - `@corbits/migration-runner`'s `applyPackageMigrations` holds - `pg_advisory_lock(hashtext(ledgerTable))` for the whole bootstrap-and-apply - run and releases it in a `finally`, so the second replica simply waits and - then finds every migration already applied. + `INSERT` run inside one `sql.begin(...)` transaction, so a failure + partway through never leaves a half-applied table with no ledger row + to explain it. +- **A session-level advisory lock guards the whole run**, so two hub + replicas booting at once don't race the same ledger insert. + +Mechanics — schema/ledger bootstrap, the transactional apply loop, and +the advisory lock — live once in `@corbits/migration-runner`'s +`applyPackageMigrations`; a package's own `migrations.ts` supplies only +its schema name, ledger table name, and literal migration array. See +`packages/webhook-triggers/src/migrations.ts` for the reference shape. + +## Wiring a new package in + +`scripts/db-setup.ts` imports each installed package's +`applyXMigrations(databaseUrl)` and calls it, in an explicit literal +list, right after the platform's own migrations — no discovery magic, +no globbing. Add the new package's entry to that list. + +A package that ships its own independently-owned migration runner as +part of a larger engine (its own schema, its own connection handling — +`@corbits/mailbox` is the example) can instead be wired as a thin +delegate: `scripts/db-setup.ts` opens a connection and calls the +package's own exported runner, with no opinion on what SQL it runs. +Reach for this only when the package already has such a runner, never as +a shortcut to skip writing a ledger table. ## Why not `drizzle-kit` codegen -Drizzle is used as a query builder and for the platform's own type -definitions, but package migrations are never generated by pointing -`drizzle-kit` at a schema and letting it diff. The SQL a PR reviewer approves -must be the SQL that runs — a generated migration reviewed as a Drizzle -schema diff, with the actual `CREATE TABLE`/`ALTER TABLE` statements -produced later by a separate codegen pass, breaks that: the reviewed -artifact and the applied artifact are two different things, and a codegen -bug or a stale generated file becomes a way for them to diverge silently. -Literal SQL closes that gap — what's in the file is what runs. - -## The three shapes - -Looking at `INSTALLED_PACKAGE_MIGRATIONS` in `scripts/db-setup.ts`, a -package's migration runner takes one of two live shapes today (a third, -non-transactional self-contained shape existed until CL-6017 unified every -package onto the transactional pattern — see below): - -1. **Self-contained, transactional** — `@corbits/chat`, `@corbits/notify`, - `@corbits/webhook-triggers`, `@corbits/insights`, - `@corbits/skills`, `@corbits/bench`, - `@corbits/inference-catalog`, `@corbits/evals`. - (`@corbits/workflows`'s `./deploy-source` followed this shape until - CL-7591 deleted it; it is no longer wired through - `scripts/db-setup.ts`.) - The package's `src/migrations.ts` owns only a literal `{ name, sql }[]` - array and a thin `applyXMigrations(databaseUrl)` wrapper; the mechanics — - schema/ledger bootstrap, the transactional apply loop, and the advisory - lock — live once in `@corbits/migration-runner`'s - `applyPackageMigrations`, called with the package's schema name, ledger - table name, and migration array. `scripts/db-setup.ts` imports the - package's `applyXMigrations(databaseUrl)` function directly and calls it - after the platform's own migrations. See - `packages/inference-catalog/src/migrations.ts` for the reference - implementation every other package now matches. -2. **Delegated to an external package** — `@corbits/mailbox`. The package - ships and owns its entire migration story (its own literal SQL, its own - ledger, its own `mailbox` schema) behind a single exported runner - (`runMailboxMigrations`). `scripts/db-setup.ts` (via - `packages/inbox/src/migrations.ts`'s thin wrapper) only opens a - connection and calls that runner — it has no opinion on what SQL runs. - `@corbits/artifacts` and `@corbits/memory` follow the same delegated - shape but aren't wired through `scripts/db-setup.ts` at all: they run - their own migration runner at hub-boot mount time instead (see - `apps/hub/src/artifacts-mount.ts` and `apps/hub/src/memory-mount.ts`), - since both are optional engines that may not be configured in every - deployment. - -Before CL-6017, `@corbits/chat`, `@corbits/notify`, -and `@corbits/webhook-triggers` were self-contained but -**not** transactional: each migration's SQL and its ledger insert were two -separate statements, so a failure between them could leave a table created -with no ledger row to show for it. CL-6017 brought them onto the same -`sql.begin` pattern `@corbits/insights` already used, closing that gap. - -Before CL-6005, `@corbits/chat`, `@corbits/insights`, -`@corbits/notify`, and `@corbits/webhook-triggers` put their tables directly -in `public`, on the reasoning that a `tenant_id`/`principal_id` column tied -them to the platform's own schema. CL-6005 gave each of them a schema named -for itself instead (`chat`, `insights`, `notify`, -`webhook_triggers`) — those columns are plain text, never a real foreign -key, so nothing about the reference actually required sharing `public`. -Schema changes like this are hard cutovers, not data-preserving migrations: -pre-GA there is nothing to migrate, and an existing dev database picks up -the new schema by running `bun run reset` rather than by an in-place -`ALTER TABLE ... SET SCHEMA`. - -## Which shape to use for a new package - -**Transactional, self-contained** (shape 1 above), in a Postgres schema -named for the package. Copy `packages/inference-catalog/src/migrations.ts`'s -shape: a `pgSchema("")` in `schema.ts`, a literal migration array -whose SQL qualifies every table/index with that schema, a package-named -ledger table living in the same schema, and an `applyXMigrations` that -calls `@corbits/migration-runner`'s `applyPackageMigrations` with that -schema, ledger table, and migration array. Reach for the delegated shape -only when the package already ships its own migration runner as part of a -larger, independently-owned engine (its own schema, its own connection -handling) — not as a shortcut to skip writing a ledger table. +Drizzle is used as a query builder and for type definitions, but package +migrations are never generated by pointing `drizzle-kit` at a schema and +diffing. A generated migration reviewed as a schema diff, with the +actual SQL produced later by codegen, lets the reviewed artifact and the +applied artifact drift. Literal SQL closes that gap. diff --git a/docs/portable-client-bootstrap.md b/docs/portable-client-bootstrap.md deleted file mode 100644 index c1bc140f71..0000000000 --- a/docs/portable-client-bootstrap.md +++ /dev/null @@ -1,86 +0,0 @@ -# Portable client bootstrap - -The client holds a needs-list manifest and drives a stock Interchange hub -with stock routes only. No server proxies, tables, mounts, grant planting, -or shims — the lane ports to nearly any stock hub. - -## Contracts - -- No parent bench. Auth creates the user's primary tenant (global - workspace). Myra runs at top level as a `kind: "workflow"` principal. -- Bench = one user workbench. Workbenches are child tenants of the primary - tenant — a trust boundary with separate creds, grants, wallet, and - policy — and each workbench's chat is its **primary thread**: the first - `conversation.message` sent in the child tenant. Other agent work and - background comms (delivery threads, onboarding blocks) fork as - **sub-threads** via `In-Reply-To`/`References` on the sent first message. -- DMs are threads only: never tenants, never created here. A DM is the - participant-filtered thread set between the user's principal chain and - exactly one agent chain, derived from stock mail (`To`/`Cc` sets; `M:N` - via the `List-ID` header where the hub supports it). Creation - idempotency rides the native `Message-ID` the hub stamps and returns — - nothing custom. Legacy `dm:` rows in the client store never - resolve to tenants and are dropped on load. -- Workbench metadata beyond `Subject`/`List-ID` (icon, prefs) stays in the - existing client-held store beside created ids — never a server table. - -## Files - -- `apps/web/src/threads.ts` — thread-native derivation: `deriveThreads` - (native `Message-ID` grouping with `In-Reply-To`/`References`, - `List-ID`, and subject fallback), `deriveDmThreads` (DMs as - participant-filtered threads), and `buildForkReference` (the sub-thread - ancestry a fork first message carries). No idempotency keys anywhere. -- `apps/web/src/needs-list.ts` — manifest (`buildNeedsList`, - arktype `NeedsListSchema`/`parseNeedsList`) plus the client-held stores - (`childTenantStore`, scoped by hub origin and account id, carrying the - primary-thread `Message-ID` beside created ids; `threadLinkStore` - carrying sub-thread fork links). Invalid rows are skipped per row, - never discarding the whole store. -- `apps/web/src/needs-converge.ts` — the `StockHub` port, snapshot read, - and `convergeNeedsList`. Creation sequence per workbench: stock - `POST /api/tenants { parentId }` → send the primary-thread first - message → fork sub-threads as needed via `forkSubThread`. Writes happen - only after every gap check passes. DMs derive from the caller-supplied - stock mail snapshot (`deriveDmThreads`); per-agent mailbox search - (`agent-mailbox-reads`) and full thread reads (`thread-fork-context`) - stay typed upstream gaps until the hub exposes stock routes for them. -- `apps/web/src/client-bootstrap.ts` — `bootstrapClientSession` wires the - manifest, port, and store together for signup/open bootstrap and returns - a typed result (`ready`, `stock-capability-missing` with an upstream-gap - note, `stock-hub-request-failed`, `client-config-missing`). Myra's - definition refId resolves from the client workflow catalog - (`assistant`, productized as Myra); deployment source/offering ids come - from explicit caller config only — never guessed. -- Wired into first-open (`main.tsx`) and signup (`pages/onboarding-page.tsx`) - beside server provisioning. Non-gating: a stock gap is logged, not shown. - -## Stock endpoints used - -- `GET /api/me/principals` — owned memberships (finds the primary tenant), - every page followed via `nextCursor` -- `GET /api/tenants/:id` — tenant read -- `GET /api/tenants/:id/principals?limit=100` — principals incl. workflows, - every page followed via `nextCursor` -- `POST /api/tenants` — create workbench child tenants (never for DMs) -- `POST /api/tenants/:id/members/invite` — invite by email (membership is - the `To`/`Cc` participant set; no membership tables) -- `POST /api/tenants/:id/workflows/deployments` — deploy Myra when absent - (exact caller-supplied input, posted unchanged) -- `POST /api/tenants/:id/mailbox/messages` — send run-mail: the - primary-thread first message, then sub-thread forks with native - `In-Reply-To`/`References` ancestry -- `GET /api/tenants/:id/mailbox/messages` — list run-mail per agent - mailbox, every page followed via `nextCursor`; the mailbox read splits - into the primary thread plus forked sub-threads - -## Known upstream gaps - -Stock Interchange exposes no per-agent mailbox search route -(`agent-mailbox-reads`) and no full thread route -(`thread-fork-context`), so DM derivation reads the mail snapshots the -caller supplies and fork context stays client-held until those stock -routes land. Likewise Myra deployment without caller-supplied inputs -(`deploy-workflow-inputs`), projecting a workflow identity into a child -room by `refId` (`project-workflow-principal`), and non-member role -assignment (`principal-roles`) stop before writes rather than guessing. diff --git a/docs/posix-tools.md b/docs/posix-tools.md deleted file mode 100644 index adb8186fa1..0000000000 --- a/docs/posix-tools.md +++ /dev/null @@ -1,23 +0,0 @@ -# Posix tools (design note — not implemented) - -Nothing under this name ships in workbench today. There is no -`@corbits/posix-tools` package anywhere in this tree. Upstream Interchange -does publish a real package in this space, `@intx/tools-posix` — its -`sidecar-bundle:run_shell` tool already appears in this repo's fixtures -(`packages/mocks/src/ollama/scenarios.ts`, -`vendor/intx/hub-sessions/src/event-collector.test.ts`) — but workbench -does not depend on it and has built no integration against it. Treat -everything below as a proposal for that integration, not a description of -current behavior. - -The design: run only on a workbench placed on isolated capacity -(`sidecarPlacement` enabled, backed by a real `SidecarProvisioner`) — the -dedicated container is the sandbox boundary, so there is no in-process -sandboxing to build. Every write-capable tool it exposes (shell, write) -would declare `approval: "ask"`, so each call surfaces through the -existing human-approval path before it runs — no tool in this package -would ever get a standing auto-approve. The sandbox policy would be a -named, auditable, per-workbench setting, defaulting closed until an -operator opts a workbench in — never a single all-or-nothing flag. This -mirrors the pattern Interchange's own sidecar capability rules use: -capability is inert until named, scoped, and explicitly turned on. diff --git a/docs/proposals/experience-redesign.md b/docs/proposals/experience-redesign.md deleted file mode 100644 index 5d2f242e72..0000000000 --- a/docs/proposals/experience-redesign.md +++ /dev/null @@ -1,117 +0,0 @@ -# CL-6077: Experience redesign — current → proposed - -One page. Names real surfaces (`apps/web/src/routes.tsx`, -`packages/settings-ui/src/section-registry.tsx`, -`packages/chat-ui/src/workbench-settings/*`). No hypotheticals. - -## The core move: spaces are tenants - -`packages/chat-ui/src/workbench-settings/surface.tsx` already says it — -_"workbenches are tenants, so their settings replace the whole stage."_ We -finish what's started: a space is its own tenancy. It can hold **its own -named credentials, its own agents, its own grants**, inherited from the -bench unless overridden. That's why settings stop being a buried tree — -they're not a separate destination, they're a property of the space (or -agent, or routine) you're already in. - -## Rail: 9 destinations → 5 - -| Current (`routes.tsx`) | Verdict | Proposed | -| ---------------------------------- | ------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------- | -| Myra (`/`) | keep | folds into **Spaces** — Myra is a chat, not a special rail slot | -| Spaces (`/c/:id`) | keep | **Spaces** | -| Inbox (`/inbox`) | keep, scope tighter | **Inbox** — needs-you only (already the header intent) | -| Routines (`/routines`) | keep | **Routines** | -| Library (`/library`) | keep | **Library** | -| Insights (`/insights`) | kill as rail item | metrics move into the space/routine they measure (Sturgeon: low-traffic page, `insights-page.tsx`) | -| Agents (`/agents`, redirect only) | already killed (CL-5990) | confirm: no rail entry, no settings-tree page either — agents are created in-context per this proposal, directory lives one click from Spaces | -| Skills (`/skills`, redirect only) | already killed (CL-5990) | same as Agents — fold into agent creation, not a standing page | -| Settings (`SETTINGS_PATH`, footer) | keep, shrink | footer icon stays, but it opens **account + bench-wide admin only** | - -**Target rail: Spaces, Routines, Library, Inbox, Search — Settings stays a footer icon, not counted in the 5.** - -## Settings tree: `section-registry.tsx` flattened - -Current groups: Personal (Agent, Notifications, Account) / Workspace (Bench, -People, Roles, Grants, Connections, Audit). - -| Section | Verdict | Where it goes | -| --------------------------------------- | ------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `agent` (personal agent config) | move-into-context | agent's own chat — same pattern `workbench-settings/surface.tsx` already gives workbenches | -| `chat`/Notifications | keep | Settings (genuinely global) | -| `account` | keep | Settings | -| `bench` | keep | Settings (bench is the outer tenant) | -| `people` | keep, fix | Settings — filter machine principals from the human roster (in flight per owner) | -| `roles` / `grants` | keep, contextual add | Settings for bench-wide; **also surfaces inline** on a space's Access tab (`workbench-settings/access-section.tsx` already exists — extend it to grants, not just membership) | -| `connections`/`credentials-section.tsx` | merge + move-into-context | kill as the _only_ entry point. Settings → Connections becomes bench-key **management** (rotate, name, revoke); **creation happens at point of use** (see below) | -| `granola-webhook-card.tsx` | kill the circularity | binding a webhook no longer requires leaving the routine. See flow 3. | -| `audit` | keep | Settings | - -**Result: Settings page shrinks from 8 sections to 5 (Account, Notifications, Bench, People, Connections+Audit), all bench-scoped. Nothing space-scoped lives in the settings tree anymore.** - -## Connect/credentials model: named, multi-key, scoped - -Today (`credentials-section.tsx`, `connections-section.tsx`): one flat list -of credentials per tenant, no name-at-point-of-use picker, and Granola's -webhook (`granola-webhook-card.tsx`) requires an _existing_ routine bound -before you can get a URL to paste into Granola — you leave the flow, -create the routine, come back. - -Proposed (CL-6078 hangs off this): - -- Keys are **named** ("Anthropic — prod", "Anthropic — client X") and live - at **bench scope or space scope**. A space with no key of its own - inherits the bench's. -- **Creation is inline, not a settings trip.** Every place a key is - needed — space creation, agent creation, routine creation, the Granola - webhook card — offers a picker defaulting to "use the bench key," with - "add a key for this space" one click away, inline, without leaving the - dialog. -- **Settings → Connections is management only**: rename, rotate, revoke, - see what's using a key. It is never the only door to unblock a creation - flow. -- Granola specifically: the webhook secret is minted as part of creating - the `granola-call` routine, not after — the routine wizard's last step - _is_ the copy-this-URL-into-Granola step, no return trip. - -## Three golden flows - -**1. First run: connect → talk to Myra** (`onboarding-page.tsx` today: 3 -steps — name workbench, add inference credential, orient) - -1. Name the workbench. -2. Add one inference key (unnamed default is fine here — it's the bench's - first key, naming matters once there's a second). -3. Landed straight in a chat with Myra — no orientation tour, no - dashboard. The "orient" step becomes Myra's first message, not a screen. - -**2. Make an agent** (`create-agent-dialog.tsx`, CL-6074 — treat as done) - -1. From Spaces, "New chat" → agent tab (`new-workbench-dialog.tsx` already - has this picker) → "Create new agent." -2. Two fields: name, what it's for. (System prompt/model/skills stay, but - collapsed under "Advanced" — YAGNI on exposing them by default.) -3. If the agent needs its own key (a client-specific provider key, say), - "add a key for this agent" is inline in the same dialog, defaulting to - inherit the space's/bench's. -4. **Create & chat** — lands directly in the new DM. No settings - round-trip, no directory page. - -**3. Make a routine from a space** (`routines-page.tsx` today: standalone -page, delivery workbench required at creation, Granola webhook is a separate -Connections-section card) - -1. From inside a space, "New routine" (not from the standalone Routines - page — routines page becomes the _list_ of what's running, not the only - place to start one). -2. Pick source: catalog template or describe-to-agent. -3. Delivery workbench is pre-filled as _this_ space — the one you're - already in, not a picker over every workbench. -4. If the routine needs a credential (a provider key, or — for Granola — - the inbound webhook secret), it's minted right here, inline, scoped to - this space by default. For Granola: the last step of the wizard _is_ - the copy-this-URL-into-Granola step. No trip to Settings → Connections - first. -5. Routine is live; runs post into the space as messages — results - accumulate as conversation, not a separate runs table you have to go - find. diff --git a/docs/proposals/workbench-architecture.md b/docs/proposals/workbench-architecture.md deleted file mode 100644 index 15b49ade63..0000000000 --- a/docs/proposals/workbench-architecture.md +++ /dev/null @@ -1,323 +0,0 @@ -# Workbench architecture: what's wrong, what Buzz does, what to build - -> **Superseded in part (CL-6322).** The diagnosis, the Buzz teardown, and the -> evidence below all stand and are still the reference. The _recommendation_ -> does not: sections 4-6 propose a bench event log as a new layer over the -> existing anchor-run model. The decision taken instead is **room = data, -> turn = run** — retire the workbench anchor entirely, which makes most of -> that layer unnecessary. See CL-6322 and its attached plan of attack. - -A decision memo. Three symptoms — workbenches feel slow, they feel laggy, -and nobody can tell whether multi-agent actually worked — are three -different causes at three different layers. This walks the evidence, reads -Block's [Buzz](https://github.com/block/buzz) as the closest prior art, -inventories what Interchange already offers, and puts three architectures -side by side with a staged recommendation. - -## 1. What is actually wrong - -### Laggy: the live stream carries pings, not data - -`packages/chat-ui/src/use-workbench-stream.ts:133-137` states the design -outright: `chat.agent` "is what makes a connected stream refresh the -timeline at all." Every server event triggers a refetch rather than -delivering the change. - -One send, from a real request log: - -``` -19:30:13.960 POST .../messages 201 28ms -19:30:13.982 PUT .../read-state 200 15ms -19:30:14.222 GET .../pins 200 9ms -19:30:14.225 GET .../threads 200 12ms -19:30:14.230 GET .../messages 200 18ms -19:30:14.242 PUT .../read-state 200 5ms -``` - -Six requests to show one message the client already had in hand. Every -agent event repeats the pattern. Two more taxes ride along: duplicate -identical fetches in the same millisecond (`providers?inherited=false` -twice, `mcp-servers` three times) from double-mounted hooks with no -dedupe, and presence maintained by `POST /presence/rooms/:room/heartbeat` -every 15 seconds per client with `provider-health` polled every 30. - -Nothing here is a slow query — every response is under 30ms. The read path -is just built to re-derive instead of receive. - -### Slow: a message costs a deploy - -A message travels hub → mail → sidecar → agent → reply-bridge → hub. Each -participant that isn't resident pays a full `deployAtHead` before it can -hear anything (`packages/chat/src/platform-adapter.ts:324-383`). The -durable envelope is MIME plus a PGP detached signature per message; the -run's state moves as git packs over the sidecar WebSocket. - -Worth killing one assumption: the git substrate is **not** the per-token -bottleneck. A full run repo holds 9 commits, and events arrive batched -("append 2 workflow events"). The cost is cold start and hop count, not -the substrate. - -### Unverifiable: no per-message identity - -One workbench is one immortal run. Every message, tool call, and reply -interleaves in that single run's event log. To answer "why did this bench -die" earlier today, the only available method was reading -`.data/sidecar/workflow-runs//runs//events.jsonl` off disk by -hand. - -Interchange ships `run-view.ts`, `run-events-view.ts`, and -`observability.ts` in `hub-api/src/routes/`. Nothing in this repo consumes -any of them. - -### The structural knot underneath - -Two properties combine into the failure class we hit today: - -1. **Bench identity is fused to run identity.** `platform-adapter.ts:618` - resolves a workbench by `findFoldedRunById(deps.db, input.workbenchId)` - — the workbench id _is_ the run id. That comes from self-anchoring - (`packages/folded-runs/src/launch.ts:440-473`, tagged `[Intx gap] -CL-6044`): a folded run sets `anchorRunId` to its own id because - `receiveWorkflowRunPack` refuses packs from a row that doesn't look - like a deployment anchor. - -2. **Failure granularity is the whole run.** `workflow/src/runtime/run.ts:1448` - — an unbounded step whose turn exhausts retries throws, ending the run. - `run.ts:2083` — an `onTrigger` body child that fails ends the section - run: _"Terminal-is-final … the run does not relaunch."_ - -So one bad message kills a bench permanently, and because identity is -fused, the bench cannot be given a new host. That is exactly what happened -to `run_26da93eb…`: an event-only mail (empty text part, JSON attachment) -resumed the parked anchor with `""`, `agent.send` threw on empty content, -`StepFailed(retriesExhausted)` → `RunFailed`, 676ms after the run started. -Every later chat message then mailed a terminal run. - -## 2. How Buzz works - -[Buzz](https://block.xyz/inside/introducing-buzz-where-humans-and-agents-work-together) -is Block's open-source humans-and-agents workspace, Apache-2.0, built on -Nostr. Six mechanics matter here. - -**One signed event log is the source of truth.** Every message, reaction, -workflow step, review approval, and git event is a signed event in one -log. The relay is the only path: "no peer-to-peer event exchange, no -gossip, no replication." - -**Kind numbers are the dispatch switch.** Each event carries a `kind` -integer. "Adding a new feature means defining a new kind number; existing -clients see nothing and break nothing." - -**Clients subscribe with filters and receive the data.** The relay's -`SubscriptionRegistry` indexes subscriptions three ways — -`(channel_id, kind)` for O(1) exact match, `channel_id` wildcard, then -global — and fans out matching events over WebSocket. Multi-node -propagation goes through Redis pub/sub with local-echo dedupe. A client -never refetches to learn what changed. - -**Ephemeral kinds never touch storage.** Kinds 20000–29999 (typing, -presence heartbeat) fan out through the same subscriptions but skip both -the database and the audit log — "enabling real-time collaboration without -bloat." - -**Threads are parent references, not objects.** A reply tags its ancestor -(`["e", ""]`); a thread is a filter query -(`{"kinds":[9],"#e":[""]}`). No separate thread entity, no -separate execution. - -**Agents are ordinary clients, and the harness owns turn-taking.** Agents -authenticate the same way humans do, subscribe with the same filters, and -get no special server path — "scoped identity, not permission flags." The -decision to _prompt a model_ lives in the harness (`buzz-acp`): when an -agent is `@mention`ed, the event is queued per channel, and "at most one -prompt is in-flight per channel; subsequent @mentions queue until the -agent responds," with multiple mentions batched into a single prompt. - -That last one answers the multi-agent question directly. Everyone hears -everything because everyone is subscribed; nobody pile-ons because the -harness serializes prompts per room. It's "everyone reads, one speaks" -implemented at the agent, not enforced by the room. - -### Buzz vs workbench, honestly - -| | Buzz | Workbench today | -| ------------------- | ----------------------------------------------- | -------------------------------------------- | -| Source of truth | one append-only event log | `session_mail` + chat projection tables | -| Delivery to clients | subscription fan-out, carries data | SSE ping → 4-6 refetches | -| Typing / presence | ephemeral kinds, never stored | HTTP heartbeat every 15s | -| Agent connection | long-lived subscribed client | deployed run, woken per message | -| Turn-taking | harness queue, one prompt in flight per channel | host-routes unmentioned, fan-out per mention | -| Threads | parent tag, queryable | own tables, no execution identity | -| Extensibility | new kind number | new route + client change | -| Per-message trace | every step is an event in the log | interleaved in one immortal run | - -Two things Buzz does _not_ solve for us, and which we should not copy -away: Interchange's credential resolution and approval gating are real -assets, and Buzz's Nostr keypair-per-participant model is a different -identity story than tenants and grants. The lesson to take is the -**transport and turn-taking shape**, not the identity substrate. - -## 3. What Interchange already gives us - -Used today: agents, inference, credentials, grants/authz, approvals, -mail bus, git-backed run state, workflow state machine, sidecar -allocation. - -Available and unused: - -- **`onTrigger` sections** (`workflow/src/definition/primitives.ts:210-237`) - — a long-lived section that runs a body sub-DAG "once per occurrence of - that trigger, each occurrence a separate child run of `body` (own run - id, own event log), all WITHIN the one living workflow run." Per-message - runs, which is the observability primitive we're missing. The sidecar - already wires `spawnSuspendableChild` - (`apps/sidecar/src/workflow-substrate-factory/index.ts:601`). Chat was - built this way once (`3ae984f3`) and moved off it in `0daf6bcf` to kill - a relay DAG, not because the primitive was wrong. -- **Run views and observability routes** — `run-view.ts`, - `run-events-view.ts`, `observability.ts`. -- **Non-fatal edges as an established pattern** — `loop` has - `onExhausted`, `gate` has `else`, `awaitSignal` has `onTimeout`. - Long-lived sections simply never got one. - -What Interchange does not offer, and won't without upstream work: - -- A subscription/fan-out plane that delivers conversation data to - browsers. Its wire is hub↔sidecar, not hub↔client. -- An ephemeral event tier (typing, presence, token deltas). -- Failure containment per message. -- Any notion of a room whose membership spans humans and many agents; - mail is point-to-point, and fan-out is workbench's own loop. - -## 4. Three architectures - -### Option 1 — Bench event log, mail as transport - -**What.** A workbench-owned append-only event log (`kind`, `workbench_id`, -`thread_id`, `parent_id`, `payload`, `run_id`) plus a fan-out service and -one subscribe endpoint that streams events _with payloads_. Writes append -to the log first — instant echo — then dispatch to participants over -Interchange mail asynchronously. Agent token deltas stream as ephemeral -kinds that are never stored. Threads become a parent reference on the -event. Presence and typing move to ephemeral kinds and stop hitting -Postgres. - -**Why.** It fixes all three symptoms with one mechanism. Local echo and -token streaming become natural rather than bolted on; the four-to-six -refetches per event disappear; `run_id` on every event turns -"did multi-agent work" into a query. Interchange keeps everything it is -good at — agents, inference, credentials, approvals — and stops being -asked to be a browser transport, which it never claimed to be. - -**How.** New package `@corbits/bench-log`: schema, kind registry, append -API, in-process fan-out (Redis later, only when multi-node), and a -`GET /workbenches/:id/subscribe` stream. `packages/chat` writes through -it instead of writing tables directly. `chat-ui` consumes typed events and -stops refetching. The reply bridge publishes deltas as ephemeral kinds. -Existing chat tables become projections rebuilt from the log. - -**Cost / risk.** The largest new surface workbench would own — but -incremental, and each stage ships alone. The real risk is two sources of -truth: the log must be authoritative and mail must become a dispatch -detail, or we get drift. Mitigation: the log is the only write path from -day one. - -### Option 2 — Stay inside Interchange - -**What.** Adopt `onTrigger` for per-message child runs, consume the run -and observability routes, forward existing agent-event payloads through -SSE instead of pinging, dedupe client fetches, pre-warm participants, add -the non-fatal failure edge. - -**Why.** Least code owned here; gaps go upstream where other builds -benefit. Genuinely fixes the unverifiable symptom and much of the laggy -one. - -**How.** Rebuild the bench definition as an `onTrigger` section (deploy -materializes the body to its own asset), containment edge at -`run.ts:2083`, SSE payload forwarding, client dedupe, warm pool. - -**Cost / risk.** Caps out. Mail's MIME+PGP envelope per message is a -latency floor; there is no ephemeral tier, so typing and token deltas stay -awkward; threads still have no execution identity; every new event type -needs a route plus a client change rather than a number. - -### Option 3 — Fork the conversation model - -**What.** A workbench becomes a first-class room with its own event log -_and_ its own agent-connection model: agents connect as persistent -subscribed clients, Buzz-style, instead of being deployed and woken per -message. Interchange supplies agent execution, inference, credentials, and -approvals only. - -**Why.** Highest ceiling. Cold start disappears because agents are already -connected. Real per-thread parallelism — one in-flight prompt per (agent, -thread) rather than per bench. This is Buzz's architecture with -Interchange as the execution and credential plane. - -**How.** Everything in Option 1, plus a resident agent-client host -replacing wake-per-message, and a rethink of where grants and approvals -attach when an agent is a subscriber rather than a run. - -**Cost / risk.** Largest divergence. Grants, approvals, and credential -delivery all currently hang off the run; moving agents off runs means -re-seating those seams. Not a first move. - -### Side by side - -| | Option 1 | Option 2 | Option 3 | -| ------------------------ | --------------------------------------------- | ------------------ | -------- | -| Fixes laggy | yes | partly | yes | -| Fixes slow | partly (echo + streaming; cold start remains) | partly (warm pool) | yes | -| Fixes unverifiable | yes | yes | yes | -| Per-thread parallel work | yes | no | yes | -| New surface owned here | medium | none | large | -| Upstream dependency | none | high | none | -| Reversible | yes, projections rebuild | yes | hard | - -## 5. Recommendation - -**Option 1 as the spine, staged, with Option 2's cheap wins taken first -and Option 3 kept as the direction Option 1 already points at.** Nothing -in Option 1 forecloses Option 3; Option 3 is Option 1 plus resident agent -clients. - -**Stage 0 — stop the bleeding (days).** Dedupe the duplicate fetches. -Forward event payloads on the existing SSE stream and stop refetching -`pins`/`threads`/`messages` on every event. Land the failure containment -so a bad message can't kill a bench. These are independently valuable and -survive any later choice. - -**Stage 1 — the log becomes the read model (1-2 weeks).** Introduce -`@corbits/bench-log`. All chat writes append; the subscribe stream carries -data; chat tables become projections. Local echo lands here. - -**Stage 2 — ephemeral tier and streaming (1 week).** Typing, presence, and -token deltas as ephemeral kinds. Presence heartbeats leave Postgres. - -**Stage 3 — per-message runs and turn-taking (1-2 weeks).** Every message -dispatch carries a `run_id` on its event; adopt Buzz's queueing rule -generalized to threads — one in-flight prompt per (agent, thread). Threads -get real parallel work. Observability becomes a query. - -**Stage 4 — resident agents (later, optional).** Replace wake-per-message -with subscribed agent clients. Decide only once Stages 1-3 are in. - -Note what this ordering makes unnecessary: the bench-identity indirection -(~150 lines and a migration) is only needed to resurrect dead benches. If -a bench can't die, it never needs resurrecting. - -## 6. Open decisions - -- **Threads as parallel work** was the stated intent. That means a thread - needs turn-taking identity (Stage 3), not just a parent reference. Buzz - queues per channel; we'd queue per (agent, thread). Confirm that's the - semantic wanted: two agents can be mid-turn in two threads of one - workbench simultaneously. -- **Who hears what.** Buzz's answer — everyone subscribed, harness - serializes — is the recommendation. It costs delivering context to every - participant agent and inference for the one that speaks. -- **Does the log replace `session_mail` as truth, or shadow it?** The memo - assumes replace, with mail as dispatch. Shadowing is the drift risk. -- **Where per-thread grants land.** A thread with its own agent turn is - arguably its own authorization scope. Currently grants hang off the run. diff --git a/docs/revendor-inventory.md b/docs/revendor-inventory.md deleted file mode 100644 index 7f67bb5d15..0000000000 --- a/docs/revendor-inventory.md +++ /dev/null @@ -1,1235 +0,0 @@ -# CL-5514 re-vendor: pin delta, breakage, retirement - -> Superseded pin: CL-6254 bumped the vendored tree from `55c4431e` to -> `59f5e7b9` (upstream `main` tip, 2026-08-18). `VENDORED.md` is the current -> pin of record; every `55c4431e` reference below is historical. The bump -> added upstream's source-format workflow line and cost one contract -> adoption (`applyAtomic` now takes `gitDirs`); the retirement verdicts below -> still hold — nothing in it retires `packages/folded-runs`, -> `packages/agent-lifecycle`, or the self-anchoring mechanism. - -This has since landed for real: all five breakage items below are fixed on -`cl-5514-revendor`, the branch merged the `cl-5879-shell-mock-v2` shell/plugins -work forward onto the new pin, and `bun run check` plus a full walkthrough run -green against it. What follows is the historical record of what the re-vendor -pass found, kept as the rationale for the fixes and for the retirement -verdicts below — not a still-open remediation list. - -## Pin delta - -| | Commit | Date | -| ------------------------------------------ | ------------------------------------------ | -------------------------------------------------- | -| Old pin (`VENDORED.md` before this change) | `26ae23e8` | tip of upstream's now-merged `intr-358-...` branch | -| New pin | `55c4431e60cc97dae2f63bfd52de56166e42b13b` | upstream `main` tip, 2026-08-15 | - -95 commits landed on upstream `main` between the two pins. The old pin was -itself a branch-tip pin (see the removed paragraph in `VENDORED.md`) explicitly -waiting for `intr-358-dissolve-the-workflow_deployment-cache-and-re-anchor-onto` -to merge into `main`; it has, so this re-pin is a plain main-tip bump rather -than a branch re-target. - -Ground commits confirmed present on `main` since the old pin: - -- `e2732d32` "Collapse the workflow deployment onto one self-anchored run" — - the PR #130 change the ticket asked to verify. -- `db42bf3e` "Anchor every source-ref deployment with a workflow_run row" — - isolated-workflow anchoring (PR #123 territory). -- `a3b90e69` "Deliver credential material on the source-ref deploy frame", - `16a4a2d7` "Migrate the workflow-definition tables to content-hash identity" — - credential delivery / definition-identity work (PR #125-127/#132 territory). -- `b5c1525b` "Retire the folded agent-instance launch route", - `bb323d9a`/`cbe00ec2`/`b3ea566d`/`f9893011` — rename deployment/instance - identity to run-first, `anchor_run_id`. -- `ed1bc056`/`a8f44e98` — xAI Grok model catalog additions. -- Durable suspend/resume (PR #97-101 territory) landed **before** the old pin - — nothing new there since 26ae23e8; the closest post-pin work is the - source-ref body-ref pinning series (`55c4431e`, `6531c929`, `e1fffb32`, - `db42bf3e`, …), which extends the same suspend/resume machinery to - code-sourced (non-asset) workflow bodies. - -No newer PR beyond what the ticket named was found to be load-bearing for -workbench; the tail of the 95 commits is xAI catalog additions, run-first -renames, and the source-ref body-ref pinning series above. - -## Re-vendor mechanics - -All 20 `vendor/intx/*` packages were re-copied from upstream `packages/*` at -`55c4431e`, following the existing convention: `src/`, `package.json`, -`tsconfig.json`, `README.md`, `CONVENTIONS.md` (+ `migrations/`, -`drizzle.config.ts`, `tsconfig.config.json` for `db`) synced, then: - -- `package.json` exports repointed from the upstream `intx-src` condition to - direct source resolution (`types`/`default` → `./src/...`), `files: dist` - → `files: src`. -- `tsconfig.json` `extends` path adjusted for the shallower vendor tree - (`../../tsconfig.base.json` → `../tsconfig.base.json`). -- `tsconfig.base.json`'s `customConditions: ["intx-src"]` entry dropped. -- `harness` still drops the unvendored `@intx/inference-testing` - devDependency. -- `hub-api` still adds the `@types/ssri` devDependency and still excludes the - reserved `needs-you` segment from its `:approvalId` param routes - (`vendor/intx/hub-api/src/routes/approvals.ts`) — reapplied by hand since - the route file itself was re-copied wholesale from upstream. -- **New this pass:** `hub-agent`'s upstream `package.json` picked up a - `@intx/test-harness` devDependency (`tests/lib` upstream, not a `packages/*` - entry we vendor). Dropped it under the same precedent as - `harness`/`@intx/inference-testing`. This orphans - `vendor/intx/hub-agent/src/ws/hub-link.test.ts`'s import of - `@intx/test-harness` — see breakage below. - -`VENDORED.md` and `scripts/checks/kill-dates.txt` both updated: new pin, -renewed kill date (2026-09-14, +30 days from today), and recomputed sha256 -tree hashes so `check:killdates` passes clean. - -## Breakage inventory (one full `bun run typecheck`) - -Five distinct root causes, surfacing across 7 workspace packages: - -### 1. `WorkflowDefinitionSelector` now requires a wire hash, not just an asset id - -**Cause:** semantic. `16a4a2d7` ("Migrate the workflow-definition tables to -content-hash identity") changed `WorkflowDefinitionSelector` in -`vendor/intx/db/src/workflow-definition-store.ts:19-22` from a bare -`assetId: string` to `{ assetId: string; wireHash: string }` — "a single -asset backs many definitions distinguished by their wire hash." The vendored -helper `ensureWorkflowDefinitionForAsset` in -`vendor/intx/hub-sessions/src/workflow-definition-ensure.ts:34-37` now takes -that selector object instead of a bare string. - -**Breaks (all pass a bare asset-id string):** - -- `apps/hub/src/index.ts:1654` -- `packages/agent-directory/src/routes.ts:284` -- `packages/chat/src/platform-adapter.ts:340` - -**Cascades** (typecheck depends on the packages above) into -`@corbits/chat`, `@corbits/agent-directory`. - -**Fix shape:** semantic. Each of the 3 call sites needs to produce a -`wireHash` for the workflow JSON it just wrote (upstream's wire-definition -hash helper, promoted in `3514d6c4`, is the natural source) and pass -`{ assetId, wireHash }` instead of the bare id. - -### 2. `createIsogitStore` is no longer a top-level export — it's a bound method off a runtime-injected storage object - -**Cause:** semantic. `5f798bea`/`c3e160d8`/`70a81df6` restructured -`@intx/storage-isogit` so a filesystem `IsogitRuntime` is injected once via -`createIsogitStorage(runtime)` (`vendor/intx/storage-isogit/src/index.ts:59-96`), -which returns an object exposing `createIsogitStore` (and everything else) -bound to that runtime. There's no more bare `createIsogitStore` export; the -Node-side runtime is available via `createNodeIsogitRuntime()` / -`runtime` in `vendor/intx/storage-isogit/src/node.ts`. - -**Breaks:** - -- `apps/sidecar/src/conversation-state.ts:131,312` -- `apps/sidecar/src/workflow-substrate-factory/step-env.ts:13,256` - -**Fix shape:** mechanical-plus. Import `createNodeIsogitRuntime` from -`@intx/storage-isogit/node`, call `createIsogitStorage(createNodeIsogitRuntime())` -once, and call `.createIsogitStore(dir, signer, gcPolicy)` on the result at -both call sites. No new plumbing needed since a Node runtime is a stable -singleton — just an import + call-site rewrite. - -### 3. Spawning a suspendable child now requires a `referencedDefinitionHashes` map - -**Cause:** semantic. Part of the source-ref body-ref pinning series -(`db42bf3e` and its followers through `55c4431e`). -`WorkflowSpawnSuspendableChildOpts` in -`vendor/intx/workflow-host/src/adapters/spawn-child.ts:277-311` now requires -`referencedDefinitionHashes: Record` — "the parent's signed -deploy frame['s]... `SpawnTimeEnv.referencedDefinitionHashes`" — used to -re-verify a spawned child body's definition hash against what the parent's -deploy actually approved. - -**Breaks:** - -- `apps/sidecar/src/workflow-substrate-factory/index.ts:591` - (`createWorkflowSpawnSuspendableChild({...})` call, missing the new field) - -**Fix shape:** semantic. The sidecar's `SpawnTimeEnv` needs the -`referencedDefinitionHashes` map threaded from wherever the sidecar receives -its deploy frame at boot (mirrors how `deployRef` already gets there) down -into this call. Not a local one-line fix — needs the boot-time deploy frame -plumbing traced and extended. - -### 4. `hub-agent`'s dropped `@intx/test-harness` devDependency orphans a test import - -**Cause:** mechanical (vendoring-boundary fallout, not an upstream API -change). `vendor/intx/hub-agent/src/ws/hub-link.test.ts` imports -`@intx/test-harness`, which now only exists as `tests/lib` upstream (not a -`packages/*` entry, so not something this ledger vendors). Dropping the -devDependency (same precedent as `harness`/`@intx/inference-testing`) leaves -that one test file with a dangling import. - -**Not surfaced by `bun run typecheck`** at the app/package level because bun -in this repo doesn't typecheck vendored packages' own test files as part of -consumer builds — but `tsc -p vendor/intx/hub-agent` would fail on it -directly. Flagging it here since it's a direct consequence of this pass. - -**Fix shape:** mechanical. Either delete/adapt that one test (matching how -other retired-dependency test fallout gets handled), or extend the ledger to -also vendor `tests/lib` as `@intx/test-harness` if the test is worth keeping. - -## Retirement list verdicts - -Evidence gathered by a parallel research pass over both repos (workbench -`packages/`/`apps/`/`workflows/` vs. interchange commits since `26ae23e8`). -7 of the 14 tickets had no findable workaround code under the names/CL -numbers searched — flagged as NOT FOUND rather than guessed at. - -| Ticket / workaround | Verdict | Evidence | -| ------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Self-anchoring, `packages/folded-runs/launch.ts` + marker table + `reconnect.ts` | **CHANGED (confirmed, not retirable)** | Post-merge check against the landed `55c4431e` pin: `vendor/intx/hub-sessions/src/hub-session-lookups.ts:374` still gates `receiveWorkflowRunPack` on `anchor.anchorRunId === anchor.id` — the exact check self-anchoring exists to satisfy — so the mechanism itself is still required, not just the doc citation. The dead citation is fixed: `packages/folded-runs/src/launch.ts`'s header comment pointed at `vendor/intx/hub-api/src/routes/instances.ts` (deleted by `b5c1525b`); the route moved to `vendor/intx/hub-api/src/routes/runs.ts`, confirmed still self-anchoring at line 291/307/840, and the comment now cites that file with a note on the rename. The marker table (`packages/folded-runs/src/schema.ts`) and `reconnect.ts`'s `lookupFoldedRunReconnectKey` patch are both still needed — upstream has no per-chat-launch concept and its `isLiveWorkflowRunStatus` reconnect gate is unchanged since the old pin. Nothing here was deleted; this is a doc-accuracy fix only. | -| Chat write-claims (`packages/chat/src/write-claims.ts`, CL-6039) | **STILL-OPEN** | Redelivery-dedup via `tryClaim`/`release` (`write-claims.ts:24-49`) has no upstream equivalent; no dedup/idempotency commits touching this surface since the pin. | -| Tasks chain in-process hand-off (`packages/tasks/src/chain.ts:1-27`, CL-6059/6060) | **RETIRED** | `@corbits/tasks` was deleted (tasks primitive removed in favor of workflows/routines) — the citing code no longer exists in this repo. The upstream gaps it named (no `sidecar_allocation` row minted for folded runs, `buildRuntimeEnv` never binding `invokeAction`) may still be real for `@corbits/folded-runs`' own dispatch path; re-file against that package if a future workaround needs them. | -| Skills consumption (CL-6021) | **NOT FOUND** | No "CL-6021" or comparable marker in `skills/skills` / `tools/tools-skills`; likely workbench's own feature build, not a documented upstream-gap workaround. | -| Approvals scope always (CL-5956) | **NOT FOUND** | No "CL-5956", `approvalScope`, or "scope: always" workaround found anywhere in the worktree. | -| Asset delete (CL-6040) | **NOT FOUND** | No "CL-6040", `deleteAsset`, or asset package at all in the worktree. | -| Tenant-mint gating (`apps/hub/src/tenant-create-guard.ts`, CL-6041) | **STILL-OPEN** | Wraps the native `POST /api/tenants` because it's ungated; `git log 26ae23e8..origin/main -- packages/hub-api/src/routes/tenants.ts` upstream shows zero commits — route unchanged. | -| IDKind task (`packages/tasks/src/launcher.ts:264-269`, CL-6056) | **RETIRED** | `@corbits/tasks` was deleted (tasks primitive removed in favor of workflows/routines) — `mintTaskId()` and its citing code no longer exist in this repo. | -| `receiveWorkflowRunPack` anchors (`packages/folded-runs/src/launch.ts:240-247`, CL-6044) | **STILL-OPEN mechanism / CHANGED context** | Upstream `vendor/intx/hub-sessions/src/hub-session-lookups.ts:369-373` still requires `anchor.anchorRunId === anchor.id` before accepting a pack — same check the workaround targets. But per `e2732d32`'s commit message, the folded-launch surface it mimics (stop, mail send/history, turns, event stream) is being retired upstream in favor of run-first surfaces — the ground workbench stands on is shifting even though the specific check hasn't. **2026-08-28:** `hub-session-lookups.ts` is still untouched upstream at `a8bc06ae` (`git -C ~/abklabs/interchange log --oneline b5580a02..a8bc06ae -- packages/hub-sessions/src/hub-session-lookups.ts` is empty), so this row's verdict stands unchanged. | -| Mail-trigger anchor (CL-6020) | **NOT FOUND** | No "CL-6020" or comparable mail-trigger-anchor workaround found. | -| Abort race (CL-5960) | **NOT FOUND** | No "CL-5960" or abort/race workaround found. | -| Self-update grants (`packages/agent-directory/src/workflow-capability-routes.ts:16-40`, `tools/capability/src/client.ts:14-22`, CL-6085) | **STILL-OPEN** | A run's own `kind: "workflow"` principal is never seeded a `workflow-definition:/update` grant; upstream's `715c8be6` (removes unenforced approved-grant shipment) and `09d4cfd1` (freezes per-deployment grant walk) don't add own-definition self-update grant materialization — unrelated to this gap. | -| Tool-id binding (CL-6034) | **NOT FOUND** | No "CL-6034" or tool-id-binding workaround found. | -| Audit API (CL-6025) | **NOT FOUND** | No "CL-6025" or dedicated audit-log workaround found; only generic, unrelated `audit` string hits in logging/telemetry code. | - -## Open upstream gaps patched in the vendored tree - -Fixes carried locally in `vendor/intx/*` because no published `@intx/*` covers -them. Each one is a candidate to drop on the next re-pin — check the upstream -code first, then delete the local delta and its note in `VENDORED.md`. - -### Terminal anchors could not consume their own in-flight mail - -`vendor/intx/hub-sessions/src/hub-session-lookups.ts` — -`receiveWorkflowRunPack` - -Upstream gated the anchor lookup on `liveWorkflowRunStatuses` -(`deployed | running`), so once a run went terminal the hub refused every -pack pushed from that run's own address with `path_violation`. Two writes a -terminal run legitimately still owes both live behind that gate: - -- `enqueueInbox` for mail that arrived during the teardown window, and -- `markConsumed` carrying the `workflow_run_terminal` rejection that retires - such mail (`workflow-host/src/supervisor/supervisor.ts:2061`). - -Neither can land, so the sidecar withholds the ack, the hub redelivers, and -the pair spins forever — observed as this trio repeating every ~10s against a -finished workbench run: - -``` -WRN hub·lookups: Workflow-run pack rejected for : source address has no live deployment anchor -WRN workflow-host·supervisor: rejecting inbound mail : workflow run is terminal -ERR workflow-host·supervisor: dispatch loop iteration failed: failed to markConsumed for run -``` - -**Local fix:** the live-status filter is gone; ownership is the exported pure -helper `ownsWorkflowRunRepo` — a self-anchored `workflow_run` row with a -routable address. The `sidecar_allocation` generation/ownership fences below -it are untouched, so who may write is unchanged; only _when_ relaxes. -Accepting a post-terminal pack is safe because the substrate's per-commit -walk yields no new terminal events for an already-settled run. Covered by -`vendor/intx/hub-sessions/src/hub-session-lookups.test.ts`. - -**Retire when** upstream either accepts consume-only writes from a terminal -anchor, or stops delivering mail to a terminal run instead of redelivering it. - -### An event-only mail killed the run it was delivered to (CL-6164) - -`vendor/intx/workflow-host/src/supervisor/supervisor.ts` — the dispatch loop's -`signal.deliver` branch - -Attachments-only mail is legitimate on the bus: `@corbits/chat`'s `encodeParts` -leaves `content` empty for an event-only send, so a `workbench.agent-joined` -post is a `conversation.message` whose `text/plain` part is empty and whose -payload is a JSON attachment. `extractConversationText` returns `""` for it, -and upstream delivered that `""` as the resume payload. Downstream, -`step-invoker.ts` hands it to `agent.send`, which throws -(`createInboundMessage: content, when provided, must be a non-empty string`) — -surfacing as `StepFailed` with `retriesExhausted`, then `RunFailed`. - -Workbench already knew this hazard and had closed the **turn-1** door: -`packages/chat/src/platform-adapter.ts`'s `WORKBENCH_HOST_STEP_INPUT` pins the -workbench-host step's input to a literal so `trigger.payload` can never carry -an empty body into `agent.send`. The **turn-2+** door was still open, because -the parked-resume path never consults the step's `input` selector — the -supervisor extracts the text from the mail itself. Observed shape, a workbench -anchor dead 676ms after start because someone joined its bench: - -``` -StepStarted input: inline:"workbench-host anchor turn" -SignalAwaited __signal__:corr-1-... parkKind: input -SignalReceived payload: "" -StepFailed createInboundMessage: `content`, when provided, must be a non-empty string retriesExhausted -RunFailed one or more steps failed -``` - -Every later chat message then mailed a terminal run, which is what produced the -`path_violation` redelivery loop above — the two gaps compound. - -**Local fix:** mail whose extracted body has no text is dropped and consumed -with an `empty_conversation_content` rejection, exactly as the adjacent -malformed-mail branch already does, instead of being delivered. The gate is the -pure helper `hasConversationText` in `workflow-host/src/conversation-text.ts`, -covered by `conversation-text.test.ts` against a fixture built to `encodeParts`' -real event-only shape. - -**Retire when** upstream drops or otherwise guards empty-body mail on the -resume path. Two adjacent issues this surfaced, neither patched here: the same -mail was delivered twice under one `signalId` 110ms apart (a single mail can -resume a parked run twice), and `run-child.ts`'s turn-1 `resolveTriggerPayload` -still has the unguarded extraction — harmless for workbench only because of the -pinned literal above, but live for any workflow using the default selector. -Worth reporting alongside the adjacent question this surfaced: an anchor run -whose status flips terminal while its deployment stays mounted and its -workbench stays the active chat surface is what generates the in-flight mail -in the first place. - -**2026-08-28: RETIRED at the `a8bc06ae` re-pin.** Upstream `81ef5ad9` -("Deliver decoded inbound mail to deployed workflows") decodes each inbound -message into a `Mail` and commits its parts before dispatch instead of -extracting `text/plain` alone, so an attachments-only, empty-text mail is no -longer collapsed to `""` on the resume path — the local `hasConversationText` -guard has no upstream gap left to patch. Re-vendored on this branch in -`84a2e6cb` ("Re-vendor @intx/workflow-host at a8bc06ae; vendor harness and -hub-agent; adopt the warm-agent mailbox in the sidecar"). - -**On the 7 NOT FOUND items:** the workaround may live under a name not -guessed here, may not have landed yet, or the ticket may describe a gap with -no code-side mitigation to retire. Recommend re-running this check against -the actual Linear ticket bodies before concluding they're stale — this pass -searched by inferred keyword/CL-number, not by reading each ticket. - -### A failed body occurrence permanently ended a long-lived onTrigger section (CL-6326) - -`vendor/intx/workflow/src/runtime/run.ts` — `runOnTrigger`'s steady-state -drive loop and its crash-recovery counterpart, `planOnTriggerResume` - -Both treat a body run that ends `failed` identically to one that ends -`cancelled`: either throws terminal-is-final, ending the whole section run. -For a long-lived, event-driven section (chat-style: one occurrence per -inbound message), that means a single bad turn — one agent step throwing — -permanently ends the entire conversation's run, with no way to keep servicing -later occurrences. - -**Local fix:** a `BodyFailurePolicy` (`"end" | "continue"`) field, -`onTrigger.onBodyFailure`, defaulting to `"end"` (current behavior, -byte-compatible). `"continue"` lets the section re-arm on its input park -after a `failed` occurrence instead of throwing; `cancelled` is untouched and -always still ends the section — cancellation reflects a drain/operator -decision, not a turn-level error. The failed occurrence stays durably -recorded and loudly observable: the existing `ChildCompleted{terminalStatus: -"failed"}` commit lands on the run's own audit log before the policy check -runs either way, so a `"continue"`-policy failure is never silent, only -non-fatal to the section. Covered by -`vendor/intx/workflow/src/runtime/run.test.ts` — the intended home -`runlocal/run-local.ts:8` already names for this layer's coverage. - -**This delta targets the current pin** (`59f5e7b9`). Upstream's `486a6b1b` -changed `run.ts` after this pin; the delta above re-diffs against the -re-pinned tree once PR #59 lands. - -**Retire when** upstream ships a non-fatal body-failure edge on `onTrigger` -(or the whole source-format workflow line reaches a published `@intx/*` npm -version, per `vendor/intx/workflow`'s broader retirement condition in -`VENDORED.md`). - -**2026-08-28: RETIRED at the `a8bc06ae` re-pin.** Upstream `b977ade6` ("Add a -tolerate body-failure policy to onTrigger sections") ships the same edge -under `onBodyFailure: "tolerate"` (default `"end"`, a cancelled body always -terminates), so the local `"continue"` delta has no upstream gap left to -patch. The literal was renamed `"continue"` → `"tolerate"` at the authoring -site (`@corbits/agent-runtime`), with `@intx/db` migration `0088` rewriting -the retired literal inside stored wire projections. Re-vendored on this -branch in `7110d93a` ("Re-vendor @intx/workflow and @intx/workflow-deploy at -a8bc06ae with no delta"). - -## Estimated fix effort - -| Item | Shape | Rough size | -| ---------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `WorkflowDefinitionSelector` wire-hash threading (3 call sites + cascading typecheck in 5 more packages) | Semantic | Half day — need to source/compute a wire hash at each of 3 sites; the cascading packages need no code changes, just the root fixed. | -| `createIsogitStore` → `createIsogitStorage().createIsogitStore` | Mechanical-plus | 1-2 hours — 2 files, straightforward import + call-site rewrite. | -| `referencedDefinitionHashes` threading through sidecar's `SpawnTimeEnv` | Semantic | 1-2 days — needs the sidecar's boot-time deploy-frame plumbing traced and extended; the riskiest item here since it touches trust-sensitive spawn verification. | -| `hub-agent` test-harness fallout | Mechanical | Under an hour — delete or adapt one test file. | -| **Total to a green `bun run check`** | | **~2-3 days**, dominated by the `referencedDefinitionHashes` plumbing. | -| Retirement follow-through (closing STILL-OPEN items is out of scope here; CHANGED items need their workaround comments/reference-file citations refreshed) | — | Separate ticket-by-ticket work, not part of this dry run. | - -This pass reduces vendored _code volume_ only modestly (no upstream package -was fully retired this cycle — the un-vendoring checklist in `VENDORED.md` -still applies to all 20 rows), but it does close the gap to the run-first / -self-anchored-run model the workbench-side workarounds were built against, -which is where the eventual retirement payoff sits once the STILL-OPEN items -above get their own passes. - -## CL-6325: `invokeAction` bind (action-handler registry) - -Partially closes the "Tasks chain in-process hand-off" STILL-OPEN row above: -`buildRuntimeEnv never binds invokeAction` is still true as of this commit -(the run-child wiring lands in a separate, clearly-marked follow-up commit — -see below), but the two adapters that wiring needs now exist. - -**What landed:** - -- `vendor/intx/workflow-host/src/adapters/{action-invoker,effect-ledger,run-blobs}.ts` - (+ their tests), copied from gtm-workbench's `packages/workflow-host/src/ -adapters/`. Important provenance note carried over from the port scope - report: **these three files are gtm-OWNED, not upstream Interchange.** - Upstream `faremeter/interchange`'s own `packages/workflow-host/src/ -adapters/` (at the pinned `59f5e7b9`) has none of them — only blob- - substrate/repo-store/spawn-child/step-invoker. So there is no upstream - publish this sub-delta tracks toward; it is copy-with-attribution from a - sibling internal fork, not a normal "vendored until the next npm publish" - entry. The ordinary `VENDORED.md` kill date on the `vendor/intx/ -workflow-host` row does not really apply to this slice — treat it as - retired only once workbench builds its own equivalent inline (dropping the - gtm-origin copy) or gtm's fork itself gets folded into a real publish, - whichever happens first. -- `apps/sidecar/src/action-tool-handler.ts` (+ tests) — the workbench-native - action-handler registry. Two deliberate departures from gtm's own - `apps/sidecar/src/action-tool-handler.ts`: - 1. gtm enumerates action steps by re-reading the deployed `workflow.json` - off disk. Workbench retired `workflow.json` as a host-read format in - favor of the resolved in-memory `WorkflowDefinition` the closure loader - hands the sidecar — this registry takes that object directly as an - argument instead. - 2. gtm dispatches through `step-tool-harness.ts`'s - `runDeterministicToolStep`, a deterministic-tool-dispatch harness - workbench has no analog of (workbench dispatches tools only inside an - agent reactor). This registry instead materializes a step's tool - closure via the existing `materializeStepTools`/`attachStepCredentials` - seam and dispatches the named tool through the materialized - `ToolBundle.run` directly, all still routed through `ctx.perform` for - the capability/ledger floor. - -**Follow-up landed (CL-6325, second pass):** the run-child wiring described -below is now in place. `RunWorkflowChildBindings.resolveActionHandler` is -awaited once per child (post re-verify, with the resolved definition and the -live `CredentialWiring`), `loopFns` rides beside it, both default to the -fail-closed empty registries, and `buildRuntimeEnv` binds `effects`, -`invokeAction`, `loopFns`, and `runLoopIteration` into every run's env -(exported for the runtime-env probe in -`apps/sidecar/test/action-runtime-env.test.ts`). The sidecar's -`workflow-substrate-factory` builds the registry over -`createActionToolHandlerRegistry` with a stable per-action-step scratch root -(`actionStepStorageRoot`) and passes it as `resolveActionHandler`. The -re-application warning stands: a `vendor/intx` re-pin regenerates -`run-child.ts` and drops this wiring. - -**What did NOT land in the first pass (deliberately, time-boxed):** the run-child wiring -(`resolveActionHandler` bindings field, `effects`/`invokeAction` construction -in `buildRuntimeEnv`, `loopFns`) is a separate final commit on this branch, -explicitly marked in its message as needing re-application after the -in-flight `vendor/intx` re-pin lands (a sibling change was re-pinning -concurrently with this port) — `buildRuntimeEnv` is exactly the kind of -generated/copied vendor surface a re-pin silently regenerates, so the wiring -diff would otherwise be dropped on the floor. The app-seam wiring in -`apps/sidecar/src/workflow-substrate-factory/index.ts` (building the registry -and passing it as `resolveActionHandler`) is likewise deferred to that same -follow-up, since it depends on the run-child binding field existing first. - -`run-blobs.ts` duplicates the write/read pattern `vendor/intx/workflow-host/ -src/adapters/blob-substrate.ts` already has inline (private `writeBlob`/ -`readBlob` helpers) rather than reconciling the two into one shared helper — -left as a known follow-up per the port scope report, not a blocker. - -**Extraction landed (CL-6435):** the three gtm-origin adapter files (+ tests) -left the vendor tree for the first-class package -`packages/workflow-host-actions` (`@corbits/workflow-host-actions`), retiring -the "no ordinary kill date applies" sub-delta above — the code is now -workbench-owned, transportable to any Interchange instance (deps: -`@intx/workflow`, `@intx/hub-sessions`, `@intx/types`, arktype), and the -effect-ledger envelope read gained an arktype parse at the disk boundary. -The vendored `run-child.ts` delta shrank to the bindings fields, the -once-per-child registry resolution, and `buildRuntimeEnv`'s import of the -package; the re-application warning after a `vendor/intx` re-pin still -stands for that residual bind, but no longer covers any adapter code. - -## CL-6324 re-pin: `59f5e7b9` → `4ed8baf4` (the workflow.json retirement) - -The vendored trees are re-copied at upstream `main` tip `4ed8baf4` -(2026-08-19, 45 commits on). `VENDORED.md` is the pin of record. This -section is the map of what the bump costs on the workbench side, because -the app-side conversion does **not** land with it. - -### What landed cleanly - -- All 21 `vendor/intx/*` rows re-copied. Only nine trees actually changed - (`hub-sessions`, `workflow-host`, `workflow`, `workflow-deploy`, `types`, - `hub-api`, `db`, `agent`, `hub-agent`); the other twelve are byte-identical - at both commits and were re-pinned so the ledger records one commit rather - than a mix (this also collapses `inference-catalog`'s separate `5d2aa94a` - pin). -- Every workbench-local delta re-applied unchanged — upstream subsumed none - of them, and none of the five files they touch was modified upstream in the - 45 commits: the `inference.usage` forward, `ownsWorkflowRunRepo`, - `hasConversationText`, and the `needs-you` approval-route carve-out. Their - tests pass. -- `packages/folded-runs`' `wrapHarnessAsSingleStepWorkflow` call moved onto - `buildSingleStepAgentDefinition`, which survives the deletion. - -### What the bump breaks, and why it is one migration - -Upstream retired the on-disk `workflow.json`. A deployed workflow's -definition is no longer serialized into the deploy tree and re-read by the -sidecar; it is evaluated from the deployment's own **source closure** and -re-verified in-child against the approved wire hash. Source-ref is now the -only deploy lineage, and the live-authored and instance chains are deleted: -`createWorkflowDeployOrchestrator`, `SessionService.deploySingleStepAtHead`, -`SessionService.deployInstanceAtHead`, `wrapHarnessAsSingleStepWorkflow`, -`createWorkflowSpawnChild`, `createWorkflowSpawnSuspendableChild`, -`loadVerifiedWorkflowDefinition`, and the `definition` field on the deploy -frame. `SpawnTimeEnv` drops `referencedDefinitionHashes` and gains -`closurePackageDir`; `RunWorkflowChildBindings` drops -`workflowDefinitionRepoId`. - -Workbench has no code-sourced deploy front. Every run it launches — chat, -routines, agent lifecycle — goes through `packages/folded-runs`' -`deployAtHead`, which synthesizes a single-step definition in memory from a -system prompt plus tool-package pins and hands it to `deploySingleStepAtHead`. -The new front (`deployWorkflowFromSource` / `installAndApproveWorkflowSource` -/ `deployPreparedCodeSourcedWorkflow`) takes a registry `name@range` pin or -an asset tarball and resolves a dependency closure from it. Converting means -giving a folded run a real source package, not renaming a call. - -That is why the remaining breakage cannot be split by tree: -`hub-sessions` (deploy front), `workflow-deploy` (orchestrator), `types` -(deploy frame), `workflow-host` (child definition load), `db` (frozen -approval bundle, migrations 0082/0083) and `hub-api` (run trigger) all move -together, and `apps/sidecar` reads the frame both sides write. Leaving any -one on the old pin leaves the frame contract split down the middle. - -Open conversion sites: - -| Site | What it needs | -| ------------------------------------------------------------------------------------------- | ----------------------------------------- | -| ~~`packages/folded-runs/src/launch.ts` (`deployAtHead`), `wake.ts`~~ | **Done** — see "Conversion step 2" below. | -| ~~`apps/sidecar/src/workflow-host-wiring/index.ts`, `asset-materialization.ts`~~ | **Done** — see "Conversion step 3" below. | -| ~~`apps/sidecar/src/workflow-substrate-factory/index.ts`, `child-runtime.ts`, `config.ts`~~ | **Done** — see "Conversion step 3" below. | -| ~~`apps/sidecar/src/workflow-deployment-record.ts`~~ | **Done** — see "Conversion step 3" below. | - -Upstream's own diff over the same span is the reference implementation: -`apps/sidecar/src/workflow-substrate-factory.ts` and -`workflow-host-wiring.ts` at `4ed8baf4` show every one of these conversions -against the same contracts, and `apps/sidecar`'s `VENDORED.md` row stays at -`59f5e7b9` until workbench's fork is reconciled with them. - -### Conversion step 1: `packages/agent-runtime` - -`packages/agent-runtime` holds the definition builder every workbench -agent run deploys. `AgentRuntimeConfig` is the arktype contract for -everything that differs per run — the mailbox it answers on, its system -prompt, its resolved inference chain, its tool-package pins, its -credential bindings — and the config's `mode` selects the shape: -`buildAgentRuntimeWorkflow` returns either the folded unbounded step or -the per-turn `onTrigger` section. Because the mode lives in the config, -the deploy front keeps one parameter set and no call site ever branches -on which shape it wants; `deployCodeSourcedWorkflow` already enumerates -inert `onTrigger` bodies on every deploy, so the section shape needs -nothing extra from the API. - -#### There is no out-of-band config channel — the config IS the bytes - -The obvious design, one static published package whose entry reads a -per-run config from its environment, does not work at this pin and -cannot be made to work by the sidecar. - -The approval probe and the run child each evaluate the entry module -independently, and the child refuses to run a definition whose recomputed -wire hash differs from the approved one -(`workflow-host/src/child/verified-definition-loader.ts`). The hashed -preimage (`workflow/src/live-inert-projector.ts`) covers the trigger -address, the agent's system prompt, its `(provider, model)` pairs, its -tool-package pins, and the definition's credential bindings — every field -of the config. So a config read from anywhere outside the closure's bytes -diverges between the two evaluations and fails closed. And there is -nowhere to read it from anyway: `WorkflowDefinitionSource` has no overlay -or params on any variant, `AgentDeployWorkflow` carries no config bag -(the code-sourced route builds `HarnessConfig` with an empty -`systemPrompt`, empty `tools`, empty `grants`), `SpawnTimeEnv` has no -config field, and `WorkflowProbeRequestFrame` carries no env at all — so -even a sidecar willing to inject one could not make the probe see it. - -`renderAgentRuntimeSourceTree` is the consequence: it renders a thin -per-run package — a `package.json` plus a four-line entry module that -pins `@corbits/agent-runtime` and calls `buildAgentRuntimeWorkflow` with -the run's config as a literal. All the behaviour stays in the one -versioned package; what varies per run is a JSON literal inside the -hashed bytes. A host commits that tree into a `workflow`-kind asset and -deploys `source.kind: "asset"`, `package.format: "source"`, `commitSha` — -the only source variant whose pin is cheap enough to mint per run, since -the registry and tarball variants each need a publish. - -Two in-tree prerequisites remain for any of this to execute, both already -on the conversion table above: nothing produces `CLOSURE_PACKAGE_DIR`, and -no `WorkflowProbeExecutor` is wired on the sidecar, so every probe -currently answers `workflow.probe.error`. Both are conversion step 2. - -#### Conversion step 2: `deployAtHead` is on the seam - -`deployAtHead` no longer synthesizes a definition. It renders the run's -config into a per-run `@corbits/agent-runtime` package, commits that tree -into the run's OWN `workflow`-kind definition asset on -`refs/heads/runs/`, and deploys the resulting `commitSha` through -`deployAdoptedWorkflowFromSource` — the adopting shared-capacity front, -the only one that accepts a pre-minted anchor row and threads a -`credentialCipher`. `wake.ts` takes the same path. The old -synthesize-in-memory branch is deleted, not gated. - -Reuse, not a second asset: one definition asset can back many runs (a -chat's workbench host, an invited agent's every launch), so each run gets -its own ref inside that asset rather than its own asset per deploy. The -pin is the `commitSha`, so the ref is bookkeeping. - -The step's input selector became the config's `mode`: `step` (with the -workbench host's optional `literalInput`, the CL-6164 pin) or `section` -with a per-turn timeout. The Phase 1.3 swap changes a caller's argument, -never a branch inside `deployAtHead`. Section mode authors -`onBodyFailure: "continue"`, which the vendored surface now carries -through the live→inert projection. - -##### What still blocks EXECUTION - -Deploying works at the type and call level; nothing has run it end to -end, because the two sidecar prerequisites are untouched: nothing -produces `CLOSURE_PACKAGE_DIR`, and no `WorkflowProbeExecutor` is wired, -so every probe still answers `workflow.probe.error`. The remaining -in-tree typecheck failures are exactly the sidecar rows in the table -above — `projection.definition` reads, `createWorkflowSpawnChild` / -`createWorkflowSpawnSuspendableChild`, `SpawnTimeEnv.referencedDefinitionHashes`, -and `RunWorkflowChildBindings.workflowDefinitionRepoId`. - -##### Defect surfaced by the conversion - -`renderAgentRuntimeSourceTree` parses the config before writing it, which -is the first time a folded run's credential bindings are validated -against the platform's `CredentialBinding` schema. `apps/hub`'s -`mcp-credential-bindings.ts` mints `handle: "mcp:"`, and -`ToolCredentialHandle` is `/^[a-z0-9][a-z0-9._-]*$/` — the colon is not -in it, so every MCP-pinned launch would now fail closed at render time. -Nothing caught this before because the in-memory definition was never -parsed. Either the handle shape changes here (and with it the -`env.credentials.resolve("mcp:")` key `@corbits/mcp-tools` uses) or -upstream widens the handle grammar; it is not fixed in this change. - -#### Conversion step 3: the sidecar is on closures - -The execution host is converted. A deploy no longer writes a definition -into the deploy tree and reads it back: it materializes the frame's frozen -closure, evaluates the pinned code, and runs that. The boot-time restore -replays the same pin through the same helper, so both paths reach the -runnable definition by one computation rather than two that can drift. - -What moved: - -- **Closure staging.** `workflow-host-wiring/closure-staging.ts` owns the - durable per-deployment source stores (plain-file and indexed-git, - siblings of the reclaimed instance dir so they survive a restart with no - re-delivery), the `assetId -> mount` resolution both paths derive from - the pin alone, and the apply. It is an injectable router dependency, so - a test stands in for fetch + SRI-verify + layout + evaluate without - publishing a package. -- **`CLOSURE_PACKAGE_DIR` exists.** It is threaded on the frozen substrate - env, so the run child evaluates the pinned code and re-verifies its own - projection against `DEFINITION_HASH` — which is now the hub's - `approvedWireHash`, never a sidecar recompute. A frame carrying no - approved hash fails closed rather than substituting one. -- **The probe answers.** `createWorkflowProbeExecutor` is wired at the boot - edge against a closure materializer rooted in the sidecar data dir, so - `workflow.probe.request` returns a real inert projection, its advisory - grant set, and its wire hash instead of `workflow.probe.error`. -- **Child spawns are in-memory.** `createWorkflowSpawnChild` / - `createWorkflowSpawnSuspendableChild` are gone. A rung lifts its inline - children to refs and serves grandchildren from that map, so no rung reads - a definition off disk at any depth. An onTrigger body's `sources.json` is - still staged (a body child is in-process and loses its env across a - restart); its definition is not. -- **The deployment record carries the pin.** `referencedDefinitionHashes` - is gone; `approvedWireHash` and `sourceRef` are required, so a record - that cannot be restored fails at the scan boundary rather than - half-restoring. -- **`WORKFLOW_DEFINITION_REPO_ID`/`_REF` are gone.** What survives is - `WORKFLOW_DEFINITION_ID`: identity for the run-authenticated - capabilities route a step tool calls, never a repo to read from. - -Four modules are near-verbatim copies of upstream's own sidecar at -`4ed8baf4` — the probe handler, the closure materializer, the closure -apply, and the inline source-asset delivery — plus `bin/workflow-probe-child`. -`VENDORED.md` records them and the two adaptations the fork's module layout -forced. - -##### What is still unproven - -`bun run typecheck` is green repo-wide and the sidecar, folded-runs, and -chat suites pass, but **no run has executed end to end on these rails**. -The remaining wire, in order: - -1. **The MCP credential-handle defect from step 2 still stands.** Any - MCP-pinned launch fails closed at render time (`mcp:` is not a - legal `ToolCredentialHandle`). It gates a real chat launch, not the - deploy path itself. -2. **Nothing has published or committed a real per-run source package - through the deploy.** Every test injects the closure materializer, so - the fetch/SRI/layout/evaluate path itself is exercised only by - upstream's own tests at the vendored pin, never against a - `renderAgentRuntimeSourceTree` output. -3. **The probe has never been driven by a hub.** The executor is wired and - typechecks; nothing has sent it a `workflow.probe.request` frame. -4. **The pinned tool-package arm is untouched, deliberately.** Upstream - went all-source-tools (`req.agent.toolFactories`); workbench's - `agent-runtime` pins tool packages instead, so the sidecar keeps - `materializeStepTools`. Whether the source-format deploy still stages a - `tool-packages-manifest.json` for those pins is the first thing an - end-to-end run will answer. - -## CL-6324 e2e proof: what a real boot found - -The stack's first real boot (scratch database, real signup, real Ollama) -walked the seed/launch path with nothing mocked. It got as far as a fully -seeded tenant with every default workflow deployed by source-ref, then -stopped at the folded launch. What it found, in the order it found it: - -1. **`@corbits/mcp-tools` shipped new `src/` under an unchanged version.** - PR #98's `mcp.` handle change edited `src/tool.ts` and left - `0.0.4` in place, which `assertToolPackagesFresh` refuses — the seed's - tool-registry publish failed before any workflow deployed. Fixed by the - bump the check asks for. -2. **The seed pushed the retired `workflow.json` envelope.** A workflow - asset now accepts only a source codebase declaring an - `interchange.workflow` entry (`workflow-kind.ts`), so every asset push - was rejected `path-violation`. `createGitWorkflowPusher` now renders the - serialized definition into that two-file form. -3. **The seed deployed by bare `assetId`.** `POST /workflows/deployments` - takes the code-sourced pair — a `source` with - `package: { format: "source", commitSha }` plus the declared `entry`. - The pusher reports the sha it left on `main` and the deploy pins it. -4. **STILL OPEN — a folded launch reads its body from `workflow.json`.** - `packages/folded-runs/src/definition.ts`'s `readDefinitionJSON` reads - `WORKFLOW_JSON_PATH` out of the definition asset, which a source-format - asset does not carry, so minting a chat 409s `not_launchable`. The same - read appears four more times in `packages/agent-directory/src/routes.ts`. - This one is not mechanical: under the retirement a definition's body is - whatever its closure evaluates to, and nothing hub-side persists that - evaluated projection for a shared deploy — `workflow_definition` holds - only the wire hash and the manifests, and `workflow_run_launch_spec`'s - frozen bundle covers exclusive placement only. Deciding where a folded - launch body comes from is the next blocker, and it blocks the RunStarted - milestone behind it. -5. **STILL OPEN — no `deploy/tool-packages-manifest.json` is staged.** The - source-ref front (`deployAdoptedCodeSourcedWorkflow` → - `emitSourceRefDeployFrame`) never calls `executeLaunchPhases`, so - nothing runs `agentRepoStore.writeDeployTree` and no deploy tree lands - for the step. `materializeStepTools` reads its manifest off that tree, - so a folded run's pinned tool packages would materialize empty — the - prompt moved into the rendered bytes, the tool manifest did not. - `stageWorkflowStep` is the seam that still writes one; wiring it into - `deployAtHead` is the shape of the fix, unproven until (4) clears. - -## CL-6324: the persisted projection, and what the second real boot found - -The design blocker from the run above is closed. The ruling — the hub -persists a definition's evaluated inert projection, stored WITH the -definition, keyed to the approved wire hash — landed at the **freeze** -rather than at the deploy front, because that is where the projection and -the hash it is addressed by are already written in one transaction: -`createDbFrozenApprovalWriter` now stamps a `wire_projection` jsonb -column onto `workflow_definition_version` beside `approved_wire_hash` and -`grant_snapshot`, and `loadFrozenWireProjection` reads it back validated -as a `WorkflowProjectionDefinition`. Both code-sourced deploy fronts -consume `args.approved.projection`, which IS that value, so nothing can -drift between what the hub stores and what the sidecar re-verifies. - -`packages/folded-runs/src/definition.ts` is now the single reader of that -projection: `readDefinitionProjection` (one definition), -`resolveNewestProjectedDefinition` (the newest sibling that actually -carries one — the DB-side successor to CL-6357's asset-drift walk), and -`readFoldedBody(projection, grantRequirements)`. A pre-cutover row with -no stored projection fails as the named `DefinitionProjectionMissingError` -carrying re-deploy guidance, mapped to a 4xx at every chat route -boundary, never a raw 500. - -Two shape facts the conversion turned up, both load-bearing: - -- The inert projector **renames and flattens** the agent's inference - chain: `agent.inference.sources` becomes a top-level - `agent.modelSources: { provider, model }[]`. A reader written against - the live shape silently sees no step primitive at all. -- The projector **drops `grantRequirements` entirely** — it is not in the - projection and therefore not in the wire hash. Its hub-side home is the - `workflow_definition.grant_requirements` column, so the folded body - reader takes it as a second argument rather than reading it off the - projection. - -One live-shape reader survives, deliberately: `readLiveFoldedBody`, for -the workbench host, which builds its definition in process -(`buildWorkbenchHostWorkflow`) and launches it in the same breath without -ever round-tripping through a deploy freeze. - -### What the second real boot found - -1. **The rendered per-run tree could never resolve its own dependency.** - `renderAgentRuntimeSourceTree` pinned `@corbits/agent-runtime` at - `workspace:*`, but an asset tree is a standalone codebase with no - workspace root, so `resolveSourceWorkflowClosure` rejected every - folded deploy outright. Fixed by rendering the tree the way the seed's - default workflows already render theirs (`renderWorkflowSourceTree`): - the hub evaluates `buildAgentRuntimeWorkflow` at render time and - writes the definition out as a JSON literal, so the closure is the two - files and nothing else. The config-IS-the-bytes property is unchanged. -2. **The per-run tree lives on a per-run ref, and the pack shipped the - default one.** A folded run commits its source into the shared - definition asset on `refs/heads/runs/`; - `bindAssetAttachmentResolver` packed `DEFAULT_ASSET_REF`, so the - sidecar got a history the pinned commit was not reachable from and - failed "could not find ". `DeployWorkflowFromSourceParams` now - carries an optional `sourceRef` (a ledgered vendored delta) that - `deployAtHead` sets; omitted, the default ref is packed exactly as - upstream. -3. **The tool manifest is now staged.** `deployAtHead` calls - `stageWorkflowStep` before the deploy frame, writing the step's - `deploy/tool-packages-manifest.json` where `materializeStepTools` - reads it. This is workbench's deliberate divergence: upstream's - source-ref front stages no per-step tree at all, but the sidecar's - tool loader still reads pins off one. With it wired, a real turn comes - back naming its own pinned tools. -4. **STILL OPEN — a folded `step`-mode run publishes no `RunStarted`.** - With everything above in place a real boot reaches: seed fully green - by source-ref, chat minted, probe answered, closure materialized for - real on the sidecar (`materialized workflow-probe closure for -folded-run-`), definition loaded from that closure, run grants - written into the workflow-run repo, the anchor row flipped - `deployed` → `running`, and a real Ollama reply to a real message. But - `GET /workflows//runs//events` stays empty: the folded - conversational shape is one unbounded step servicing every inbound - mail, and its per-message bracket is the `message.run.started` AGENT - event (`packages/agent-events/src/agent-events.ts`), not a - workflow-host `RunStarted` in the run's durable event log. The - milestone's RunStarted assertion is therefore an assertion about - CL-6329's `section` mode — where every message is an `onTrigger` - occurrence with its own child run id and event log — not about the - `step` mode every launcher deploys today. Proving it means either - flipping the default mode or asserting the section shape directly; - `scripts/e2e/cl-6324-launch-proof.ts` keeps the assertion as written - rather than weakening it to something the current shape happens to - satisfy. -5. **CLOSED — the agent-directory authoring lineage now writes the source - form.** `createAgentDefinitionCore` and the read/modify/write routes in - `routes.ts`, `workflow-capability-routes.ts`, and - `workflow-skill-pin-routes.ts` used to populate a `workflow`-kind asset - with a bare `workflow.json`, which `workflowKindHandler.validatePush` - now refuses. That lineage is authoring, not launching — a projection is - a read-only artefact and cannot be written back through — so it got its - own cutover. See the section below. - -## CL-6324: the agent-directory authoring cutover - -The renderer that had been living in `@workbench/hub-client`'s -`workflow-push.ts`, and a second copy of it in `@corbits/agent-runtime`'s -`source-tree.ts`, moved into a new dependency-free package, -`@corbits/workflow-source` (since folded into `@corbits/workflows`'s -`./source`, CL-7373). Every authoring path in the repo now writes -its asset tree through that one `renderWorkflowSourceTree` — the seed -pusher, the per-run agent-runtime package, and the agent-directory -lineage — so there is a single producer of the bytes a workflow-kind -asset carries. - -`@corbits/agent-directory`'s `definition-asset.ts` is the lineage's own -seam onto it: `agentDefinitionSourceTree` renders a definition's tree -under a `@workbench-agent/` package name, and -`readAgentDefinitionWorkflowJson` reads the definition back out of the -entry module. Both replace the private `AGENT_DEFINITION_ASSET_PATH = -"workflow.json"` constant each of the four writers used to declare for -itself; `apps/hub`'s planner deploy and `@corbits/evals`' world snapshot -route through the same two functions rather than re-deriving the path. -The `workflow.json` write path is gone, not kept beside the new one. - -Reading the definition back is a strict slice of the exact bytes the -renderer emits (the `export default ` prefix and `;\n` suffix), never an -eval and never a pattern search. Anything else — in practice, an asset -last written before this cutover, whose tree still holds a bare -`workflow.json` — throws `RetiredWorkflowEnvelopeError`, defined in -`@corbits/workflow-source` (now `@corbits/workflows`'s `./source`) and -re-exported from `@corbits/agent-directory`. -It carries re-author-and-re-deploy guidance and is mapped to a 409 in -every route module that can reach it: `routes.ts`, -`workflow-capability-routes.ts`, and `workflow-skill-pin-routes.ts` each -answer it from their own `app.onError`, so a stale asset never reads as a -server fault. - -One behavioural change rode along: `POST /:definitionId/restore` now -reports the definition it just wrote, parsed from the entry module it -restored, instead of re-reading the asset immediately afterward. - -## CL-6324: what the four proofs found on a real stack - -`scripts/e2e/cl-6324-launch-proof.ts` is the harness: one scratch -database, a real signup, a real local Ollama, nothing mocked. It runs as - -``` -E2E_PROVIDER=ollama OLLAMA_BASE_URL=http://localhost:11434 \ -E2E_OLLAMA_MODEL= \ -DATABASE_URL=postgres://localhost:5432/ \ -bun run scripts/e2e/cl-6324-launch-proof.ts -``` - -### Proof 2, in the shape each deploy actually takes - -The earlier revision asserted a workflow-host `RunStarted` for a folded -`step`-mode run and hung on it forever. That was an assertion about the -section shape aimed at the step shape's run. The two are now asserted -separately, each against the artefact it really produces. - -**Step mode.** One unbounded step services every inbound mail, so the run -starts no child run per message and its own workflow event log stays -empty — the harness asserts exactly that, as a falsifiable statement -rather than a footnote. Its per-message bracket is the -`message.run.started` / `message.run.ended` AGENT event pair, which -travels the sidecar's `agent.event` channel and is never committed to the -workflow-run repo. The durable, HTTP-observable evidence that the bracket -opened AND closed is `@corbits/insights`' `turn_latency` row: the tracker -opens it on `message.run.started` and commits it on `message.run.ended`, -and `GET /api/tenants/:tenantId/insights/latency` reports it as a sample -count. That, plus the reply row itself, is what the proof asserts. - -**Section mode — and CL-6329's first live validation.** `mode: "section"` -had existed as a config argument since the agent-runtime cutover and -nothing had ever deployed or run one. The proof now does, for real: it -renders a section-mode `AgentRuntimeConfig` into its own source package, -pushes it as a `workflow`-kind asset, deploys it by source-ref through -`POST /workflows/deployments`, and drives it with real mail. Every -message is an `onTrigger` occurrence with its own child run and its own -event log, which is where `RunStarted` genuinely lives: - -``` -turn__0 events: RunStarted, StepStarted, StepCompleted, RunCompleted -parent run events: RunStarted, StepStarted, ChildSpawned, - ChildCompleted, SignalAwaited -``` - -The parent's `SignalAwaited` after `ChildCompleted` is the section -re-arming for the next message. This is the shape the milestone's -`RunStarted` assertion was always about, and it works. - -### Proof 4 FAILS, in both shapes, for the same reason - -Kill the sidecar mid-turn, restart it, and boot restore does its half of -the job: the scan finds every deployment record, replays each pin, and -re-materializes the closure — the restarted stack reports the run's own -`liveness: "ok"` 9.2s after the restart. But the run inside it comes back -**terminal**, and every later message is refused: - -- the folded chat run: `workflow-host·supervisor: rejecting inbound mail -'<...>': workflow run 'run_...' is terminal` — the hub accepts the room - message, nothing ever answers it, and the turn times out with no reply - and no notice; -- the section deployment: `POST /workflows//mail` answers `409 -workflow_run_terminal`, "is terminal and cannot receive more mail". - -So a mid-turn sidecar death is permanent for the run. `onBodyFailure: -"continue"` does not rescue the section here, and could not: the failure -is on the TOP-LEVEL run, not on a body occurrence, so there is no failed -child edge for the policy to act on. Nothing in the chat layer recovers -either, because its wake choke point only redeploys an address that is -NOT routable — and this address is routable, just dead. - -This is the milestone's real remaining gap, and it is a platform-level -one rather than anything the app-side conversion introduced: restoring a -deployment is not the same as resurrecting its run, and at this pin -nothing does the second half. - -### Timings, from a clean boot - -| Phase | Wall clock | -| ---------------------------------------------------- | ---------- | -| section-mode deploy (push + probe + freeze + deploy) | 4.3s | -| section occurrence to its own `RunStarted` | 1.0s | -| sidecar restart + boot restore to `liveness: "ok"` | 9.2s | - -Two phases read as ~0s and should not be mistaken for speed: a chat mint -is DB-only (the deploy happens later, on the first traffic), and the join -greeting is canned copy `@corbits/chat` posts in the agent's voice, not a -model turn. The first REAL token is proof 3's reply. - -### Two environment traps the harness now closes explicitly - -1. **The onboarding seed pins the curated model; the catalog is seeded - from the live instance.** `modelSourceFor` takes its model from - `CATALOG_SEEDS.ollama`, so every default workflow deploys pinned at - that name, while `seedCatalog` fills the tenant catalog from the - instance's own `/api/tags`. On an instance that never pulled the - curated model the two never meet, and every chat turn dies as - `InferenceResolutionError: No launchable inference source for model -""` — surfaced to the reader as "I can't reach a model right - now". The harness names its model through `E2E_OLLAMA_MODEL` instead - of inheriting the curated one. -2. **A live Ollama connect seeds embedding models as offerings with no - capability metadata** (CL-6351), so default-model resolution breaks - the tie alphabetically and `all-minilm` can win the bench default. The - harness narrows the bench catalog to its one pinned model through the - catalog API before it asserts anything about a turn. - -## CL-6365: why the restored run comes back terminal - -Reproduced on a second real stack (scratch database, real signup, real -Ollama), with one correction to the harness first. - -### The kill has to be mid-turn, and it was not - -Proof 4's kill window was three seconds after "count slowly from one to -twenty". On an instance whose model answers that in about one second, -the turn had already completed and the run had parked by the time the -sidecar came down — so the restart that followed was a clean restart, -not a crash, and proof 4 passed. It passes that way at this branch's -tip; the whole finding turns on the kill actually landing while -inference is running. - -Proof 4 now sends a workload no model finishes inside the window and -asserts, at the moment of the kill, that nothing has answered yet. -"Mid-turn" is a claim the harness checks rather than a sleep it hopes -for. With that in place the failure is deterministic. - -### What the mid-turn kill leaves behind - -Both top-level runs — the folded chat run and the section deployment — -end up `workflow_run.status = 'failed'` in the hub's own database. The -sidecar's SIGTERM drain tears the workflow-process child down while the -step is in flight, and the resulting terminal event is committed to the -run's durable event log before the process exits. - -Boot restore then does its half correctly: the scan finds the -deployment records, replays each pin, re-materializes the closure, and -the address is routable again (`liveness: "ok"` about ten seconds after -the restart). But the run inside it is over: - -- the section deployment answers `POST /workflows/:id/mail` with `409 -workflow_run_terminal`; -- the folded chat run's mail is dequeued and dropped by - `vendor/intx/workflow-host/src/supervisor/supervisor.ts`, which finds - no in-memory cohort for the run, reads - `readWorkflowRunLifecycle` off the durable log, sees `terminal`, and - rejects permanently. - -`onBodyFailure: "continue"` cannot rescue either shape: the failure is -on the TOP-LEVEL run, not on a body occurrence. - -### Why "wake it again" is not the fix by itself - -`@corbits/chat`'s `wakeByAddress` returns early for any routable -address (CL-6267 handed respawn of a parked-but-announced deployment to -the sidecar's own park/wake handler), so a routable-but-dead run is -never woken. Restoring the deleted CL-6147 branch — undeploy the -resident, then redeploy — is necessary but not sufficient, for two -reasons found while tracing it: - -1. A run's id is derived from its address - (`deriveWorkflowRunId(deploymentMailAddress)`), so redeploying the - same address re-creates the _same_ run id. -2. The destructive teardown (`teardownDeployment` with `reclaimDirs`) - removes the deployment record and the per-step scratch, but NOT the - workflow-run repo that holds the run's durable event log. The - terminal event survives the redeploy, and the supervisor rejects the - next message for exactly the same reason. - -So a relaunch has to produce a genuinely new run, not the same one -again — either a fresh run id adopting the room's continuity (the room -is data now, so no history is lost), or a destructive teardown that -also reclaims the run's durable log. Which of the two is right is the -open design decision; the detection signal it hangs off is already -available to the hub as `workflow_run.status`. - -The hub-side detection and the relaunch itself are not implemented -here. What is on the record is a deterministic red: the proof now fails -at "PROOF 4 — the section survives the restart and runs its next -occurrence" with `409 workflow_run_terminal`, every time. - -### The ruling: relaunch, and un-fuse the room from the run - -The open design decision above is settled in favour of the FRESH run. -The durable log is never reclaimed and never erased: it is the audit -trail, and a resurrection that overwrote it would trade the one thing -this shape is better at for a shortcut. A relaunch mints a new run, -which the platform gives a new address and therefore a new event-log -repo (`/workflow-runs//runs//`), -leaving the dead run's log intact and readable through the ordinary run -routes. - -That is only possible because the room stops being the run. Three -independent points in the platform fuse a run's id to its address — -`deriveWorkflowRunId(address)` is the address's local part, the -code-sourced deploy front refuses an `(anchorRunId, agentAddress)` pair -that does not name the same run -(`vendor/intx/hub-sessions/src/session-service.ts`'s coherence guard), -and `receiveWorkflowRunPack` marks runs terminal by an id read out of -the address-derived repo — so a fresh run CANNOT keep the old address. -Nothing can be done about that from here, and nothing needs to be: the -address only has to be stable for the ROOM, not for the sidecar. - -`chat.workbench_launch` is where the two come apart. `instance_id` is -now the stable participant id the room addresses forever — the -workbench id for a host, the first-mint id for an invited agent — and -`current_run_id` names the run executing behind it, re-pointed on every -relaunch. `packages/chat/src/agent-binding.ts` owns both directions: -outbound (`sendMail`, `ensureAwake`, `subscribeToWorkbench`) resolves -the stable id to the live address, and inbound -(`chat-orchestrator.ts`'s `resolveMemberWorkbenches`) resolves a live -address the sidecar reported back to the room address the participant -records, the mention handles, and every already-posted message's -`sender_address` carry. Nothing else in the room moves, because none of -the room's tables were ever keyed on the run — only on the id that is -now the stable one. - -Detection is `workflow_run.status` plus `@corbits/folded-runs`' -previously orphaned `isFoldedRunSettled`: a terminal status that is NOT -a folded run parked between messages is a run beyond waking, and -`wakeByAddress` relaunches it instead of redeploying an address whose -log already says terminal. - -### Closing it: the sweep, the notice, and the old attachments - -Three things stood between the relaunch and a reader actually -experiencing it. - -**The sweep.** The relaunch was send-triggered — it fired on the next -`sendMail` through the wake choke point — so a room whose agent died in -a crash stayed silently dead until somebody wrote into it, which is -precisely what nobody does when the room looks broken. -`createHubChatPlatform.sweepTerminalRuns` scans `workbench_launch` -(bounded at 100 rows), keeps the participants `isBeyondWake` says are -terminal-and-not-merely-parked, and relaunches each through the same -path a send would; every relaunch and every failure is logged under -`chat·wake`, and one failure never aborts the pass. - -The hub runs it at boot and re-arms it on `sidecar.disconnect`, as a -short bounded series of passes rather than a single one. That is not -belt-and-braces: "this run is dead" is not knowable at the instant the -execution plane comes back. The dying sidecar commits the terminal event -to the run's own durable log, and `workflow_run.status` only follows once -the RESTARTED sidecar has packed that log back to the hub — seconds -after the reconnect. A single pass at boot would look at a run still -marked `running` and find nothing. - -**The notice.** `chat-orchestrator.ts`'s turn-drop notice hangs off -`message.run.ended`, which a killed sidecar never sends — so the -interrupted turn had no way to surface at all. `relaunch-notice.ts` is -the other half: `createHubChatPlatform` takes a notice port (a ref, since -the adapter is built before the room-message store it posts through), the -hub arms it beside `roomMessages`, and every relaunch — swept or -send-triggered — posts into each room the replaced participant belongs -to, under the STABLE address the room has always known it by, so the -notice reads as the same teammate rather than a stranger. The wording is -cause-aware (crashed / stopped / ended) and stays in the reader's -language: the turn didn't finish, it's back now, say it again. - -**Pre-relaunch attachments.** `fetchBlob` read through the live run's -session, and a folded run's mail session is resolved from its PRINCIPAL -— a fresh run has a fresh principal, so every attachment sent before the -crash became unreachable. `workbench_launch.prior_run_ids` records each -run as it is retired (capped at 20), and `fetchBlob` walks those -sessions newest-first after the live one. Room messages were never -affected: they are `chat.workbench_messages` rows, not mail. - -**The audit assertion.** Proof 4 now ends by reading the replaced run's -events back through the ordinary run routes AFTER its replacement is -already answering. That is the fresh-run ruling's whole justification, -and it is now a hop rather than an argument. - -### And what re-running proof 4 found: the detection signal does not fire - -Proof 4 was re-run on a real stack (scratch database, real signup, real -Ollama, both shapes). Proofs 1, 2, 2b and 3 are green; proof 4's restart -and boot restore are green at 9.2s. Hop 3 — "the turn the kill -interrupted surfaces visibly" — is still red, and the sweep is not why. -Nothing in the sweep's log fired at all, because it found nothing to -sweep: - -``` -run_10dd09d1… failed <- the SECTION deployment -run_4de07624… running <- the folded CHAT run, after the mid-turn kill -``` - -The claim recorded above — that both top-level runs end up -`workflow_run.status = 'failed'` — holds for the section deployment -only. The folded chat run's terminal event goes to its durable log -(which is why its supervisor rejects the next message) while its -`workflow_run` row stays `running` forever. `isBeyondWake` reads that -row, so for the shape hop 3 measures it answers "still alive" and -neither the sweep nor the send-triggered relaunch ever fires. The room -keeps the reader's message with nothing after it — the exact silent -drop the hop asserts against. - -So the remaining gap is the DETECTION signal, not the relaunch: the hub -needs to learn a folded run's lifecycle from the same durable log the -supervisor reads (`readWorkflowRunLifecycle`) rather than from -`workflow_run.status`, either by reconciling the row when a restarted -sidecar packs a terminal log back, or by having `isBeyondWake` consult -the log directly. Everything downstream of that signal — sweep, notice, -repoint, attachment history, audit read — is built and unit-proven. - -The section shape stays red for its own separate reason: a plain -workflow deployment has no room, so nothing maps a stable id onto a -fresh run for it. That is out of CL-6365's scope, which is the room. - -### Correction: the row was not lying, the terminal event never arrived - -The account above named the wrong culprit. Reading the two sides of a -real crashed stack settles it. After the mid-turn kill and the restart, -the sidecar's own copy of the folded chat run's workflow-run repo held -three commits the hub had never seen: - -``` -9c14859 compact run run_4de07624… events <- sidecar only -2cea7c9 append workflow event RunFailed <- sidecar only -ae962fb append workflow event StepFailed <- sidecar only -34fe89a SignalReceived … <- the hub's tip -``` - -So the hub's `workflow_run` row was not disagreeing with the hub's copy -of the durable log — the hub's copy of the log stopped at the last -pre-crash event too. Reconciling the row against a received terminal -log, or having `isBeyondWake` read the log instead of the row, would -both have read the same "still live" answer. The signal was not -mis-read; it was never delivered. - -Why it is not delivered is an ordering race in the sidecar's boot edge. -`restoreWorkflowDeployments()` runs BEFORE `hubLink.connect()` — each -restored deployment's mailbox must be live before the hub can route to -it. A restored supervisor that finds a step which died mid-invocation -commits that run's `StepFailed`/`RunFailed` right then, and the -pack-pushing store schedules the push into a link that does not exist -yet. The sidecar log says it plainly: - -``` -11:17:33.480 Sidecar connecting to ws://…/api/sidecars/ws -11:17:33.507 Connected to hub -11:17:33.508 workflow-run pack push failed for deployment run_2c1e98ac…: 'Connection lost' -11:17:33.509 … × 8 more, every restored deployment -``` - -Those failures latch on their slot, and the only thing that re-ships -them is `notifyAddressRoutable`, fired when the reconnect challenge -passes. That recovery works when the rejection latches before the -challenge — one full re-run of proof 4 came back with the hub and the -sidecar at the identical tip, the row flipped to `failed`, the sweep -firing, and the room repointed onto a fresh run -(`prior_run_ids: ["run_8da929dd…"]`). It is lost when the rejection -lands after the challenge: nothing re-arms the slot, the terminal event -stays on sidecar disk for good, and the row says `running` for a run -whose supervisor rejects every further message. - -The fix holds rather than recovers. `createBootRestorePushHold` marks -every address the boot restore registers unroutable — the same block -the link already applies across a mid-life disconnect — and the -challenge lifts it, so no workflow-run push is ever attempted against a -link that has not been established. Both deploy shapes take the same -path, so both terminal signals now land on the hub at the same, stable -point in the boot sequence. - -One consequence worth stating rather than burying: with the signal -arriving reliably, proof 4's section hop now fails deterministically -with `409 workflow_run_terminal` instead of intermittently. That is the -section shape's own gap — a plain workflow deployment has no room, so -nothing maps a stable id onto a fresh run for it — surfacing on -schedule rather than racing the assertion. It stays out of CL-6365's -scope, which is the room. diff --git a/docs/seed-reconciliation.md b/docs/seed-reconciliation.md deleted file mode 100644 index 314aba6fd7..0000000000 --- a/docs/seed-reconciliation.md +++ /dev/null @@ -1,293 +0,0 @@ -# Seed reconciliation - -How workbench's automatic seeding converges on the shipped defaults -without ever fighting a member. Every seed pass — an onboarding run or -an explicit `@corbits/seeding` caller — must satisfy four properties. -Hub production boot is not a seed pass: it does not insert agents, -tools, workflows, or skills. - -1. **Idempotent restart** — re-running creates nothing twice. -2. **Content convergence** — a changed shipped default updates the - seeded row to the current version. -3. **Member edits win** — a row a member touched is never overwritten, - re-enabled, resurrected, or deleted by a later pass. -4. **Orphan handling** — a default that no longer ships is retired - (hidden or removed), never left dangling — unless a member made the - row theirs. - -Ambiguity always resolves toward property 3: when a pass cannot prove a -row is still seed-owned, it leaves the row alone. - -## Template library (first library read) - -`createTemplateLibrarySeeder` -(`packages/artifacts-hub/src/template-library.ts`) runs -`seedTemplateLibrary` for the tenant whose shelf is being read, on the -`GET /api/tenants/:id/library/templates` routes themselves. Each shipped -template manifest becomes one versioned artifact row (`kind: -workbench-template`, title = template id), and the seed records the -SHA-256 of the content it last wrote in the artifact's -`source.seededContentHash`. - -The trigger is the read, not a boot window, because template rows are -tenant-scoped: every bench owns its own shelf, so a bench created long -after the hub booted converges the first time its picker opens, in any -boot order. One pass per tenant per process, shared by concurrent first -reads; a failed pass is not remembered, so the next read retries rather -than the shelf staying empty until a restart. A read whose pass failed -answers 503 — never a 404, which would read as "no such template". - -- The marker distinguishes "the shipped manifest moved" (head content - still equals the marker: write a new version) from "a member edited - this" (head content diverged: keep, outcome `kept`). -- A template dropped from the manifests is archived with - `source.retired` when still seed-owned; re-adding it unarchives and - converges (`restored`). A member's own archive — no `retired` flag — - stays archived. -- Rows seeded before the marker existed adopt one when their content - still matches the shipped manifest, and are otherwise preserved. -- A member-created artifact sharing a template's title is never touched - and never duplicated. - -## Default scheduled workflows (onboarding) - -Seed never POSTs `/routines`. Native `ScheduleTrigger` ticks digest; -last-30-days-research stays a deployed workflow, not a wrapper row. -`scripts/db-setup.ts` drops a leftover `routines` schema after the -digest enablement handoff. There is no preset-wrapper prune: seed does -not plant wrapper rows. - -## Default vs. on-demand catalog workflows (CL-7074) - -`DEFAULT_WORKFLOWS` (`packages/seeding/src/seed.ts`) is the set every -real signup gets automatically: `assistant` (Myra), and nothing else. A -fresh bench used to also pay a git push and a sidecar probe for `echo`, -`workbench-digest`, and `last-30-days-research` — three workflows -nobody had asked for yet. Those three, plus every other -`workflows/` source package that exports a builder -(`code-review`, `granola-call`, `morning-brief`, `exa-topic-watch`, -`process-granola-call`, `attio-task-agent`, `pain-point-collateral`, -`reddit-opportunity-scanner`, `collateral-generation`), now live in -`CATALOG_WORKFLOWS`, same shape -(`DefinitionWithAgentSteps`-backed `DefaultWorkflow` entries) and -deployable through the catalog instantiate route (CL-7073), but never -automatically at signup. - -Registration is total, not partial: `packages/seeding/test/workflow-source-registration.test.ts` -asserts every directory under `workflows/` appears in exactly one of -`DEFAULT_WORKFLOWS`, `CATALOG_WORKFLOWS`, `CATALOG_TEST_WORKFLOWS`, or -the explicit `EXCLUDED_WORKFLOW_SOURCES` list (a one-line reason per -entry, currently empty — every source directory is registered -somewhere today). A new `workflows/` package that nobody wires up -fails that test instead of 404ing silently through the catalog -instantiate route, which is exactly the drift CL-7073's critique -caught: `CATALOG_WORKFLOWS` had four entries while a template's blocks -(the GTM template's, in particular) named source packages that were -never registered at all. - -There is no orphan-retire for an entry that moved from -`DEFAULT_WORKFLOWS` to `CATALOG_WORKFLOWS`: an asset already deployed -on an existing bench from before the move is left exactly as it is. -`CATALOG_TEST_WORKFLOWS` remains the separate, never-reaches-a-real- -signup set for workflows that exist only to exercise the platform -continuously. - -### Deployable through the catalog (CL-7073) - -`CATALOG_WORKFLOWS` is the one source of truth for which catalog asset -names have a source package under `workflows/` and can be -deployed on demand. Two callers reuse it, sharing the same `buildJson` -per entry rather than each hand-rolling a definition: - -- `seedTenant` (`workbench seed`, the first-login provisioning hook) - can deploy any of `CATALOG_WORKFLOWS` the same way it deploys - `DEFAULT_WORKFLOWS`, over its HTTP self-call path - (`ensureWorkflowAsset` → `pushWorkflow` → `ensureDeployment`). -- The hub's `POST /:assetName/deploy` template-block route - (`apps/hub/src/templates/template-block-routes.ts`, - `apps/hub/src/templates/block-workflows.ts`) deploys any - `CATALOG_WORKFLOWS` entry natively, in-process, against the tenant's - real inference preferences — the same route that already deployed - `code-review` for template instantiation, generalized (CL-7073) so - `code-review` is just another `CATALOG_WORKFLOWS` entry rather than a - hardcoded special case. Idempotent: a tenant that already carries a - deployed definition under that asset name answers with the existing - definition (`created: false`) rather than deploying a second time. - `assistant` (seeded already) and `heartbeat` (test-only, - `CATALOG_TEST_WORKFLOWS`) are never reachable through this route. - -## Default skills - -`plantDefaultSkills` (`packages/seeding/src/seed.ts`) plants each -`DEFAULT_SKILLS` entry through `POST /api/tenants/:id/skills`, after -first checking `GET /api/tenants/:id/skills/:name`. - -- An existing row the by-name GET finds is skipped outright. -- A `409` from the create call itself — a row the GET missed (an - inherited/other-scope row, or a race with a concurrent seed pass) — - is also a skip, never a fatal error. Every seed step treats - "already exists" as done, not as a reason to abort the run the - hub's own error advice told the operator to re-run. - -## Tool registry publish - -`publishCorbitsToolsRegistry` (`packages/tool-registry-publish/src/publish.ts`) -finds-or-creates the tenant's `corbits-tools` package-registry asset, -then PUTs whatever tarball is missing. Packing a package needs `fs` and -`bun build`, so nothing hub-triggered may call it (CL-8190): hub boot -never publishes, `seedTenant` never packs, and -`reconcileTenantDesiredState` (`packages/onboarding/src/desired-state.ts`) -reports an absent `workspace-pack` tool pin `blocked` rather than -publishing it. The only caller of `publishCorbitsToolsRegistry` is the -operator-facing `bun run publish-tools` (`scripts/publish-tools.ts`), a -plain out-of-band API client: it signs an existing admin in and -publishes onto an already-existing tenant — the same find-or-create, -409-tolerant asset flow, never a new provisioning path. Each packed -tarball's `package.json` also carries the `ToolSurfaceManifest` the -hub reads back (`readToolSurfaceManifests`) to mint pinned-tool grants, -so grants always describe the bytes actually installed on the tenant. -Two properties keep a failed publish from stranding a -usable-looking-but-empty asset: - -- `checkToolPackageFreshness` runs **before** the asset is ever - created — a version-bump violation aborts the publish with no HTTP - call made and no asset row planted, so this exact failure can never - leave a dangling registry asset behind on a fresh tenant again. -- Listing tarballs on an asset whose repo has no commits yet (a - never-published asset, or one whose row survived from before the - point above shipped) answers an empty list rather than throwing — - so a re-run of `publishCorbitsToolsRegistry` treats it exactly like - a brand-new registry and pushes every package, which is what - actually creates the repo's first commit. Repairing a tenant with - this history is the same operation as publishing the registry for - the first time: call `publishCorbitsToolsRegistry` again. - -## Workflow deployments - -`ensureDeployment` (`packages/seeding/src/seed.ts`) treats a -workflow's `workflow_run` deployment row as seed-owned state, but the -row's `status` column is not the whole story: the hub only routes mail -to a deployment through an in-memory table (`sidecarRouter`'s -`addressIndex`, `vendor/intx/hub-sessions/src/ws/sidecar-handler.ts`) -that binds an agent address to whichever sidecar socket most recently -proved ownership of it. That table lives in the hub process, not the -database — a hub or sidecar restart empties it, while the persisted row -still reads `deployed`. - -Before skipping a `deployed`/`pending` row as "already deployed", -`ensureDeployment` checks `GET -/api/tenants/:tenantId/workflows/runs/:runId/health` (a live read of -`sidecarRouter.getRoutableAddresses()`, not the stored status) and -skips only when `liveness` answers `"ok"`. A row whose sidecar is gone -is stale, not deployed: seed logs it as stale and pushes a fresh -deployment, which mints a new `workflow_run` (new anchor run id, new -agent address) on whichever sidecar is currently connected. The stale -row is left in place rather than rebound — a sidecar carries no durable -state of its own, so handing an old run's identity to a new sidecar -process would silently pretend session state survived that never did. -A genuine redeploy is the only honest repair. - -## How a person adds a catalog workflow (CL-7073) - -The Routines page's Available section is the read side of this -invariant, surfaced to a person rather than an operator: `GET -/api/tenants/:tenantId/workflows/available` -(`packages/workflows/src/schedule/scheduled-route.ts`, -`available-catalog.ts`) answers every `CATALOG_WORKFLOWS` asset name -with no deployed `workflow_definition` on the caller's tenant yet, -each carrying `WORKFLOW_CATALOG`'s display name and one-line -description (`packages/workflows/src/catalog.ts`), its required -connections, and whether the tenant already satisfies every one of -them — a `provider` row named after the connector id exists and its -newest `credential` is `active` (mirroring -`@corbits/settings-ui`'s `connectorStatus` exactly, without importing -a UI package into a domain package). The route is injected the -catalog's asset-name list from `apps/hub` rather than importing -`@corbits/seeding` directly: that package already depends on -`@corbits/workflows`, so the reverse import would cycle. - -The web page's Add action POSTs the same -`/template-blocks/:assetName/deploy` route the GTM template's blocks -already use (`docs/seed-reconciliation.md`'s "Deployable through the -catalog" section above) — there is no second create path. A -disabled Add (a missing required connection) links to Plugins rather -than letting the request 404/500; the request itself is only ever -made once every required connection reads satisfied. On success the -entry moves out of Available into the ordinary scheduled/deployed -list the rest of the page already reads. - -## Env provider credentials - -The hub's env-key auto-plant was removed (CL-7579): hub boot never reads -provider credentials from the environment. Connect a provider in the UI -(or call the same connect API a granted setup step uses) — that flow -plants via `persistConnectorCredential` and then runs `seedCatalog`. -`plantEnvProviderCredentials` (`packages/onboarding`) is kept as a -library for a future granted setup step; nothing in production calls it -today. Its semantics, should a setup step adopt it: keyed by the -provider's stable credential name, a provider already carrying an -active credential is not probed and its key is not overwritten — a -rotated or hand-renamed key is never touched. `seedCatalog` still -runs against an existing credential (`existingCredentialId`, no -`apiKey`) so a re-connect backfills newly curated models additively: -missing rows are planted, existing ones 409-skip, nothing is deleted. -Credentials are operator data once planted, not seeds to -garbage-collect. - -## Credential-bound catalog workflows (CL-7073) - -Five `CATALOG_WORKFLOWS` entries — granola-call, morning-brief, -process-granola-call, pain-point-collateral, collateral-generation — -declare `credentialBindings` in their definition. -`deployCodeSourcedWorkflow` (`vendor/intx/hub-sessions`) refuses to -resolve those bindings without a `credentialCipher`, and the current -Interchange pin's `POST /template-blocks/:assetName/deploy` front (the -route the Routines "Available" catalog's Add action drives) has no -seam to supply one. `catalogWorkflowDeployableOnThisPin` -(`packages/seeding/src/seed.ts`) is the one place that knows this: the -deploy route refuses these six with a 409 `not_deployable_yet` before -the deployer ever throws, and the available-catalog route marks them -`deployable: false` so the UI offers them with disabled, honest copy -instead of a working-looking Add button. These six become addable at -the Interchange re-pin (CL-7107 / PR #632, pin 692c3106), which adds -the `credentialCipher` parameter this front is missing — no code in -this ledger's callers needs to change, only the entry's derived -`requiresCredentialCipher` result once the seam exists. - -## Tenant desired state (CL-7584) - -What every real tenant should have is data, not a procedure: the -tenant desired-state document (`TENANT_DESIRED_STATE` in -`packages/onboarding/src/desired-state.ts`) pins the workflows, tool -packages, and skills by name, composed BY REFERENCE over the existing -single-source constants (`DEFAULT_WORKFLOWS`, -`REQUIRED_SEED_TOOL_PACKAGES`, `DEFAULT_SKILLS`). It is the pin -source for per-tenant onboarding — no hub table, no migration, and -nothing seeded from hub boot. - -`reconcileTenantDesiredState` is the only installer. It reads the -tenant's real state (native GETs) and installs ONLY absent pins; -with everything present it is a read-only pass — `seedTenant` is -never entered. An absent `workspace-pack` tool pin is never packed or -published here (see "Tool registry publish" above): it reports -`blocked`, and an operator installs it with `bun run publish-tools`. A -`tarball-url` pin is still installed here — that's a plain fetch + PUT, -and a tarball already published under its `name@version` is skipped. -Sidecar-unavailable failures report `blocked` (the same class -`ensureSeeded` treats as pending-agents); other failures report -`failed` and are safe to re-run. - -Three triggers drive it, one reconciler: a tenant-create observation -(`POST /api/tenants` 201), the pending-seed drain over a connected -credential, and the revisit kick from `POST -/api/onboarding/provision`. The `pending_seed` row remains the only -durable work item; the tenant-create kick and the revisit kick are -deliberately in-memory — convergence, not delivery. - -Member-edit posture: a reconcile only ever installs what the document -names. A member (or anyone) deleting or editing a seeded workflow or -skill afterwards is not reverted by a later pass — the document is a -floor for provisioning, not a drift-enforcement policy. The only -redeploy a pass performs is the absent-pin install; existing live -deployments are skipped (`ensureDeployment` staleness behavior -unchanged). diff --git a/docs/sidecar-provisioners.md b/docs/sidecar-provisioners.md index 4557734e5e..6d61c41c1d 100644 --- a/docs/sidecar-provisioners.md +++ b/docs/sidecar-provisioners.md @@ -1,29 +1,24 @@ # Sidecar provisioners -A **sidecar** is the execution host a workbench's agents and workflow runs -actually run on. A **provisioner** is the backend that creates and destroys -one: Interchange's `SidecarProvisioner` contract, `ensure`/`destroy`, keyed -by allocation and generation. - -This is the one page for deciding where an install's sidecars run. +A **sidecar** is the execution host a tenant's agents and workflow runs +actually run on. A **provisioner** is the backend that creates and +destroys one — Interchange's `SidecarProvisioner` contract. ## The shipped backends -| id | Where a sidecar runs | Isolation | Requires | Package | -| --------- | --------------------------------------- | ------------------------------------------------ | ------------------------------------------------------- | ------------------------------ | -| `process` | A child process of the hub, same host | Process only — shared kernel, filesystem, user | nothing | `packages/process-provisioner` | -| `docker` | A container on the hub's host | Container — own filesystem and network namespace | `DOCKER_PROVISIONER_IMAGE`, a local `docker` CLI | `packages/docker-provisioner` | -| `e2b` | A remote [E2B](https://e2b.dev) sandbox | VM — separate machine | `E2B_API_KEY`, `E2B_TEMPLATE`, a publicly reachable hub | `packages/e2b-sandbox-sidecar` | +| id | Where a sidecar runs | Isolation | Requires | Package | +| --------- | ---------------------------------------- | ----------------------------------------------- | -------------------------------------------------------- | ------------------------------- | +| `process` | A child process of the hub, same host | Process only — shared kernel, filesystem, user | nothing | `packages/process-provisioner` | +| `docker` | A container on the hub's host | Container — own filesystem and network namespace | `DOCKER_PROVISIONER_IMAGE`, a local `docker` CLI | `packages/docker-provisioner` | +| `e2b` | A remote [E2B](https://e2b.dev) sandbox | VM — separate machine | `E2B_API_KEY`, `E2B_TEMPLATE`, a publicly reachable hub | `packages/e2b-sandbox-sidecar` | -`process` is the **default**: a hub with `SIDECAR_PROVISIONERS` unset -registers it as the sole backend, so one server runs many chats and -workflows with no operator configuration at all. Pick `docker` or `e2b` -when a sidecar runs untrusted code or needs its own filesystem and network. +`process` is the default: with `SIDECAR_PROVISIONERS` unset, the hub +registers it alone, so one server runs many tenants with no operator +configuration. Pick `docker` or `e2b` when a sidecar runs untrusted code +or needs its own filesystem and network. ## Configuring it -Two variables decide everything: - ```sh # Register a set of backends. Unset means exactly `process`. SIDECAR_PROVISIONERS=process,docker @@ -32,84 +27,33 @@ SIDECAR_PROVISIONERS=process,docker SIDECAR_DEFAULT_PROVISIONER=docker ``` -Setting `SIDECAR_PROVISIONERS` **replaces** the default rather than adding -to it — list `process` explicitly to keep it alongside another backend. An -unknown id, a duplicate id, a missing backend setting, or a -`SIDECAR_DEFAULT_PROVISIONER` naming an unlisted id all fail the boot with -a message naming the variable; nothing silently degrades to a different -backend than the one asked for. - -Per-backend settings, and the `HUB_SIDECAR_WEBSOCKET_URL` override a -container or remote sandbox needs to dial back, are documented against each -variable in [`.env.example`](../.env.example) and in -`apps/hub/src/config.ts`'s env schema. - -### Where a sidecar's state lives +`SIDECAR_PROVISIONERS` replaces the default rather than adding to it — +list `process` explicitly to keep it alongside another backend. An +unknown id, a missing backend setting, or a default naming an unlisted id +all fail boot with a message naming the variable. Per-backend settings +are documented in `.env.example` and `apps/hub/src/config.ts`. -Every backend keeps its hub-side allocation state — generation fences, -destroy tombstones, external refs — under the hub's own `HUB_DATA_DIR`, -derived and never separately configurable, so no two backends can drift or -collide: - -``` -$HUB_DATA_DIR/process-provisioner/ -$HUB_DATA_DIR/docker-provisioner/ -$HUB_DATA_DIR/e2b-provisioner/ -``` - -The `process` backend additionally keeps each allocation's own -`SIDECAR_DATA_DIR` there, one directory per generation, removed when the -allocation is destroyed — see -[`packages/process-provisioner/README.md`](../packages/process-provisioner/README.md). - -### What a backend declares it can do +### Capability matching Interchange selects a provisioner by matching a deployment's required -capabilities against what each registered backend **declares**, and fails -closed when nothing matches — an undeclared capability reads as `unknown`, -never as "probably fine". The shipped backends declare the isolation they -actually give the code they run, as a ladder (`isolation:process` → -`isolation:container` → `isolation:vm`): each reaches every rung at or -below its own and blocks every rung above it, plus `runtime:sidecar`, which -all three provide. +capabilities against what each registered backend declares, failing +closed when nothing matches. The shipped backends form a ladder +(`isolation:process` → `isolation:container` → `isolation:vm`): each +reaches its own rung and below, never above. | Declaration | `process` | `docker` | `e2b` | -| --------------------- | ----------- | ----------- | --------- | -| `runtime:sidecar` | available | available | available | -| `isolation:process` | available | available | available | -| `isolation:container` | **blocked** | available | available | -| `isolation:vm` | **blocked** | **blocked** | available | - -So a deployment that requires `isolation:vm` will not land on the default -`process` backend; it selects `e2b` if registered, and otherwise fails -provisioner selection with a message naming the mismatch. The ladder lives -in one place — `sidecarCapabilityDeclarations` in -`packages/sandbox-sidecar/src/capabilities.ts` — so a new backend states -its isolation level and inherits the rest. +| ---------------------- | ----------- | ----------- | --------- | +| `isolation:process` | available | available | available | +| `isolation:container` | blocked | available | available | +| `isolation:vm` | blocked | blocked | available | ## Adding a backend -Three steps, no hub surgery: - -1. **Implement the backend.** Copy `packages/process-provisioner`: a - `SidecarBackend` (`startUnit` / `stopUnit` / `findUnitsByAllocation`) - plus identity (`id`, `apiVersion`, `bindingFingerprint`). Idempotence, - generation fencing, destroy tombstones, and obsolete-unit sweeping come - from `@corbits/sandbox-sidecar`'s shared core — never reimplement them - per backend. -2. **Name it.** Add its id to `SIDECAR_PROVISIONER_IDS` and a member to - `SidecarProvisionerConfig` in `apps/hub/src/config.ts`, with its own - required settings parsed there (the hub's one env boundary). -3. **Wire it.** Add a case to `buildSidecarProvisioner` in - `apps/hub/src/index.ts`. - -The registry itself is composed at the hub's composition root and takes -whatever the config named, so a new backend is a package plus two switch -cases — an operator then reaches it by id, exactly like the shipped three. - -## Related +1. Implement a `SidecarBackend` (copy `packages/process-provisioner`) — + idempotence, generation fencing, and destroy tombstones come from + `@corbits/sandbox-sidecar`'s shared core. +2. Add its id to `SIDECAR_PROVISIONER_IDS` in `apps/hub/src/config.ts`, + with its own required settings parsed there. +3. Add a case to `buildSidecarProvisioner` in `apps/hub/src/index.ts`. -- [`packages/process-provisioner/README.md`](../packages/process-provisioner/README.md) - — the default backend's layout, environment, and trade-offs. -- [`packages/e2b-sandbox-sidecar/README.md`](../packages/e2b-sandbox-sidecar/README.md) - — building the E2B template. +No other hub surgery is needed. diff --git a/docs/toasts.md b/docs/toasts.md deleted file mode 100644 index 9087285e92..0000000000 --- a/docs/toasts.md +++ /dev/null @@ -1,30 +0,0 @@ -# Toast confirmations - -Every successful mutation confirms itself with the shared toast from -`@corbits/react-ui`: a single plain-text line that fades in over the bottom -center, holds briefly, and fades out. A new confirmation replaces the one on -screen rather than stacking, so rapid mutations read as one status line. - -## How to adopt it - -- The `` mounts once, in `apps/web/src/main.tsx` inside the - `ThemeProvider`. Never mount a second one. -- Fire `toast(message)` from the mutation's success path — after the awaited - call resolves, alongside the reload/invalidate. Failures keep their - existing inline error states; the toast is for success only. -- Copy lives with the surface's other copy: `CHAT_STRINGS` for chat, - `SETTINGS_STRINGS` for settings, and pure helpers (`artifactUploadToast`, - `routineCreatedToast`, `routineRunStartedToast`) next to the API module - they describe. No component inlines bespoke toast copy. -- Copy is a plain confirmation. Where it names a subject, `·` separates the - action from it: `Workbench created · Launch planning`, `Uploaded · -q3-report.pdf`, `Credential saved · value hidden`. A multi-file upload is - the one count form: `Uploaded 4 files`. State toggles read as the state - just entered (`Pinned Deploy notes`), and bare confirmations stay bare - (`Grant created`, `Settings saved`). - -## Where it fires today - -Workbench create, rename, pin/unpin, and workbench settings save; routine create -and run-now; grant create and revoke; credential save and revoke; artifact -upload; bench name and conversation-default saves. diff --git a/docs/workbenches.md b/docs/workbenches.md deleted file mode 100644 index 51bec06c1d..0000000000 --- a/docs/workbenches.md +++ /dev/null @@ -1,56 +0,0 @@ -# Workbenches - -A workbench is Workbench's one conversational surface: every workbench is a -multi-agent room, and there is no separate chat-vs-workbench behavioral split -— a 1:1 chat and a multi-agent workbench route messages the same way (see -[CHAT.md](CHAT.md) for the underlying workbench/thread model). - -## Host routing - -`sendWorkbenchMessage` (`packages/chat/src/workbench-service.ts`) decides who -receives a message without ever branching on the workbench's `kind`: - -- **`@mention` an agent** — that agent always receives the message. -- **Reply to an agent's message** — that agent receives it too, even - unmentioned. -- **Neither** — the message defaults to the workbench's **host**, its - first agent participant. A single-agent workbench still auto-responds; - a multi-agent one routes through its host instead of going silent. - -An agent's own reply runs the same mention fan-out (`chat-orchestrator.ts`'s -`postReply`), so an agent can delegate mid-conversation by @mentioning a -teammate in its reply. - -## Myra as standing host - -Myra (the `assistant` default workflow, `agents/assistant/src/index.ts`) -is the host every new workbench starts with. Her triage clause decides, on -every message, whether to answer directly or delegate: answer directly for -questions, drafting, or anything reasoned through in the conversation; -delegate — by dispatching a task to an existing agent, or by drafting and -creating a new specialist agent when none fits — for a distinct, boundable -job, especially recurring ones (draft a routine instead of being asked -each time). In a workbench with other agent teammates, Myra delegates by -@mentioning the specialist and saying why in a few words, rather than -answering on that specialist's behalf. - -## Approvals - -A tool gated `approval: "ask"` parks its run instead of executing: the -run's `reactor.gate.blocked` event surfaces in-chat as an approve block -(`packages/chat/src/blocks.ts`'s `ApproveBlockData`), carrying only a -platform-minted `approvalId` and the agent's framing (title, risk, body). -Deciding on the card — or through the platform's approvals API — resumes -the parked run. The orchestrator never mints an approval itself; it only -reads the row the hub's own suspension-registration co-write already -wrote. - -## Routines and tasks - -A routine's runs and a dispatched task's result both deliver into a -workbench rather than a separate destination: a routine's delivery lands -in its workbench's delivery thread, and a dispatched task's completion or -failure posts into the workbench it was dispatched from (falling back to the -tenant's assistant chat when no origin workbench was recorded) — never only -an inbox. There is no standalone Inbox page; task/approval affordances live -inside the workbench itself. diff --git a/docs/workflow-definition-access.md b/docs/workflow-definition-access.md deleted file mode 100644 index 7820137756..0000000000 --- a/docs/workflow-definition-access.md +++ /dev/null @@ -1,76 +0,0 @@ -# `workflow_definition` access in Workbench - -Analysis for CL-7275. Read against the vendored pin `a8bc06ae`. - -## The finding - -`workflow_definition` is an Interchange-owned table. The pin ships -`createWorkflowDefinitionStore` (`vendor/intx/db/src/workflow-definition-store.ts`) -to read it, and that store has **zero** callers here. Meanwhile **41 direct drizzle -queries** hit `workflowDefinition` / `workflowDefinitionVersion` across 14 non-test -files. - -## Why the store is bypassed - -Not laziness. Its whole surface is the `(assetId, wireHash)` identity selector, -`loadFrozenGrantSnapshot`, `loadFrozenWireProjection`, and `rollback`. The two -`loadFrozen*` helpers **are** used (2 and 6 call sites) — callers reach for the -store when it fits. It simply has no by-id, by-name, by-tenant or by-asset read, -and no update path. - -## What the 41 sites actually need - -| Category | Count | Shape | -| --------- | ----- | ----------------------------------------------- | -| by-id | 26 | `(definitionId, tenantId)` | -| by-tenant | 6 | every deployed definition for a tenant | -| by-name | 5 | `(name, tenantId[, status])` | -| by-asset | 2 | every definition sharing an asset, newest-first | -| update | 2 | patch `description` / `status` | - -Spread: `packages/agent-directory` (7 files), `apps/hub/src/index.ts` (12 sites), -`packages/chat/src/platform-adapter.ts` (7), plus `folded-runs`, -`folded-run-one-shot`, `webhook-triggers`, `evals`, -`routine-launcher`, `skills-mount`. - -## What a prototype migration surfaced - -A prototype routed five callers (one per category) through a widened store. Doing -so exposed a second-order problem: the store runs rows through -`parseWorkflowDefinitionRow`, which validates the full row shape — status enum, -jsonb columns. Several existing fake-db fixtures stubbed only two or three fields -and failed immediately. - -Those fixtures were asserting against row shapes the database cannot produce. The -hand-rolled queries are not merely duplicative; they let unrealistic test doubles -pass. That is the strongest argument for routing these call sites somewhere typed. - -## Why the prototype is not this PR - -The prototype widened the store **inside** `vendor/intx/db`. That is an edit within -a vendored tree, and it carries re-pin tax: every line must be hand-reapplied at -each future pin (CL-7107), and it grows the vendored delta rather than shrinking -our own re-creation. Interchange origin/main is a read-only reference we vendor or -tag from — the widening cannot go anywhere else, so it has to live somewhere that -survives a re-pin cleanly. - -The prototype is preserved on `cl-7275-vendor-edit-archive` (`8a6c9901`) for -reference. It should not merge in that shape. - -## Options - -1. **A thin `@corbits/*` wrapper** over the native store's existing surface, adding - the five reads above. Zero edits to `vendor/intx/db`, survives a re-pin, gets - callers onto `parseWorkflowDefinitionRow`. Costs one small package we own. -2. **Leave the call sites alone** until a re-pin brings a wider native surface. No - new code and no tax, but the fixture problem above persists. - -Undecided. Option 2 is the smaller re-creation and matches "use as much of the pin -as possible, re-create as little as possible"; option 1 buys type safety at the -call sites now. - -## Consequence for the parent check (CL-7257) - -A `check:*` failing on direct `workflow_definition` access cannot ship before one of -the options above lands. Today it would fail 41 times with no correct alternative -to route to — a debt ledger, not enforcement. diff --git a/docs/workflow-deploy-source-vs-launch-spec.md b/docs/workflow-deploy-source-vs-launch-spec.md deleted file mode 100644 index 3cd8146621..0000000000 --- a/docs/workflow-deploy-source-vs-launch-spec.md +++ /dev/null @@ -1,106 +0,0 @@ -# `workflow_deploy_source` vs `workflow_run_launch_spec` - -> Superseded by CL-7591: `@corbits/workflows`'s `./deploy-source` (the -> `workflow_deploy_source` table) is deleted and the workflow detail read -> is native-only. This analysis is kept as the historical decision record -> for CL-7271; do not read it as a description of the current schema. - -Analysis for CL-7271. Read against the vendored pin `a8bc06ae`. - -## The claim under test - -CL-7271 was filed on the claim that `@corbits/workflows`'s `./deploy-source` duplicates -Interchange's native `workflow_run_launch_spec`, and should be deleted in favour -of it. - -**That claim is too strong.** The two tables share a key, a purpose statement and -several columns, but they implement _different recovery models_. Deleting one for -the other is a design decision, not a cleanup. - -## What each one actually stores - -| | `workflow_run_launch_spec` (native) | `workflow_deploy_source` (ours) | -| ----------------- | --------------------------------------------------------------------------------- | ----------------------------------------------------------------- | -| Key | `anchor_run_id` (FK → `workflow_run`, cascade) | `anchor_run_id` (no FK) | -| Written by | `prepareExclusiveDeployment` only | `withDeploySourceRecording`, every placement | -| Written when | Inside the same transaction as the anchor `workflow_run` row and its `read` grant | After a deploy resolves | -| Recovery payload | `frozen_approval_bundle` — the whole approval, verbatim | `source` + `entry` + `pin` + `definition_asset_id` + `source_ref` | -| Redeploy strategy | **Replay the freeze.** No re-probe | **Re-resolve from the source.** Re-derives | -| Shared columns | `session_id`, `deployment_domain`, `source_authority_principal_id` | `tenant_id`, `deployment_domain`, `source_authority_principal_id` | -| Secrets | None — offering ids re-resolved at launch | None — inference sources re-resolved at redeploy | - -## Why native is exclusive-only - -`createWorkflowRunLaunchSpecStore` has exactly one writer and one reader, both in -`vendor/intx/hub-sessions/src/workflow-allocation-service.ts`: - -- `prepareExclusiveDeployment` (line 250) writes the spec inside the transaction - that mints the anchor run and its grant. -- `deployReadyAllocation` (line 327) reads it back and rehydrates - `InstallAndApproveResult` from `spec.frozenApprovalBundle` — approved wire hash, - approved grants, projection — then deploys with **no re-probe**. - -It is exclusive-only because it is part of the _exclusive allocation lifecycle_: -it exists to survive an allocation being replaced under a run. A shared-capacity -deploy has no allocation generation to lose, so upstream never needed it there. - -## The real difference: freeze vs re-resolve - -This is the crux, and it is a genuine design fork. - -**Native freezes.** `frozen_approval_bundle` is the approval verbatim. A redeploy -replays exactly what was approved — same wire hash, same grants, same projection. -Nothing is recomputed, so nothing can drift. The cost: an approval frozen against -a credential that later died stays frozen, and the run redeploys against a dead -chain forever. - -**Ours re-resolves.** We store where the bytes came from and re-derive at redeploy -against the tenant's live catalog. A rotated credential is picked up. The cost: the -redeploy is not guaranteed to reproduce what was approved. - -That trade-off is exactly the one CL-6687 ("Rotated API keys never reach live -agents", Done) was fixed in favour of re-resolution, and the same one -`apps/sidecar/src/workflow-host-wiring/index.ts`'s `restoreDeploymentFromRecord` -decides the same way when it returns `"deferred-to-wake"` rather than restoring -from a frozen `sources` snapshot (CL-6648). - -So the two tables encode opposite answers to the same question, and this repo has -already ruled twice in favour of re-resolution. - -## What this means for the ticket - -Adopting `workflow_run_launch_spec` wholesale would mean adopting the freeze model -for every placement, and re-opening CL-6687. That is not a cleanup. - -The defensible options, in order of preference: - -1. **Keep both, narrow ours.** Native owns exclusive-allocation recovery (it - already does). Ours owns shared-placement source recording. Document the split - and stop describing them as duplicates. Cheapest, and preserves both rulings. -2. **Extend native with a re-resolve arm.** Add the source/entry/pin columns to - `workflow_run_launch_spec` as an alternative to `frozen_approval_bundle`, and - raise it upstream. One table, two recovery strategies, chosen per row. Larger, - and needs upstream agreement. -3. **Cut over to freeze.** Rejected — reverses CL-6687. - -Option 1 does not delete any code. That is the honest outcome: the ~350 loc in -`@corbits/workflows`'s `./deploy-source` is not redundant, and CL-7271's premise that it -could simply be deleted does not survive reading the native writer. - -## What still stands from CL-7271 - -Two of the original observations survive independently of the above: - -- `workflow_deploy_source.anchor_run_id` has **no foreign key** into - `workflow_run`, where the native table has one with `onDelete: "cascade"`. - That is a real defect and belongs to CL-7258. -- The columns that genuinely have no native counterpart are `entry`, `pin`, - `definition_asset_id` and `source_ref` — the last already carries an in-tree - `WORKBENCH DELTA` comment. These are what option 2 would need to upstream. - -## Not verified - -- Whether any shared-placement deploy could ever lose its allocation the way an - exclusive one can. If it cannot, ours is recording for a recovery that never - happens, which is a separate question worth asking. -- Migration shape for existing `workflow_deploy_source` rows under option 2. diff --git a/docs/workflow-detail.md b/docs/workflow-detail.md deleted file mode 100644 index d7ff3d1340..0000000000 --- a/docs/workflow-detail.md +++ /dev/null @@ -1,52 +0,0 @@ -# Workflow definition detail (CL-7371) - -A workflow definition's own page: `GET -/api/tenants/:tenantId/workflows/definitions/:definitionAssetId/detail` -(`packages/workflows/src/detail`), read at `/workflows/ -:definitionAssetId` (`apps/web/src/pages/workflow-detail-page.tsx`). First -useful version — read-only, no editing surface here. - -## What it answers - -- **Lifecycle**: `source-only`, `pending-approval`, `deployed`, - `superseded`, or `build-failed` — derived by the pure - `deriveWorkflowLifecycle` (`packages/workflows/src/ -definition-lifecycle.ts`) from the asset's `workflow_definition` rows - alone. Native-only since CL-7591: the deleted Workbench deploy-source - store is no longer a deploy-attempt signal. No new Postgres column: - everything it reads is native. -- **Steps**: read from the frozen `wire_projection` (`@intx/db`'s - `loadFrozenWireProjection`) in `stepOrder`, each carrying its role - (`kind`), best-effort director/model/tool pins (the wire step schema is - deliberately open past `kind`/`id`/`after` — see - `vendor/intx/types/src/wire-workflow.ts` — so these read defensively and - are simply absent rather than erroring on an unrecognized shape), and the - grants the deploy-time capability walk froze onto that step - (`grantSnapshot.perStep`). -- **Access**: declared grants (the source's own `grant_requirements`) next - to approved grants (the last freeze's `grant_snapshot.grantRequirements`) - — a person can see the gap between what a workflow asks for and what was - actually approved. Credential binding **names** only - (`workflow_definition.credential_bindings[].handle`); no value is ever - read or returned. -- **Source**: always `null` since CL-7591 — the Workbench deploy-source - store is deleted, so the native-only read records no per-asset deploy - provenance. - -## Authorization - -`requireGrant(idResource("workflow-definition", "definitionAssetId"), -"read")` runs before any row is read. An asset absent or owned by another -tenant reads as 404 from the route body itself (same convention as the -vendored `.../:definitionId/versions` route), never a 403 that would -confirm the id exists. - -## What's deliberately left out of this first version - -- No cross-links from the routine target picker or an agents roster row — - neither exists yet on the branch this shipped against. Wiring those in - is a follow-up once they land. -- `superseded` is read off `workflow_definition.status: "stopped"` on the - newest row for the asset, not a full multi-row version history; a - definition's older wire hashes are not separately browsable from this - page yet. diff --git a/docs/workflow-model.md b/docs/workflow-model.md deleted file mode 100644 index d1b917c0a8..0000000000 --- a/docs/workflow-model.md +++ /dev/null @@ -1,143 +0,0 @@ -# Workflows, agents, routines, and runs — the canonical model - -Contract for CL-7349. Read against the vendored pin `a8bc06ae` and upstream -`faremeter/interchange` origin/main `d187e327` (2026-09-01). Every -implementer of the Routines & Workflow Alignment project builds to this -document; a surface that disagrees with it is wrong until this document -changes. - -## Five nouns, one execution model - -| Noun | What it is | Where it lives | -| ----------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- | -| **Workflow source** | A versioned code package: `package.json` declaring `interchange.workflow` plus the entry module it names, default-exporting a `WorkflowDefinition` | A tenant-scoped `kind: "workflow"` asset in the hub's git-backed asset store (`AssetService`) | -| **Deployed definition** | The frozen, approved projection of one source commit: `workflow_definition` row with non-null `approved_wire_hash`, `grant_snapshot`, `wire_projection`, `status: deployed` | Produced only by Interchange's source pipeline (`deployWorkflowFromSource`: bundle → sidecar probe → capability walk → gate → freeze) | -| **Agent** | A single-step conversational workflow. A product category, not an execution primitive | Same asset + same deployed definition as any workflow | -| **Routine** | Product name for a scheduled workflow: an authored definition whose frozen projection carries a native `ScheduleTrigger` | `@corbits/workflows` schedule helpers; hub `workflow-scheduler.ts` poller | -| **Run** | One execution of an approved deployed definition | `workflow_run`, launched by `launchAndCorrelate` | - -`workflow.json` is retired. It is not an authoring format, not a -compatibility format, and no path may read or write it. The push validator -(`vendor/intx/hub-sessions/src/workflow-kind.ts`) refuses it; -`@corbits/workflows`'s `./source`'s `RetiredWorkflowEnvelopeError` is the only -remaining mention, and it exists to reject. - -## Definition identity and the follow-latest rule - -Interchange keys `workflow_definition` on `(asset_id, wire_hash)` -(`vendor/intx/hub-sessions/src/workflow-definition-ensure.ts`). Every -redeploy of a source asset whose probed wire hash differs mints a **new** -definition row. There is no native "definition id → newest approved -deployment" indirection; every native launch names an exact definition. - -Product ruling (2026-09-01): a scheduled fire follows its target's latest -approved deployment and does not pin the version selected at creation. -Under the identity above that means: - -- A scheduled definition is identified by its **definition asset id**. The - asset is the stable identity of a workflow across redeploys. -- At launch, the target resolves to the newest `workflow_definition` row - for that asset with `status = 'deployed'` and non-null - `approved_wire_hash`, `grant_snapshot`, and `wire_projection`. Resolution - is atomic with the launch and fails closed when no such row exists. - -Nothing in Workbench pins a wire hash or copies a projection into a -separate routine table. - -## Authority boundaries - -| Operation | Canonical operation | Authorized as | Human approval | -| --------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------- | -| Store source (create / republish) | `@corbits/workflows`'s `./authoring` registry → `AssetService.createAsset` / `populateAsset` (hub-signed commit) | Initiating tenant + principal; `@intx/authz` `authorize` on `asset:*`/`create` or `asset:`/`write` | None (writing source is not a side effect) | -| Deploy source | `POST /api/tenants/:tenantId/workflows/deployments` → vendored `SessionService.deployWorkflowFromSource` | Tenant session or run bearer; `workflow:*`/`create` | Agent-initiated deploys go through an `approval: "ask"` tool call carrying the probed capability surface (below) | -| Create / update a schedule | Native `ScheduleTrigger` on the frozen definition; pause/resume and run-now on `@corbits/workflows` schedule routes | Tenant + principal; target validated against the resolution rule above before persisting | None; writing a cadence is not a side effect | -| Launch | Hub poller (`workflow-scheduler.ts`) ticks matching minutes → `triggerNativeWorkflowRoutineRun`; mail and webhook ingress launch the same way | The workflow run's own principal, granted natively (per-pin tool grants, mail-triggered run grants); no creator re-authorization — the fire's gates are fail-closed target resolution plus native grant materialization | Runtime tool calls with `approval: "ask"` park on the native `approval` resource | -| Approve | Native `POST /api/tenants/:tenantId/approvals/:id/approve` | A principal holding `approval:*`/`resolve` — a human; no agent holds it | This is the approval | - -Credentials and resolved provider secrets never enter source trees or deploy -requests. -Inference sources are re-resolved from the tenant catalog at deploy and -redeploy (`resolveDefinitionSources`). - -### Deploy approval for agent-authored workflows - -Upstream's deploy route freezes with `approvals: { mode: "approve-probed" }` -(`vendor/intx/hub-sessions/src/session-service.ts`), unmodified — no -vendored delta grants a caller-supplied approval policy or a -probe-without-freeze entry point (one was prototyped for CL-7362 and -reverted; see VENDORED.md). The only native pending-approval store is the -runtime `approval` resource that an `approval: "ask"` tool call parks on. -Workbench composes what exists, with no vendored delta and no approval -table: - -1. Myra calls `wf_deploy_preview`, a STATIC, read-only render of the - already-committed source at `commitSha` — package name, file list, and - any `toolPackagePins` a plain `export default {...}` entry declares. - Never installs, probes, gates, or freezes anything, so it truly cannot - deploy. -2. Myra calls `workflow_deploy` (`approval: "ask"`) with the asset id, - commit sha, entry, and the preview's `packageName`/`toolPackagePins` - carried along on the call. The tool call parks; the approval headline - reads "Deploy workflow \ @ \ — tools: \" — the committed source the human is approving, not yet the - grants/capabilities the deploy will freeze (no no-freeze probe seam - exists to preview those; see the vendored-delta revert above). -3. On approval the tool posts to the native deployments route, which runs - the real install + probe + gate + freeze under the default - `approve-probed` policy. A rejection there leaves the source intact and - the definition unlaunchable; runtime tool calls against the deployed - definition remain approval-gated regardless. - -Myra cannot resolve approvals: `approval:*`/`resolve` is never minted for an -agent principal. - -## Deleted in CL-7364 - -- Routine target inference from chat membership: `apps/web/src/shell/ -routine-panel.tsx` picks a target only through `DefinitionTargetPicker` - now; no `resolveCreateTarget`, `agents[0]?.definitionId`, or "no agent - invited" guard remains. `check:routine-target-inference` guards this — - it fails on `agents[0]?.definitionId` / `agents[0].definitionId` in - `apps/web` and `packages/chat-ui`. -- The template-block route's hub-local self-freeze: `apps/hub/src/index.ts` - (`createTemplateBlockRoutes`'s `deployWorkflowSource` binding) writes the - block's source tree and calls the same `workflowDeployer.deploy` the - agent-authored deploy path uses, instead of - `@corbits/workflow-freeze`'s `freezeInertWorkflowDefinition`. -- `packages/workflow-host-actions` — already gone (no tracked source, no - importers) by the time this landed. -- Any code path that reads or writes the retired `workflow.json` path, - except `@corbits/workflows`'s `./source`'s own `RetiredWorkflowEnvelopeError`; - `check:routine-target-inference` guards this too. - -`@corbits/workflow-freeze` itself is deleted: `packages/agent-directory` -(create, restore, skill-pin, and capability routes in `apps/hub/src/ -index.ts`) now deploys agent definitions through the same injected -`WorkflowDeployer` the template-block path uses, instead of -`DefinitionFreezer.freeze`/`.refreeze` (CL-7364). - -No compatibility shim, feature flag, or dual-write period accompanies any -of the deletions above. - -## Deleted in CL-7375 - -The routine draft/review state machine — `POST /routine-drafts` (create), -`GET /routine-drafts`/`GET /routine-drafts/:id` (review), `POST -/routine-drafts/:id/approve`, and `POST /routine-drafts/:id/discard`, the -`routine_draft` table, `drafts.ts` and `myra-drafting.ts`, and -`suggestRoutineNameFromPrompt`. Cadence now lives as a native -`ScheduleTrigger` on the frozen definition; there is no second, -review-first path. `routine_draft` was dropped by a migration -(`0007_drop_routine_draft`), not left as inert dead weight. - -## What is not native, and stays in Workbench - -Checked against upstream origin/main `d187e327`: Interchange has no -routine table and no definition-level scheduler — its `ScheduleTrigger` -type is projected, and Workbench's hub poller (`workflow-scheduler.ts`) -is the consumer (per-minute CAS on -`workflow_definition.schedule_claimed_minute`, skip-missed). Per-principal -"launchable definitions" and the agent-facing tool that writes assets -remain Workbench compositions, built over native rows and `@intx/authz`. -Before adding anything else, check upstream first; if upstream has it -past our pin, re-vendor at that commit rather than reimplementing. diff --git a/docs/workflow-source-authoring.md b/docs/workflow-source-authoring.md deleted file mode 100644 index 3c7fccec3c..0000000000 --- a/docs/workflow-source-authoring.md +++ /dev/null @@ -1,44 +0,0 @@ -# Authoring workflow source as a hub asset - -Contract for CL-7352: how an agent (Myra) or a person writes a workflow code -package into the hub and gets it deployed through Interchange's native -pipeline. Companion to [workflow-model.md](workflow-model.md). - -## The package shape - -A `kind: "workflow"` asset carries an ordinary code package. The push -validator (`vendor/intx/hub-sessions/src/workflow-kind.ts`, -`workflowKindHandler.validatePush`) accepts nothing else. - -``` -package.json — name, version, "type": "module", "interchange": { "workflow": "./workflow.ts" } -workflow.ts — default-exports defineWorkflow({ ... }) from @intx/workflow - — anything the entry imports; no secrets, no .env -``` - -Rules the authoring boundary enforces before any write: - -- Paths are repo-relative, normalized, no `..`, no leading `/`, no - `.git/`. Names matching secret-like patterns (`.env*`, `*.pem`, `*.key`, - `id_rsa*`) are rejected. -- `package.json` parses, declares a non-empty `interchange.workflow`, and - that entry exists in the same tree. -- Total tree size and per-file size are capped. -- The asset name is lowercase-kebab (`ASSET_NAME_PATTERN`). - -Inert JSON definitions (`export default {...}` written by -`renderWorkflowSourceTree`) remain a valid package shape for hub-generated -single-step agents, because they are code the sidecar evaluates. They are -not what an agent authors by hand. - -## The operations, in order - -| Step | Operation | Authorized as | Returns | -| ---- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------- | -| 1 | `POST /api/workflow-workflow-authoring/author` (`@corbits/workflows`'s `./authoring`) → `AssetService.createAsset` + `populateAsset` | Run bearer + run address → tenant/principal; `asset:*`/`create` | `{ assetId, name, commitSha }` | -| 1' | `.../republish` → `populateAsset` on `refs/heads/main` | `asset:`/`write`, own-tenant row check first | `{ assetId, name, commitSha }` | -| 1'' | `GET .../:assetId/source` → `RepoStore.resolveRef` + `openCommittedReads` on `refs/heads/main` | `asset:`/`read`, own-tenant row check first | `{ assetId, name, headSha, files }` | -| 2 | `POST .../:assetId/deploy/preview` (CL-7362) — a STATIC, read-only render of the already-committed source at `commitSha` off `RepoStore` alone; never installs, probes, gates, or freezes | Same run scope | `{ commitSha, entry, files[], toolPackagePins[], packageName }` or an invalid-package error | -| 3 | `POST /api/workflow-workflow-authoring/:assetId/deploy` (CL-7361) → same `sessionService.deployWorkflowFromSource` call the native `POST /api/tenants/:tenantId/workflows/deployments` route drives, with `sources` resolved server-side from the tenant catalog (`modelRequirements: null` — a per-workflow model requirement, if the package ever declares one, is NOT considered at this step; resolution always targets the tenant's default/first-preference model) | Run bearer + run address → tenant/principal; `workflow:*`/`create`, own-tenant row check first; the `workflow_deploy` tool call itself carries `approval: "ask"`, and the approval card shows the package name and any statically-declared tool pins from step 2 — not the grants/capabilities the deploy will freeze, see CL-7362 below | `{ deploymentId, definitionAssetId, status }` | -| 4 | Human resolves the parked approval (native `approvals` route) | `approval:*`/`resolve` | Deploy continues or is rejected | -| 5 | `workflow_definition` row frozen; appears in routine target discovery | — | Launchable |