From 4db3307e7b003e054fe2edd3f5b3968b92565888 Mon Sep 17 00:00:00 2001 From: Edward Irby Date: Wed, 23 Sep 2026 22:06:11 -0700 Subject: [PATCH 01/55] feat: wire sessionId alongside instanceId on the trace wire MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The engine self-mints `instanceId = ueid('bp_')` and stamps it on every trace. With an ACP ingress landing later, the host owns session identity policy (mints on `session/new`, passes a client's old id on `session/load`), so the engine must ACCEPT a host-supplied session id — it never mints one and never returns ids. No rename: the two ids are separate axes and both live on the wire. - `TraceBase` gains `sessionId: string` alongside `instanceId`; a new exported `TraceBaseSchema` is the one home for the trace wire's common shape, so per-kind validators derive from it instead of hand-mirroring. - `behavioral()` gains `{ sessionId?: string }`; the frozen return object is unchanged. Session id is computed once at factory time (`options?.sessionId ?? instanceId`) and stamped on every trace next to `instanceId`; `resumePendingThreadsForSelectedEvent` stamps it on interrupt traces too. - The frontier faculty speaks the same wire with no drift: replay/explore/ verify inputs accept `sessionId` (AJV `nullable`, defaulting to the `instanceId`), and every synthetic trace it builds (frontier, selection, deadlock) carries both ids. - `serve.ts`/`trace-consumer.ts` are pass-through only — no construction sites existed in production code; only spec fixtures gained the field. - `skills/behavioral/references/behavioral.md` factory signature updated. Validation: `bun --bun tsc --noEmit` plus the behavioral, frontier faculty, and cli serve/trace-consumer suites (211 specs). --- skills/behavioral/references/behavioral.md | 6 +- src/behavioral/behavioral.ts | 23 +++++- src/behavioral/behavioral.types.ts | 25 +++++++ src/behavioral/behavioral.utils.ts | 3 + src/behavioral/tests/schemas.spec.ts | 34 ++++++--- src/behavioral/tests/session-id.spec.ts | 37 ++++++++++ src/cli/tests/serve.spec.ts | 1 + src/cli/tests/trace-consumer.spec.ts | 1 + src/faculties/frontier/faculty.ts | 73 ++++++++++++++++++-- src/faculties/frontier/tests/faculty.spec.ts | 41 ++++++++++- 10 files changed, 226 insertions(+), 18 deletions(-) create mode 100644 src/behavioral/tests/session-id.spec.ts diff --git a/skills/behavioral/references/behavioral.md b/skills/behavioral/references/behavioral.md index 9c6157ac0..63bf08e50 100644 --- a/skills/behavioral/references/behavioral.md +++ b/skills/behavioral/references/behavioral.md @@ -34,9 +34,13 @@ import type { `useAddHandler`, no `sendTrace`, no generic type parameter: ```ts -const { useAddThread, trigger, useTrace } = behavioral({ instanceId?: string }) +const { useAddThread, trigger, useTrace } = behavioral({ sessionId?: string }) ``` +The optional `sessionId` is host-supplied session identity stamped on every +trace alongside the self-minted `instanceId` (an ACP ingress host owns session +identity policy); absent it defaults to the `instanceId`. + Threads are JSON objects: `{ label: string, rules: Idioms[], once?: true }`. Each idiom is one sync point with `request` (propose an event), `waitFor` (block until an event), `block` (forbid an event), `interrupt` (terminate the diff --git a/src/behavioral/behavioral.ts b/src/behavioral/behavioral.ts index 59782adf5..cbebba82c 100644 --- a/src/behavioral/behavioral.ts +++ b/src/behavioral/behavioral.ts @@ -87,6 +87,12 @@ const createSubject = (): SendTrace => { * and threads to run. If no events can be selected (either because all requests are blocked * or there are no requests), the program will pause until an event is admitted via `trigger`. * + * @param options - Optional factory options. + * @param options.sessionId - Host-supplied session identity stamped on every + * trace alongside `instanceId`. The engine never mints or returns session + * ids — an ingress host (e.g. the ACP host) owns session identity policy. + * Defaults to the self-minted `instanceId` when omitted. + * * **Channel invariant:** a selected event carries `ingress: true` iff it was admitted * externally through `trigger`; everything internal (dispatch-bridge results, transform * targets, `threads.registered`) arrives as a thread request added through `useAddThread`. @@ -97,8 +103,10 @@ const createSubject = (): SendTrace => { * absent = either). `trigger` is therefore external admission plus one super-step, * nothing else. */ -export const behavioral = () => { +export const behavioral = (options?: { sessionId?: string }) => { const instanceId = ueid('bp_') + /** @internal Host session identity — accepted at factory time, never minted. */ + const sessionId = options?.sessionId ?? instanceId /** * @internal * Set of threads that have yielded and are waiting for event selection. @@ -133,6 +141,7 @@ export const behavioral = () => { step: stepId, ingress, instanceId, + sessionId, }) advanceRunningToPending(running, pending) selectNextEvent() @@ -159,6 +168,7 @@ export const behavioral = () => { timestamp: Date.now(), step, instanceId, + sessionId, threads: [...pending].map(({ generator: _, ...rest }) => rest), }) @@ -168,6 +178,7 @@ export const behavioral = () => { timestamp: Date.now(), step, instanceId, + sessionId, ...frontier, }) @@ -185,6 +196,7 @@ export const behavioral = () => { timestamp: Date.now(), step, instanceId, + sessionId, }) } if (frontier.status === FRONTIER_STATUS.idle) { @@ -193,6 +205,7 @@ export const behavioral = () => { timestamp: Date.now(), step, instanceId, + sessionId, }) } } @@ -216,6 +229,7 @@ export const behavioral = () => { kind: TRACE_MESSAGE_KINDS.thread_added, timestamp: Date.now(), instanceId, + sessionId, thread: args, }) } catch (err) { @@ -223,6 +237,7 @@ export const behavioral = () => { kind: TRACE_MESSAGE_KINDS.add_thread_error, timestamp: Date.now(), instanceId, + sessionId, error: [err instanceof Error ? err.message : String(err)], space, }) @@ -232,6 +247,7 @@ export const behavioral = () => { kind: TRACE_MESSAGE_KINDS.add_thread_error, timestamp: Date.now(), instanceId, + sessionId, error: validateThread.errors ?? [], ...(typeof attemptedSpace === 'string' && { space: attemptedSpace }), }) @@ -258,6 +274,7 @@ export const behavioral = () => { pending, sendTrace, instanceId, + sessionId, step: stepId, }) if (transformers.length) { @@ -266,6 +283,7 @@ export const behavioral = () => { timestamp: Date.now(), step: stepId, instanceId, + sessionId, transformers, }) for (const { query, target, thread, space } of transformers) { @@ -284,6 +302,7 @@ export const behavioral = () => { timestamp: Date.now(), step: stepId, instanceId, + sessionId, transformer: { query, target, thread, space }, reason: result.reason, ...(result.stderr !== undefined && { stderr: result.stderr }), @@ -297,6 +316,7 @@ export const behavioral = () => { timestamp: Date.now(), step: stepId, instanceId, + sessionId, selected: selectedEvent, }) /** @@ -326,6 +346,7 @@ export const behavioral = () => { kind: TRACE_MESSAGE_KINDS.trigger_error, timestamp: Date.now(), instanceId, + sessionId, error: validateBPEvent.errors ?? [], ...(typeof attemptedSpace === 'string' ? { space: attemptedSpace } : {}), }) diff --git a/src/behavioral/behavioral.types.ts b/src/behavioral/behavioral.types.ts index 216d751d9..e14159011 100644 --- a/src/behavioral/behavioral.types.ts +++ b/src/behavioral/behavioral.types.ts @@ -319,14 +319,39 @@ export type Threads = Thread[] * `TRACE_MESSAGE_KINDS` so narrowing by `kind` remains unambiguous in the * unified `Trace | T` stream. * + * The two id axes are separate and both live on the wire: `instanceId` is the + * per-process identity the engine self-mints; `sessionId` is the host's + * session identity (an ACP/ingress host mints and loads sessions), defaulted + * to the `instanceId` when no host supplies one. The engine accepts a session + * id at factory time — it never mints one and never returns ids. + * + * @see {@link TraceBaseSchema} for the runtime (JSON-schema) mirror * @see {@link Trace} for the engine's closed trace union */ type TraceBase = { kind: string timestamp: number instanceId: string + sessionId: string } +/** + * Wire schema for the fields every trace carries — the runtime mirror of + * {@link TraceBase}. The one home for the trace wire's common shape: per-kind + * trace validators derive from this (spread the properties, extend `required`) + * instead of hand-mirroring the fields. + */ +export const TraceBaseSchema = { + type: 'object', + properties: { + kind: { type: 'string' }, + timestamp: { type: 'number' }, + instanceId: { type: 'string' }, + sessionId: { type: 'string' }, + }, + required: ['kind', 'timestamp', 'instanceId', 'sessionId'], +} as const + // --------------------------------------------------------------------------- // Trace kinds // --------------------------------------------------------------------------- diff --git a/src/behavioral/behavioral.utils.ts b/src/behavioral/behavioral.utils.ts index 728817f7c..731287845 100644 --- a/src/behavioral/behavioral.utils.ts +++ b/src/behavioral/behavioral.utils.ts @@ -119,6 +119,7 @@ export const resumePendingThreadsForSelectedEvent = ({ selectedEvent, sendTrace, instanceId, + sessionId, step, }: { running: Set @@ -126,6 +127,7 @@ export const resumePendingThreadsForSelectedEvent = ({ selectedEvent: CandidateBid sendTrace?: SendTrace instanceId: string + sessionId: string step: number }) => { const transformers: Transformer[] = [] @@ -147,6 +149,7 @@ export const resumePendingThreadsForSelectedEvent = ({ timestamp: Date.now(), step, instanceId, + sessionId, selected: selectedEvent, threadLabel: label, }) diff --git a/src/behavioral/tests/schemas.spec.ts b/src/behavioral/tests/schemas.spec.ts index 3d2e9b8f8..60aa25418 100644 --- a/src/behavioral/tests/schemas.spec.ts +++ b/src/behavioral/tests/schemas.spec.ts @@ -1,17 +1,20 @@ import { describe, expect, test } from 'bun:test' -import { ajv, validateBPEvent, validateThread, validateTransformEvaluation } from '../behavioral.types.ts' +import { + ajv, + TraceBaseSchema, + validateBPEvent, + validateThread, + validateTransformEvaluation, +} from '../behavioral.types.ts' +// Derived from TraceBaseSchema — the one home for the trace wire's common +// shape — extended per-kind with the discriminating `kind` and `step`. const compileTraceValidator = (kind: string) => ajv.compile({ - type: 'object', - properties: { - kind: { const: kind }, - timestamp: { type: 'number' }, - instanceId: { type: 'string' }, - step: { type: 'integer' }, - }, - required: ['kind', 'timestamp', 'instanceId', 'step'], + ...TraceBaseSchema, + properties: { ...TraceBaseSchema.properties, kind: { const: kind }, step: { type: 'integer' } }, + required: [...TraceBaseSchema.required, 'step'], }) describe('behavioral schemas', () => { @@ -65,6 +68,7 @@ describe('behavioral schemas', () => { kind: 'selection', timestamp: 3, instanceId: 'bp_test', + sessionId: 'sess_test', step: 3, selected: { type: 'event', detail: { value: 1 } }, } @@ -73,6 +77,18 @@ describe('behavioral schemas', () => { expect(narrowed.selected.type).toBe('event') }) + test('Trace validators reject missing sessionId', () => { + expect( + compileTraceValidator('selection')({ + kind: 'selection', + timestamp: 0, + instanceId: 'bp_test', + step: 0, + selected: { type: 'event' }, + }), + ).toBe(false) + }) + test('Trace validators reject unknown kinds and missing step', () => { expect(compileTraceValidator('selection')({ kind: 'worker', response: { id: 'worker-1' }, step: 0 })).toBe(false) expect(compileTraceValidator('deadlock')({ kind: 'deadlock', timestamp: 0, instanceId: 'bp_test' })).toBe(false) diff --git a/src/behavioral/tests/session-id.spec.ts b/src/behavioral/tests/session-id.spec.ts new file mode 100644 index 000000000..97f5e39fb --- /dev/null +++ b/src/behavioral/tests/session-id.spec.ts @@ -0,0 +1,37 @@ +import { describe, expect, test } from 'bun:test' + +import { behavioral } from '../behavioral.ts' +import type { Trace } from '../behavioral.types.ts' + +const runProgram = (options?: Parameters[0]): { traces: Trace[]; instanceId: string } => { + const traces: Trace[] = [] + const program = behavioral(options) + program.useTrace((trace) => { + traces.push(trace) + }) + program.addThread({ label: 'greeter', once: true, rules: [{ request: { type: 'hello' } }] }) + program.trigger({ type: 'wake' }) + const first = traces[0] as Trace | undefined + return { traces, instanceId: first?.instanceId ?? '' } +} + +describe('session id wiring', () => { + test('a host-supplied sessionId is stamped on every trace alongside instanceId', () => { + const { traces, instanceId } = runProgram({ sessionId: 'sess_host_1' }) + expect(traces.length).toBeGreaterThan(0) + for (const trace of traces) { + expect(trace.sessionId).toBe('sess_host_1') + expect(trace.instanceId).toBe(instanceId) + expect(trace.instanceId).not.toBe('sess_host_1') + } + }) + + test('without a host session id, every trace defaults sessionId to the instanceId', () => { + const { traces, instanceId } = runProgram() + expect(instanceId).not.toBe('') + expect(traces.length).toBeGreaterThan(0) + for (const trace of traces) { + expect(trace.sessionId).toBe(instanceId) + } + }) +}) diff --git a/src/cli/tests/serve.spec.ts b/src/cli/tests/serve.spec.ts index 3c674bbb2..ebe649d35 100644 --- a/src/cli/tests/serve.spec.ts +++ b/src/cli/tests/serve.spec.ts @@ -36,6 +36,7 @@ const selectionOf = (selected: { type: string; detail?: JsonObject; space?: stri kind: TRACE_MESSAGE_KINDS.selection, timestamp: 0, instanceId: 'i', + sessionId: 'i', step: 1, selected: { priority: 0, ...selected }, }) diff --git a/src/cli/tests/trace-consumer.spec.ts b/src/cli/tests/trace-consumer.spec.ts index c2ee55fcf..28e645765 100644 --- a/src/cli/tests/trace-consumer.spec.ts +++ b/src/cli/tests/trace-consumer.spec.ts @@ -10,6 +10,7 @@ const selection = (detail: JsonObject, space?: string): SelectionTrace => ({ kind: TRACE_MESSAGE_KINDS.selection, timestamp: 0, instanceId: 'i', + sessionId: 'i', step: 1, selected: { priority: 0, type: 'shell_request', detail, ...(space === undefined ? {} : { space }) }, }) diff --git a/src/faculties/frontier/faculty.ts b/src/faculties/frontier/faculty.ts index 052c2b479..173bb4329 100644 --- a/src/faculties/frontier/faculty.ts +++ b/src/faculties/frontier/faculty.ts @@ -67,14 +67,17 @@ const createFrontierTrace = ({ frontier, step, instanceId, + sessionId, }: { frontier: Frontier step: number instanceId: string + sessionId: string }): FrontierTrace => ({ kind: 'frontier', timestamp: Date.now(), instanceId, + sessionId, step, status: frontier.status, candidates: frontier.candidates.map((candidate) => ({ @@ -97,22 +100,34 @@ const createSelectionTrace = ({ selected, step, instanceId, + sessionId, }: { selected: CandidateBid step: number instanceId: string + sessionId: string }): SelectionTrace => ({ kind: TRACE_MESSAGE_KINDS.selection, timestamp: Date.now(), instanceId, + sessionId, step, selected, }) -const createDeadlockTrace = ({ step, instanceId }: { step: number; instanceId: string }): Trace => ({ +const createDeadlockTrace = ({ + step, + instanceId, + sessionId, +}: { + step: number + instanceId: string + sessionId: string +}): Trace => ({ kind: TRACE_MESSAGE_KINDS.deadlock, timestamp: Date.now(), instanceId, + sessionId, step, }) @@ -221,6 +236,8 @@ type DeadlockFinding = { * @param args.space - Optional space stamp applied to all thread rules. * @param args.instanceId - Instance id stamped on synthetic interrupt/transform * traces emitted during resumption. Defaults to a minted `ueid('bp_')`. + * @param args.sessionId - Host session id stamped on the same traces alongside + * `instanceId`. Defaults to the `instanceId` — the faculty never mints one. * @returns The replay result containing the pending set and final frontier. * * @throws If a selection event is not enabled at its replay step. @@ -232,12 +249,15 @@ const replayToFrontierRaw = ({ messages = [], space, instanceId = ueid('bp_'), + sessionId, }: { threads: Thread[] messages?: Trace[] space?: string instanceId?: string + sessionId?: string }): ReplayToFrontierResult => { + const resolvedSessionId = sessionId ?? instanceId const pending = new Set() const running = new Set() @@ -286,6 +306,7 @@ const replayToFrontierRaw = ({ pending, selectedEvent: matched, instanceId, + sessionId: resolvedSessionId, step, }) advanceRunningToPending(resumed, pending) @@ -340,11 +361,13 @@ const getRequestSuccessors = ({ selectionPolicy, step, instanceId, + sessionId, }: { frontier: Frontier selectionPolicy: 'all-enabled' | 'scheduler' step: number instanceId: string + sessionId: string }) => { if (frontier.status !== FRONTIER_STATUS.ready) { return [] @@ -355,7 +378,7 @@ const getRequestSuccessors = ({ ? [...frontier.enabled].sort((left, right) => left.priority - right.priority).slice(0, 1) : frontier.enabled - return enabled.map((candidate) => createSelectionTrace({ selected: candidate, step, instanceId })) + return enabled.map((candidate) => createSelectionTrace({ selected: candidate, step, instanceId, sessionId })) } const getTriggerSuccessors = ({ @@ -366,6 +389,7 @@ const getTriggerSuccessors = ({ triggers, space, instanceId, + sessionId, }: { pending: Set messages: Trace[] @@ -374,6 +398,7 @@ const getTriggerSuccessors = ({ triggers: BPEvent[] space?: string instanceId: string + sessionId: string }) => { const successors: SelectionTrace[] = [] @@ -399,6 +424,7 @@ const getTriggerSuccessors = ({ const selection = createSelectionTrace({ step, instanceId, + sessionId, selected: { priority: 0, type: trigger.type, @@ -414,6 +440,7 @@ const getTriggerSuccessors = ({ messages: [...messages, selection], space, instanceId, + sessionId, }) successors.push(selection) } catch { @@ -710,6 +737,8 @@ type ExploreFrontiersArgs = { space?: string /** Instance id stamped on synthetic traces. Defaults to a minted `ueid('bp_')` — pass the analyzed kernel's id to make joins natural. */ instanceId?: string + /** Host session id stamped on synthetic traces alongside `instanceId`. Defaults to the `instanceId` — the faculty never mints one. */ + sessionId?: string } /** @@ -755,11 +784,14 @@ const exploreFrontiersRaw = ({ maxDepth, space, instanceId = ueid('bp_'), + sessionId, }: ExploreFrontiersArgs): ExploreFrontiersResult => { if (strategy !== 'bfs' && strategy !== 'dfs') { throw new Error(`Unsupported frontier exploration strategy "${String(strategy)}".`) } + const resolvedSessionId = sessionId ?? instanceId + const pending: WorkItem[] = [{ messages }] const visited = new Set() const stateGraph = new Map() @@ -774,6 +806,7 @@ const exploreFrontiersRaw = ({ messages: current.messages, space, instanceId, + sessionId: resolvedSessionId, }) const stateKey = frontierStateKey({ pending: currentPending }) @@ -796,7 +829,7 @@ const exploreFrontiersRaw = ({ successors: [], }) - const frontierTrace = createFrontierTrace({ frontier, step, instanceId }) + const frontierTrace = createFrontierTrace({ frontier, step, instanceId, sessionId: resolvedSessionId }) traces.push({ messages: [...current.messages, frontierTrace], @@ -807,6 +840,7 @@ const exploreFrontiersRaw = ({ selectionPolicy, step, instanceId, + sessionId: resolvedSessionId, }) const triggerSuccessors = getTriggerSuccessors({ pending: currentPending, @@ -816,13 +850,18 @@ const exploreFrontiersRaw = ({ triggers, space, instanceId, + sessionId: resolvedSessionId, }) const successors = [...requestSuccessors, ...triggerSuccessors] if (frontier.status === FRONTIER_STATUS.deadlock && triggerSuccessors.length === 0) { findings.push({ code: 'deadlock', - messages: [...current.messages, frontierTrace, createDeadlockTrace({ step, instanceId })], + messages: [ + ...current.messages, + frontierTrace, + createDeadlockTrace({ step, instanceId, sessionId: resolvedSessionId }), + ], }) } @@ -958,6 +997,7 @@ export type FrontierReplayInput = { messages?: SelectionTrace[] space?: string instanceId?: string + sessionId?: string } export type FrontierReplayOutput = { @@ -979,6 +1019,11 @@ export const FrontierReplayInputSchema = { nullable: true, description: 'instance id stamped on synthetic traces; defaults to a minted ueid("bp_")', }, + sessionId: { + type: 'string', + nullable: true, + description: 'host session id stamped on synthetic traces; defaults to the instanceId', + }, }, required: ['threads'], additionalProperties: false, @@ -1033,6 +1078,7 @@ export type FrontierExploreInput = { maxDepth: number space?: string instanceId?: string + sessionId?: string } export type FrontierExploreOutput = { @@ -1082,6 +1128,11 @@ export const FrontierExploreInputSchema = { nullable: true, description: 'instance id stamped on synthetic traces; defaults to a minted ueid("bp_")', }, + sessionId: { + type: 'string', + nullable: true, + description: 'host session id stamped on synthetic traces; defaults to the instanceId', + }, }, required: ['threads', 'maxDepth'], additionalProperties: false, @@ -1110,6 +1161,7 @@ export type FrontierVerifyInput = { progress?: string[] space?: string instanceId?: string + sessionId?: string } export type FrontierVerifyOutput = { @@ -1166,6 +1218,11 @@ export const FrontierVerifyInputSchema = { nullable: true, description: 'instance id stamped on synthetic traces; defaults to a minted ueid("bp_")', }, + sessionId: { + type: 'string', + nullable: true, + description: 'host session id stamped on synthetic traces; defaults to the instanceId', + }, }, required: ['threads', 'maxDepth'], additionalProperties: false, @@ -1218,9 +1275,9 @@ const OP_RUNNERS: Record = { replay: { validate: validateReplayInput, errors: () => ajv.errorsText(validateReplayInput.errors), - run: ({ threads, messages, space, instanceId }: FrontierReplayInput): FrontierReplayOutput => { + run: ({ threads, messages, space, instanceId, sessionId }: FrontierReplayInput): FrontierReplayOutput => { try { - const { pending, frontier } = replayToFrontierRaw({ threads, messages, space, instanceId }) + const { pending, frontier } = replayToFrontierRaw({ threads, messages, space, instanceId, sessionId }) return { frontier, stateKey: frontierStateKey({ pending }), pendingCount: pending.size } } catch (err) { return { @@ -1245,6 +1302,7 @@ const OP_RUNNERS: Record = { maxDepth, space, instanceId, + sessionId, }: FrontierExploreInput): FrontierExploreOutput => { try { const { traces, findings, report, stateGraph } = exploreFrontiersRaw({ @@ -1256,6 +1314,7 @@ const OP_RUNNERS: Record = { maxDepth, space, instanceId, + sessionId, }) return { traces, findings, report, stateGraph: serializeStateGraph(stateGraph) } } catch (err) { @@ -1290,6 +1349,7 @@ const OP_RUNNERS: Record = { progress, space, instanceId, + sessionId, }: FrontierVerifyInput): FrontierVerifyOutput => { try { const { status, findings, report, livelocks } = verifyFrontiersRaw({ @@ -1302,6 +1362,7 @@ const OP_RUNNERS: Record = { progress, space, instanceId, + sessionId, }) return { status, findings, report, livelocks } } catch (err) { diff --git a/src/faculties/frontier/tests/faculty.spec.ts b/src/faculties/frontier/tests/faculty.spec.ts index 28bdff7e3..9133cff82 100644 --- a/src/faculties/frontier/tests/faculty.spec.ts +++ b/src/faculties/frontier/tests/faculty.spec.ts @@ -65,7 +65,9 @@ type ReplayResult = { message?: string } type ExploreResult = { - traces: Array<{ messages: Array<{ kind: string; selected?: { type: string } }> }> + traces: Array<{ + messages: Array<{ kind: string; selected?: { type: string }; instanceId?: string; sessionId?: string }> + }> findings: Array<{ code: string }> report: { visitedCount: number; findingCount: number; truncated: boolean } stateGraph: Record }> @@ -309,6 +311,43 @@ describe('explore', () => { } }) + test('stamps the host sessionId on every synthetic trace, defaulting to the instanceId', async () => { + const frontier = spawnFrontierWorker() + try { + // Host-supplied: both id axes ride the trace wire — instanceId per-process, + // sessionId the host's session identity. + frontier.call('e1', 'explore', { + threads, + strategy: 'bfs', + maxDepth: 3, + instanceId: 'test', + sessionId: 'sess_host', + }) + const { result } = await frontier.resultFor('e1') + const explore = result as ExploreResult + expect(explore.isError).toBeFalsy() + expect(explore.traces.length).toBeGreaterThan(0) + for (const record of explore.traces) { + for (const message of record.messages) { + expect(message.sessionId).toBe('sess_host') + expect(message.instanceId).toBe('test') + } + } + + // Absent sessionId defaults to the instanceId — same treatment as the + // engine, no drift between the faculty's schemas and TraceBase. + frontier.call('e2', 'explore', { threads, strategy: 'bfs', maxDepth: 3, instanceId: 'test' }) + const defaulted = await frontier.resultFor('e2') + for (const record of (defaulted.result as ExploreResult).traces) { + for (const message of record.messages) { + expect(message.sessionId).toBe('test') + } + } + } finally { + frontier.terminate() + } + }) + test('finds deadlock', async () => { const frontier = spawnFrontierWorker() try { From dc5fde4244294bd3578df4bddbd60ef6483f260c Mon Sep 17 00:00:00 2001 From: Edward Irby Date: Wed, 23 Sep 2026 23:32:23 -0700 Subject: [PATCH 02/55] docs: drop stale ACP and Tauri references from comments MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The direction was dropped from the plan; these comment-only references survived the sweep. No behavior change — doc lines and one doc-comment re-wrap only. - behavioral.ts / behavioral.types.ts / skill reference: session identity is owned by "the host layer", no protocol names - controller.types.ts: Transport seam examples now generic ("native IPC") - store/faculty.ts: swap-backings list drops the Tauri engine example Validation: bun --bun tsc --noEmit clean; comment-only changes, no executable surface touched. --- skills/behavioral/references/behavioral.md | 2 +- src/behavioral/behavioral.ts | 2 +- src/behavioral/behavioral.types.ts | 2 +- src/controller/controller.types.ts | 4 ++-- src/faculties/store/faculty.ts | 5 ++--- 5 files changed, 7 insertions(+), 8 deletions(-) diff --git a/skills/behavioral/references/behavioral.md b/skills/behavioral/references/behavioral.md index 63bf08e50..6a8d030e9 100644 --- a/skills/behavioral/references/behavioral.md +++ b/skills/behavioral/references/behavioral.md @@ -38,7 +38,7 @@ const { useAddThread, trigger, useTrace } = behavioral({ sessionId?: string }) ``` The optional `sessionId` is host-supplied session identity stamped on every -trace alongside the self-minted `instanceId` (an ACP ingress host owns session +trace alongside the self-minted `instanceId` (the host layer owns session identity policy); absent it defaults to the `instanceId`. Threads are JSON objects: `{ label: string, rules: Idioms[], once?: true }`. diff --git a/src/behavioral/behavioral.ts b/src/behavioral/behavioral.ts index cbebba82c..04dde12d7 100644 --- a/src/behavioral/behavioral.ts +++ b/src/behavioral/behavioral.ts @@ -90,7 +90,7 @@ const createSubject = (): SendTrace => { * @param options - Optional factory options. * @param options.sessionId - Host-supplied session identity stamped on every * trace alongside `instanceId`. The engine never mints or returns session - * ids — an ingress host (e.g. the ACP host) owns session identity policy. + * ids — the host layer owns session identity policy. * Defaults to the self-minted `instanceId` when omitted. * * **Channel invariant:** a selected event carries `ingress: true` iff it was admitted diff --git a/src/behavioral/behavioral.types.ts b/src/behavioral/behavioral.types.ts index e14159011..64b17f1a5 100644 --- a/src/behavioral/behavioral.types.ts +++ b/src/behavioral/behavioral.types.ts @@ -321,7 +321,7 @@ export type Threads = Thread[] * * The two id axes are separate and both live on the wire: `instanceId` is the * per-process identity the engine self-mints; `sessionId` is the host's - * session identity (an ACP/ingress host mints and loads sessions), defaulted + * session identity (the host mints and manages session ids), defaulted * to the `instanceId` when no host supplies one. The engine accepts a session * id at factory time — it never mints one and never returns ids. * diff --git a/src/controller/controller.types.ts b/src/controller/controller.types.ts index 35b931058..deae9b784 100644 --- a/src/controller/controller.types.ts +++ b/src/controller/controller.types.ts @@ -135,7 +135,7 @@ export type ControllerConstructorArgs = { * Optional message carrier. When omitted the controller uses its built-in * WebSocket carrier (byte-for-byte the pre-seam behavior). When provided, * the controller sends/receives through it instead of opening a WebSocket — - * the injection point a non-WS carrier (e.g. Tauri IPC) plugs into. + * the injection point a non-WS carrier (e.g. native IPC) plugs into. */ transport?: Transport } @@ -358,7 +358,7 @@ export type TransportEvent = /** * The controller's message carrier — the seam a non-WebSocket transport - * (Tauri IPC, etc.) plugs into without touching controller logic. + * (native IPC, etc.) plugs into without touching controller logic. * * @remarks * The controller sends outgoing {@link ClientMessage}s via `send`, registers diff --git a/src/faculties/store/faculty.ts b/src/faculties/store/faculty.ts index 255071299..68a349cf7 100644 --- a/src/faculties/store/faculty.ts +++ b/src/faculties/store/faculty.ts @@ -17,9 +17,8 @@ * * **The sqlite schema is worker-internal.** Only JSON ops cross the wire — * no SQL, no expressions — so backings stay swappable per host (bun:sqlite - * here, sql.js or a Rust engine under Tauri, IndexedDB in a browser) without - * protocol change. The backing is one owned connection for the worker's - * lifetime (WAL for file dbs), with version-stamped migrations on boot. + * here, sql.js, IndexedDB in a browser) without protocol change. The backing + * is one owned connection for the worker's lifetime (WAL for file dbs), with version-stamped migrations on boot. * * **Not the authority surface:** threads/html learning stays files+git * (2026-09-17 growth-model decision); this store is regenerable index + From b372c359f7f8746afd9e20f2eaf71460730c83af Mon Sep 17 00:00:00 2001 From: Edward Irby Date: Wed, 23 Sep 2026 23:43:51 -0700 Subject: [PATCH 03/55] =?UTF-8?q?feat:=20instance=20lock=20=E2=80=94=20pid?= =?UTF-8?q?file=20liveness=20under=20?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Slice 1 of the attach-or-start lifecycle: the running instance owns /instance.pid. acquireInstanceLock takes the lock when the home is free, blocks with the live holder pid when one runs, and reaps a stale pidfile (pid no longer alive) before acquiring. Injectable home + pid; release removes the pidfile for the SIGINT/SIGTERM terminate path. --- src/faculties/instance-lock.ts | 77 +++++++++++++++++++++++ src/faculties/tests/instance-lock.spec.ts | 70 +++++++++++++++++++++ 2 files changed, 147 insertions(+) create mode 100644 src/faculties/instance-lock.ts create mode 100644 src/faculties/tests/instance-lock.spec.ts diff --git a/src/faculties/instance-lock.ts b/src/faculties/instance-lock.ts new file mode 100644 index 000000000..d13e43a20 --- /dev/null +++ b/src/faculties/instance-lock.ts @@ -0,0 +1,77 @@ +import { existsSync, readFileSync, rmSync, writeFileSync } from 'node:fs' +import { join } from 'node:path' + +/** + * The instance pidfile — `/instance.pid`, the single-instance lock. + * + * @remarks + * The running instance owns `/instance.pid` for its whole lifetime; the + * launcher (attach-or-start) reads it to detect a live instance and attaches + * instead of starting a second one. A pidfile whose pid is no longer alive is + * stale: acquisition reaps it and takes the lock. The pidfile is deleted on + * release (SIGINT/SIGTERM terminate path). + * + * @public + */ +export const instancePidfilePath = (home: string): string => join(home, 'instance.pid') + +/** True when a process with this pid exists (signal 0 probes liveness). */ +const pidAlive = (pid: number): boolean => { + try { + process.kill(pid, 0) + return true + } catch (error) { + // EPERM means the process exists but is not signalable by us — still alive. + return (error as NodeJS.ErrnoException).code === 'EPERM' + } +} + +/** + * The acquired instance lock — `release()` removes the pidfile. + * + * @public + */ +export type InstanceLock = { acquired: true; release: () => void } + +/** + * The lock is held: a live instance owns the home; `pid` is its pid. + * + * @public + */ +export type InstanceLockHeld = { acquired: false; pid: number } + +/** + * Acquire the single-instance lock under `home`: write `instance.pid` if the + * home is free (no pidfile, or a stale one whose pid is dead — reaped first). + * + * @param home - The harness home (the single `` root). + * @param pid - The pid to record; defaults to the current process. + * @returns The lock (with its `release`) when acquired, or the live pid that + * holds it. + * + * @public + */ +export const acquireInstanceLock = ({ + home, + pid = process.pid, +}: { + home: string + pid?: number +}): InstanceLock | InstanceLockHeld => { + const path = instancePidfilePath(home) + if (existsSync(path)) { + let holder: number | undefined + try { + const parsed: unknown = JSON.parse(readFileSync(path, 'utf8')) + if (typeof parsed === 'object' && parsed !== null && typeof (parsed as { pid?: unknown }).pid === 'number') { + holder = (parsed as { pid: number }).pid + } + } catch { + // Unreadable pidfile: treat as stale, reaped below. + } + if (holder !== undefined && pidAlive(holder)) return { acquired: false, pid: holder } + rmSync(path, { force: true }) + } + writeFileSync(path, JSON.stringify({ pid })) + return { acquired: true, release: () => rmSync(path, { force: true }) } +} diff --git a/src/faculties/tests/instance-lock.spec.ts b/src/faculties/tests/instance-lock.spec.ts new file mode 100644 index 000000000..955fefb05 --- /dev/null +++ b/src/faculties/tests/instance-lock.spec.ts @@ -0,0 +1,70 @@ +import { describe, expect, test } from 'bun:test' +import { existsSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { acquireInstanceLock, type InstanceLock, instancePidfilePath } from '../instance-lock.ts' + +const tempHome = (): string => mkdtempSync(join(tmpdir(), 'behavioral-instance-lock-')) + +/** Assert acquisition succeeded, returning the narrowed lock. */ +const acquired = (lock: InstanceLock | { acquired: false; pid: number }): InstanceLock => { + expect(lock.acquired).toBe(true) + if (!lock.acquired) throw new Error(`expected acquisition, blocked by pid ${lock.pid}`) + return lock +} + +describe('acquireInstanceLock', () => { + test('acquiring a free home writes the pidfile with the given pid', () => { + const home = tempHome() + try { + const lock = acquired(acquireInstanceLock({ home, pid: 4242 })) + const path = instancePidfilePath(home) + expect(existsSync(path)).toBe(true) + expect(JSON.parse(readFileSync(path, 'utf8'))).toEqual({ pid: 4242 }) + lock.release() + } finally { + rmSync(home, { recursive: true, force: true }) + } + }) + + test('a pidfile held by a live pid blocks acquisition and reports the pid', () => { + const home = tempHome() + try { + acquired(acquireInstanceLock({ home, pid: process.pid })) + const second = acquireInstanceLock({ home, pid: 1 }) + expect(second.acquired).toBe(false) + if (!second.acquired) expect(second.pid).toBe(process.pid) + } finally { + rmSync(home, { recursive: true, force: true }) + } + }) + + test('a stale pidfile (pid no longer alive) is reaped and acquisition succeeds', () => { + const home = tempHome() + const path = instancePidfilePath(home) + // A pid that cannot exist: high fixed value, no process holds it. EPERM + // would mean "alive but unkillable", so assert the kill probe fails first. + const deadPid = 999_999_999 + // Guard the assumption: the pid does not exist (ESRCH), so it is stale. + try { + process.kill(deadPid, 0) + } catch (error) { + expect((error as NodeJS.ErrnoException).code).toBe('ESRCH') + } + writeFileSync(path, JSON.stringify({ pid: deadPid })) + const lock = acquired(acquireInstanceLock({ home, pid: process.pid })) + expect(JSON.parse(readFileSync(path, 'utf8'))).toEqual({ pid: process.pid }) + lock.release() + }) + + test('release removes the pidfile', () => { + const home = tempHome() + try { + const lock = acquired(acquireInstanceLock({ home, pid: process.pid })) + lock.release() + expect(existsSync(instancePidfilePath(home))).toBe(false) + } finally { + rmSync(home, { recursive: true, force: true }) + } + }) +}) From a2249838b4ea5bf86b192950f8571f758135c0cd Mon Sep 17 00:00:00 2001 From: Edward Irby Date: Wed, 23 Sep 2026 23:51:10 -0700 Subject: [PATCH 04/55] =?UTF-8?q?feat:=20the=20attach=20lane=20=E2=80=94?= =?UTF-8?q?=20unix-socket=20carrier=20for=20the=20host=20dispatcher?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Slice 2: the host gains its single listener, /instance.sock — a Bun.serve with the unix option whose WebSocket carrier speaks the same JSON-RPC lane as stdio (one text frame = one message). The host-side dispatcher and egress wiring are extracted from serve.ts (dispatchToRuntime, wireRuntimeEgress) and reused, one protocol one dispatcher multiple carriers. Redacted traces and ui_* selections fan out to every client; socket file removed on close, stale file reaped at start. Transport-shaped clients exercise the real server over ws+unix in the specs. --- src/cli/serve.ts | 55 +++++--- src/cli/socket-host.ts | 137 ++++++++++++++++++++ src/cli/tests/socket-host.spec.ts | 209 ++++++++++++++++++++++++++++++ 3 files changed, 386 insertions(+), 15 deletions(-) create mode 100644 src/cli/socket-host.ts create mode 100644 src/cli/tests/socket-host.spec.ts diff --git a/src/cli/serve.ts b/src/cli/serve.ts index a77637225..07f260dd5 100644 --- a/src/cli/serve.ts +++ b/src/cli/serve.ts @@ -14,8 +14,14 @@ import { collectSecretValues, createTraceConsumer, traceLogSink } from './trace- */ export type HostRuntime = Pick, 'trigger' | 'useTrace' | 'start' | 'terminate'> -/** Map one inbound JSON-RPC message onto the engine. */ -const dispatch = (runtime: HostRuntime, message: JsonRpcMessage): unknown => { +/** + * Map one inbound JSON-RPC message onto the engine — the ONE host-side + * dispatcher. Every carrier (stdio lane, unix socket, later the controller + * WebSocket) reuses it: one protocol, one dispatcher, multiple carriers. + * + * @public + */ +export const dispatchToRuntime = (runtime: HostRuntime, message: JsonRpcMessage): unknown => { const { method, params } = message if (method === 'trigger') { runtime.trigger((params as { event: BPEvent }).event) @@ -33,6 +39,36 @@ const dispatch = (runtime: HostRuntime, message: JsonRpcMessage): unknown => { throw new Error(`unknown method: ${method}`) } +/** + * Wire the engine's egress to the JSONL trace log plus one carrier sink: + * redacted traces flow as `trace` emissions, `ui_*` selections as their own + * `` emissions with the selection detail. Shared by every carrier. + * + * @public + */ +export const wireRuntimeEgress = ({ + runtime, + home, + emit, +}: { + runtime: HostRuntime + home: string + emit: (method: string, params: unknown) => void +}): void => { + const consumer = createTraceConsumer({ + secrets: collectSecretValues(), + sinks: [traceLogSink({ root: join(home, 'traces') }), (trace) => emit('trace', trace)], + }) + runtime.useTrace(consumer) + + // Egress-as-selection: a `ui_*` selection becomes a client notification. + runtime.useTrace((trace) => { + if (trace.kind !== TRACE_MESSAGE_KINDS.selection) return + const selected = trace.selected + if (selected.type.startsWith('ui_')) emit(selected.type, selected.detail) + }) +} + /** * Wire the JSON-RPC codec to a runtime: ingress messages become triggers, `ui_*` * selections become client notifications, and redacted traces fan out to a JSONL @@ -51,21 +87,10 @@ export const createHost = ({ write: (line: string) => void home?: string }): { rpc: JsonRpcServer } => { - const rpc = createJsonRpcServer({ input, write, onMessage: (message) => dispatch(runtime, message) }) + const rpc = createJsonRpcServer({ input, write, onMessage: (message) => dispatchToRuntime(runtime, message) }) // Observability: redacted traces to the JSONL log and the client. - const consumer = createTraceConsumer({ - secrets: collectSecretValues(), - sinks: [traceLogSink({ root: join(home, 'traces') }), (trace) => rpc.notify('trace', trace)], - }) - runtime.useTrace(consumer) - - // Egress-as-selection: a `ui_*` selection becomes a client notification. - runtime.useTrace((trace) => { - if (trace.kind !== TRACE_MESSAGE_KINDS.selection) return - const selected = trace.selected - if (selected.type.startsWith('ui_')) rpc.notify(selected.type, selected.detail) - }) + wireRuntimeEgress({ runtime, home, emit: rpc.notify }) runtime.start() rpc.notify('ready') diff --git a/src/cli/socket-host.ts b/src/cli/socket-host.ts new file mode 100644 index 000000000..670d07695 --- /dev/null +++ b/src/cli/socket-host.ts @@ -0,0 +1,137 @@ +import { rmSync } from 'node:fs' +import { join } from 'node:path' +import type { ServerWebSocket } from 'bun' +import { behavioralHome } from '../faculties/behavioral-home.ts' +import type { JsonRpcMessage } from './json-rpc.ts' +import { dispatchToRuntime, type HostRuntime, wireRuntimeEgress } from './serve.ts' + +/** + * The instance socket — `/instance.sock`, the attach lane. + * + * @remarks + * The host's single listener: a `Bun.serve` bound to this unix path speaks the + * same line-framed JSON-RPC vocabulary as the stdio lane (one WebSocket text + * message = one JSON-RPC frame), so an attacher is just another client of the + * controller contract. The same server later serves the controller GUI over + * HTTP on the same listener (Carriers/H). Written at start, removed on close — + * the same lifecycle as the instance pidfile. + * + * @public + */ +export const instanceSocketPath = (home: string): string => join(home, 'instance.sock') + +/** + * The running socket host — `close()` stops the server and removes the socket + * file (the terminate-path cleanup). + * + * @public + */ +export type SocketHost = { + path: string + close: () => Promise +} + +/** + * Start the attach lane: a unix-socket `Bun.serve` over the shared host + * dispatcher, with redacted traces and `ui_*` selections fanning out to every + * connected client. + * + * @remarks + * Unlike {@link createHost}, this does NOT call `runtime.start()` — the + * foreground entry composes the runtime, the socket host, and its clients, + * then starts the composition itself. + * + * @public + */ +export const createSocketHost = ({ + runtime, + home = behavioralHome(), +}: { + runtime: HostRuntime + home?: string +}): SocketHost => { + const path = instanceSocketPath(home) + // A socket file left by a dead instance cannot be bound again — remove it. + rmSync(path, { force: true }) + + const clients = new Set>() + const frame = (method: string, params: unknown): string => JSON.stringify({ jsonrpc: '2.0', method, params }) + + const server = Bun.serve({ + unix: path, + // MINIMAL: ws idleTimeout max is 255s; long-lived attaches get the ceiling + // until a heartbeat/reconnect story is needed. + websocket: { + idleTimeout: 255, + open: (ws) => { + clients.add(ws) + }, + message: (ws, message) => { + const line = typeof message === 'string' ? message : new TextDecoder().decode(message) + let parsed: JsonRpcMessage + try { + const value: unknown = JSON.parse(line) + if (typeof value !== 'object' || value === null || typeof (value as JsonRpcMessage).method !== 'string') { + throw new Error('not a JSON-RPC message') + } + parsed = value as JsonRpcMessage + } catch { + ws.send(JSON.stringify({ jsonrpc: '2.0', id: null, error: { code: -32700, message: 'Parse error' } })) + return + } + if (parsed.id === undefined) { + // A notification has no response channel; a handler failure must not + // reject the event loop and kill the host. + try { + void dispatchToRuntime(runtime, parsed) + } catch (error) { + process.stderr.write( + `instance socket notification '${parsed.method}' failed: ${ + error instanceof Error ? error.message : String(error) + }\n`, + ) + } + return + } + try { + const result = dispatchToRuntime(runtime, parsed) + ws.send(JSON.stringify({ jsonrpc: '2.0', id: parsed.id, result })) + } catch (error) { + ws.send( + JSON.stringify({ + jsonrpc: '2.0', + id: parsed.id, + error: { code: -32603, message: error instanceof Error ? error.message : String(error) }, + }), + ) + } + }, + close: (ws) => { + clients.delete(ws) + }, + }, + fetch: (_req, server) => + // Later slices serve the controller GUI here (Carriers/H); today the + // unix carrier is WebSocket-only. + server.upgrade(_req) + ? undefined + : new Response('behavioral instance socket — a WebSocket upgrade is required\n', { status: 426 }), + }) + + // Egress: one redaction pass, the JSONL log, then fan out to every client. + wireRuntimeEgress({ + runtime, + home, + emit: (method, params) => { + for (const ws of clients) ws.send(frame(method, params)) + }, + }) + + return { + path, + close: async () => { + await server.stop(true) + rmSync(path, { force: true }) + }, + } +} diff --git a/src/cli/tests/socket-host.spec.ts b/src/cli/tests/socket-host.spec.ts new file mode 100644 index 000000000..4e4db7ce2 --- /dev/null +++ b/src/cli/tests/socket-host.spec.ts @@ -0,0 +1,209 @@ +import { afterAll, describe, expect, test } from 'bun:test' +import { existsSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { TRACE_MESSAGE_KINDS } from '../../behavioral/behavioral.constants.ts' +import type { BPEvent, JsonObject, SelectionTrace, Trace } from '../../behavioral/behavioral.types.ts' +import type { ClientMessage } from '../../controller/controller.types.ts' +import { createSocketHost, instanceSocketPath } from '../socket-host.ts' + +/** The host's runtime surface, faked: records triggers, traces, and lifecycle calls. */ +const fakeRuntime = () => { + const triggers: BPEvent[] = [] + const listeners: Array<(trace: Trace) => void> = [] + const runtime = { + trigger: (event: BPEvent): void => { + triggers.push(event) + }, + useTrace: (listener: (trace: Trace) => void): (() => void) => { + listeners.push(listener) + return () => {} + }, + start: (): void => {}, + terminate: (): void => {}, + } + const emit = (trace: Trace): void => { + for (const listener of listeners) listener(trace) + } + return { runtime, triggers, emit } +} + +const selectionOf = (selected: { type: string; detail?: JsonObject; space?: string }): SelectionTrace => ({ + kind: TRACE_MESSAGE_KINDS.selection, + timestamp: 0, + instanceId: 'i', + sessionId: 'i', + step: 1, + selected: { priority: 0, ...selected }, +}) + +const traceOf = (kind: Trace['kind'], extra: Partial = {}): Trace => + ({ kind, timestamp: 0, instanceId: 'i', sessionId: 'i', step: 1, ...extra }) as Trace + +/** + * A Transport-shaped client over the real instance socket: `send` frames a + * ClientMessage (or a trigger) as one JSON-RPC frame; inbound JSON-RPC + * notifications land raw in `frames` for assertions. + */ +type TestClient = { + send: (message: ClientMessage | { type: 'trigger'; detail: { event: BPEvent } }) => void + sendRaw: (line: string) => void + frames: unknown[] + waitFor: (pred: (frame: unknown) => boolean, what: string) => Promise + close: () => void +} + +const attachClient = (path: string): Promise => + new Promise((resolveClient, reject) => { + const socket = new WebSocket(`ws+unix://${path}`) + const frames: unknown[] = [] + let nextId = 1 + socket.addEventListener('open', () => { + resolveClient({ + send: (message) => { + const id = nextId++ + if (message.type === 'trigger' || message.type === 'ui_event') { + socket.send( + JSON.stringify({ jsonrpc: '2.0', id, method: message.type, params: { event: message.detail.event } }), + ) + return + } + socket.send(JSON.stringify({ jsonrpc: '2.0', id, method: message.type, params: message.detail })) + }, + sendRaw: (line) => socket.send(line), + frames, + waitFor: (pred: (frame: unknown) => boolean, what: string): Promise => + new Promise((resolveWait, rejectWait) => { + const start = Date.now() + const timer = setInterval(() => { + const found = frames.find(pred) + if (found !== undefined) { + clearInterval(timer) + resolveWait(found as T) + } else if (Date.now() - start > 3000) { + clearInterval(timer) + rejectWait(new Error(`timed out waiting for ${what}`)) + } + }, 5) + }), + close: () => socket.close(), + }) + }) + socket.addEventListener('message', (ev) => { + frames.push(JSON.parse(String(ev.data))) + }) + socket.addEventListener('error', (ev) => reject(new Error(`client socket error: ${String(ev)}`))) + }) + +const homes: string[] = [] +const tempHome = (): string => { + const home = mkdtempSync(join(tmpdir(), 'behavioral-socket-host-')) + homes.push(home) + return home +} + +afterAll(() => { + for (const home of homes) rmSync(home, { recursive: true, force: true }) +}) + +describe('createSocketHost', () => { + test('a trigger request lands as an engine event and answers accepted', async () => { + const home = tempHome() + const fake = fakeRuntime() + const host = createSocketHost({ runtime: fake.runtime, home }) + const client = await attachClient(host.path) + client.send({ type: 'trigger', detail: { event: { type: 'kick' } } }) + type ResponseFrame = { id: number; result?: unknown; error?: unknown } + const response = await client.waitFor( + (frame) => (frame as { id?: number }).id === 1, + 'trigger response', + ) + expect(fake.triggers).toEqual([{ type: 'kick' }]) + expect(response.result).toEqual({ accepted: true }) + client.close() + await host.close() + }) + + test('redacted traces fan back out to every connected client', async () => { + const home = tempHome() + const fake = fakeRuntime() + const host = createSocketHost({ runtime: fake.runtime, home }) + const first = await attachClient(host.path) + const second = await attachClient(host.path) + fake.emit(traceOf(TRACE_MESSAGE_KINDS.idle)) + const firstTrace = await first.waitFor<{ method: string; params: Trace }>( + (frame) => (frame as { method?: string }).method === 'trace', + 'trace on client one', + ) + const secondTrace = await second.waitFor<{ method: string; params: Trace }>( + (frame) => (frame as { method?: string }).method === 'trace', + 'trace on client two', + ) + expect(firstTrace.params.kind).toBe(TRACE_MESSAGE_KINDS.idle) + expect(secondTrace.params.kind).toBe(TRACE_MESSAGE_KINDS.idle) + first.close() + second.close() + await host.close() + }) + + test('a ui_* selection is pushed to clients as its own notification', async () => { + const home = tempHome() + const fake = fakeRuntime() + const host = createSocketHost({ runtime: fake.runtime, home }) + const client = await attachClient(host.path) + fake.emit(selectionOf({ type: 'ui_render', detail: { id: 'r1', target: 'main' } })) + const frame = await client.waitFor<{ method: string; params: JsonObject }>( + (item) => (item as { method?: string }).method === 'ui_render', + 'ui_render notification', + ) + expect(frame.params).toEqual({ id: 'r1', target: 'main' }) + client.close() + await host.close() + }) + + test('an unknown method is answered with a JSON-RPC error', async () => { + const home = tempHome() + const fake = fakeRuntime() + const host = createSocketHost({ runtime: fake.runtime, home }) + const client = await attachClient(host.path) + client.sendRaw('{"jsonrpc":"2.0","id":2,"method":"nope"}') + const frame = await client.waitFor<{ id: number; error?: { code: number } }>( + (item) => (item as { id?: number }).id === 2, + 'error response', + ) + expect(frame.error?.code).toBe(-32603) + client.close() + await host.close() + }) + + test('the socket file lives while the host runs and is removed on close', async () => { + const home = tempHome() + const path = instanceSocketPath(home) + const fake = fakeRuntime() + const host = createSocketHost({ runtime: fake.runtime, home }) + expect(existsSync(path)).toBe(true) + await host.close() + expect(existsSync(path)).toBe(false) + }) + + test('a stale socket file from a dead instance is replaced at start', async () => { + const home = tempHome() + writeFileSync(instanceSocketPath(home), 'garbage from a crashed instance') + const fake = fakeRuntime() + const host = createSocketHost({ runtime: fake.runtime, home }) + const client = await attachClient(host.path) + client.send({ type: 'trigger', detail: { event: { type: 'kick' } } }) + await client.waitFor((frame) => (frame as { id?: number }).id === 1, 'trigger response') + client.close() + await host.close() + }) + + test('a plain HTTP request on the carrier is refused with 426', async () => { + const home = tempHome() + const fake = fakeRuntime() + const host = createSocketHost({ runtime: fake.runtime, home }) + const response = await fetch('http://localhost/', { unix: host.path }) + expect(response.status).toBe(426) + await host.close() + }) +}) From b26ffab6a6f97b4ea1075aa381374b181836d5a2 Mon Sep 17 00:00:00 2001 From: Edward Irby Date: Thu, 24 Sep 2026 00:00:12 -0700 Subject: [PATCH 05/55] =?UTF-8?q?feat:=20the=20three=20TUI=20primitives=20?= =?UTF-8?q?=E2=80=94=20emit,=20prompt,=20select?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Slice 3: a single-file readline TUI (src/cli/tui.ts, no framework) with exactly three primitives: emit (egress-only, Bun.color-resolved kind severity colors that never touch the wire or a prompt), prompt (a raw line resolves to a tui_command ingress event), and select (a numbered choice resolves to tui_select). The tui_* family is ingress-only with AJV detail schemas at the top of tui.ts; the root guard pack now derives its guard entries from TUI_DETAIL_SCHEMAS alongside the controller wire, so malformed tui_* details block visibly. Lines are consumed through an explicit queue (terminal:false readline drops buffered lines between questions). --- src/cli/tests/tui.spec.ts | 103 ++++++++++ src/cli/tui.ts | 190 ++++++++++++++++++ src/faculties/faculties.threads.ts | 11 +- src/faculties/tests/faculties.threads.spec.ts | 30 ++- 4 files changed, 331 insertions(+), 3 deletions(-) create mode 100644 src/cli/tests/tui.spec.ts create mode 100644 src/cli/tui.ts diff --git a/src/cli/tests/tui.spec.ts b/src/cli/tests/tui.spec.ts new file mode 100644 index 000000000..112e9c180 --- /dev/null +++ b/src/cli/tests/tui.spec.ts @@ -0,0 +1,103 @@ +import { afterAll, describe, expect, test } from 'bun:test' +import { Readable } from 'node:stream' +import { createTui, TUI_COMMAND, TUI_SELECT } from '../tui.ts' + +/** A still-open readable whose buffered contents readline consumes line by line. */ +const scriptInput = (lines: string[], end = false): Readable => { + const stream = new Readable({ read() {} }) + for (const line of lines) stream.push(line) + if (end) stream.push(null) + return stream +} + +const out: string[] = [] + +describe('createTui', () => { + afterAll(() => { + out.length = 0 + }) + + describe('emit', () => { + test('writes the line followed by a newline', () => { + const written: string[] = [] + const tui = createTui({ input: scriptInput([]), write: (text) => written.push(text) }) + tui.emit('engine idle') + expect(written).toEqual(['engine idle\n']) + tui.close() + }) + + test('a color request is resolved through Bun.color and never touches the text', () => { + const written: string[] = [] + const tui = createTui({ input: scriptInput([]), write: (text) => written.push(text) }) + tui.emit('deadlock', 'red') + const rendered = written.join('') + expect(rendered).toContain('deadlock') + expect(rendered.endsWith('\n')).toBe(true) + // The ansi format auto-detects stdout depth: degrade means the plain + // line, support means the escape codes precede it. + const ansi = Bun.color('red', 'ansi') ?? '' + if (ansi === '') { + expect(rendered).toBe('deadlock\n') + } else { + expect(rendered.startsWith(ansi)).toBe(true) + } + tui.close() + }) + }) + + describe('prompt', () => { + test('a line of input resolves to a tui_command ingress event', async () => { + const tui = createTui({ input: scriptInput(['/space new docs\n']), write: (text) => out.push(text) }) + const event = await tui.prompt('behavioral> ') + expect(event).toEqual({ type: TUI_COMMAND, detail: { line: '/space new docs' } }) + tui.close() + }) + + test('an empty line re-prompts without producing an event; the next line resolves', async () => { + const tui = createTui({ input: scriptInput(['\n', '/kick\n']), write: (text) => out.push(text) }) + const event = await tui.prompt() + expect(event).toEqual({ type: TUI_COMMAND, detail: { line: '/kick' } }) + tui.close() + }) + + test('stdin closing while a prompt is pending rejects', async () => { + const tui = createTui({ input: scriptInput([], true), write: (text) => out.push(text) }) + await tui.prompt().then( + () => { + throw new Error('expected the pending prompt to reject on EOF') + }, + (error: unknown) => { + expect((error as Error).message).toContain('closed') + }, + ) + tui.close() + }) + }) + + describe('select', () => { + test('a numbered choice resolves to a tui_select event for that option', async () => { + const written: string[] = [] + const tui = createTui({ input: scriptInput(['2\n']), write: (text) => written.push(text) }) + const event = await tui.select('pick a space', ['docs', 'harness', 'scratch']) + expect(event).toEqual({ type: TUI_SELECT, detail: { option: 'harness' } }) + // The question and the numbered options were rendered before the input. + expect(written.join('')).toContain('pick a space') + expect(written.join('')).toContain('1. docs') + expect(written.join('')).toContain('3. scratch') + tui.close() + }) + + test('an out-of-range choice reports and re-prompts; a later valid choice resolves', async () => { + const written: string[] = [] + const tui = createTui({ + input: scriptInput(['9\n', 'not-a-number\n', '1\n']), + write: (text) => written.push(text), + }) + const event = await tui.select('pick a space', ['docs', 'harness']) + expect(event).toEqual({ type: TUI_SELECT, detail: { option: 'docs' } }) + expect(written.join('')).toContain('invalid choice: 9') + expect(written.join('')).toContain('invalid choice: not-a-number') + tui.close() + }) + }) +}) diff --git a/src/cli/tui.ts b/src/cli/tui.ts new file mode 100644 index 000000000..78a78a0b6 --- /dev/null +++ b/src/cli/tui.ts @@ -0,0 +1,190 @@ +import { createInterface } from 'node:readline' +import type { JSONSchemaType } from 'ajv' +import { TRACE_MESSAGE_KINDS } from '../behavioral/behavioral.constants.ts' +import type { BPEvent } from '../behavioral/behavioral.types.ts' + +// --------------------------------------------------------------------------- +// The tui_* vocabulary — its own small closed family, ingress-only (not a +// `ui_*` mirror). Two incoming constants with specific detail shapes; no +// outbound `tui_*` exists — the TUI's egress is the existing trace wire. +// The constants live here because they are the TUI's vocabulary, not the +// engine's; the guard-threads generator consumes the type→schema pairs. +// --------------------------------------------------------------------------- + +/** A raw (unparsed) command line from the prompt. */ +export const TUI_COMMAND = 'tui_command' + +/** A chosen option from a selection. */ +export const TUI_SELECT = 'tui_select' + +export type TuiCommandDetail = { line: string } +export type TuiSelectDetail = { option: string } + +export const TuiCommandDetailSchema: JSONSchemaType = { + type: 'object', + properties: { line: { type: 'string' } }, + required: ['line'], + additionalProperties: false, +} + +export const TuiSelectDetailSchema: JSONSchemaType = { + type: 'object', + properties: { option: { type: 'string' } }, + required: ['option'], + additionalProperties: false, +} + +/** The tui_* ingress detail schemas, keyed by the event type. */ +export const TUI_DETAIL_SCHEMAS = { + [TUI_COMMAND]: TuiCommandDetailSchema, + [TUI_SELECT]: TuiSelectDetailSchema, +} as const + +/** + * Kind/severity → `Bun.color` input for trace lines. Colors are + * presentation-only: `emit` resolves them at render time and they never enter + * the wire or a prompt. Kinds without an entry render plain. + */ +export const TRACE_KIND_COLORS: Record = { + [TRACE_MESSAGE_KINDS.deadlock]: 'red', + [TRACE_MESSAGE_KINDS.trigger_error]: 'red', + [TRACE_MESSAGE_KINDS.add_thread_error]: 'red', + [TRACE_MESSAGE_KINDS.transform_error]: 'red', + [TRACE_MESSAGE_KINDS.idle]: 'dimgray', + [TRACE_MESSAGE_KINDS.selection]: 'cyan', +} + +/** Where the TUI writes by default. */ +const defaultWrite = (text: string): void => { + process.stdout.write(text) +} + +/** + * The minimal TUI surface — exactly three primitives: `emit` (trace/log lines + * → terminal, egress-only), `prompt` (a slash-command line → a `tui_command` + * ingress event), `select` (a numbered choice → a `tui_select` ingress event). + * + * @remarks + * Readline only, no alt-buffer framework. Slash commands are NOT parsed here: + * the TUI forwards the raw line and the engine's threads validate. The TUI + * never calls the engine — every primitive that produces user intent resolves + * to a BPEvent the client wiring sends over the socket like every other + * client. The TUI is deliberately a single-file thin client (a `serve.ts`-style + * surface); if it outgrows ~300 lines that is a design smell, not structure to + * build — logic belongs in threads. + * + * @public + */ +export type Tui = { + /** Egress-only: write one terminal line, optionally colored. Produces no event. */ + emit: (text: string, color?: string) => void + /** Read one command line; resolves to the `tui_command` ingress event. */ + prompt: (text?: string) => Promise + /** Render numbered options and read one choice; resolves to `tui_select`. */ + select: (question: string, options: string[]) => Promise + /** Release the readline. */ + close: () => void +} + +/** + * Create the TUI over `input` (readline) and `write` (egress). + * + * @public + */ +export const createTui = ({ + input, + write = defaultWrite, +}: { + /** The line source (stdin for the real TUI; a scripted stream in specs). */ + input: NodeJS.ReadableStream + /** The terminal writer (injectable so specs can collect output). */ + write?: (text: string) => void +}): Tui => { + const readline = createInterface({ input, terminal: false }) + let closed = false + const eofRejectors = new Set<(error: Error) => void>() + // With `terminal: false` readline delivers every buffered line regardless of + // a pending question, so lines are consumed through an explicit queue + // instead of `question()` (whose second call never sees buffered lines). + const bufferedLines: string[] = [] + const lineWaiters: Array<(line: string) => void> = [] + + readline.on('line', (line: string) => { + const waiter = lineWaiters.shift() + if (waiter === undefined) { + bufferedLines.push(line) + } else { + waiter(line) + } + }) + + readline.on('close', () => { + closed = true + for (const reject of eofRejectors) reject(new Error('tui stdin closed')) + eofRejectors.clear() + }) + + /** Print `text`, resolve the next input line. */ + const askLine = (text: string): Promise => + new Promise((resolve, reject) => { + if (closed) { + reject(new Error('tui stdin closed')) + return + } + write(text) + const buffered = bufferedLines.shift() + if (buffered !== undefined) { + resolve(buffered) + return + } + const rejectOnClose = (error: Error): void => { + eofRejectors.delete(rejectOnClose) + reject(error) + } + eofRejectors.add(rejectOnClose) + lineWaiters.push((line) => { + eofRejectors.delete(rejectOnClose) + resolve(line) + }) + }) + + const emit = (text: string, color?: string): void => { + const ansi = color === undefined ? '' : (Bun.color(color, 'ansi') ?? '') + // Colors are presentation-only: resolved here at render time, never on + // the wire, never on a prompt line (readline width math). + write(ansi === '' ? `${text}\n` : `${ansi}${text}\x1b[0m\n`) + } + + const prompt = async (text = '> '): Promise => { + for (;;) { + const line = (await askLine(text)).trim() + // A stray enter re-prompts; it carries no intent, so no event. + if (line === '') continue + return { type: TUI_COMMAND, detail: { line } } + } + } + + const select = async (question: string, options: string[]): Promise => { + for (;;) { + write(`${question}\n`) + options.forEach((option, index) => { + write(` ${index + 1}. ${option}\n`) + }) + const line = (await askLine('> ')).trim() + const choice = Number.parseInt(line, 10) + if (Number.isInteger(choice) && choice >= 1 && choice <= options.length) { + return { type: TUI_SELECT, detail: { option: options[choice - 1]! } } + } + emit(`invalid choice: ${line}`, 'red') + } + } + + return { + emit, + prompt, + select, + close: () => { + readline.close() + }, + } +} diff --git a/src/faculties/faculties.threads.ts b/src/faculties/faculties.threads.ts index 2f8ca1d2d..de828e947 100644 --- a/src/faculties/faculties.threads.ts +++ b/src/faculties/faculties.threads.ts @@ -1,4 +1,5 @@ import type { Thread } from '../behavioral/behavioral.types.ts' +import { TUI_DETAIL_SCHEMAS } from '../cli/tui.ts' import { CONTROLLER_DETAIL_SCHEMAS } from '../controller/controller.schemas.ts' /** @@ -75,5 +76,13 @@ const invalidControllerMessages: GuardEntry[] = Object.entries(CONTROLLER_DETAIL ([type, detailSchema]) => ({ type, detailSchema: detailSchema as Record }), ) +const invalidTuiMessages: GuardEntry[] = Object.entries(TUI_DETAIL_SCHEMAS).map(([type, detailSchema]) => ({ + type, + detailSchema: detailSchema as Record, +})) + /** The root threads: default threads mounted by every composition. */ -export const facultiesThreads: Thread[] = guardThreads('guard:controller-schema', invalidControllerMessages) +export const facultiesThreads: Thread[] = guardThreads('guard:ingress-schema', [ + ...invalidControllerMessages, + ...invalidTuiMessages, +]) diff --git a/src/faculties/tests/faculties.threads.spec.ts b/src/faculties/tests/faculties.threads.spec.ts index 86425330b..4016ec2a2 100644 --- a/src/faculties/tests/faculties.threads.spec.ts +++ b/src/faculties/tests/faculties.threads.spec.ts @@ -2,16 +2,19 @@ import { describe, expect, test } from 'bun:test' import { TRACE_MESSAGE_KINDS } from '../../behavioral/behavioral.constants.ts' import { behavioral } from '../../behavioral/behavioral.ts' import type { FrontierTrace, JsonObject, SelectionTrace, Trace } from '../../behavioral/behavioral.types.ts' +import { TUI_COMMAND, TUI_SELECT } from '../../cli/tui.ts' import { eventGuardEntries, facultiesThreads } from '../faculties.threads.ts' -const run = (detail: JsonObject) => { +const run = (detail: JsonObject) => runType('ui_render', detail) + +const runType = (type: string, detail: JsonObject) => { const traces: Trace[] = [] const { addThread, step, useTrace } = behavioral() useTrace((trace: Trace) => { traces.push(trace) }) for (const thread of facultiesThreads) addThread(thread) - addThread({ label: 'renderer', once: true, rules: [{ request: { type: 'ui_render', detail } }] }) + addThread({ label: 'sender', once: true, rules: [{ request: { type, detail } }] }) step() const selections = traces.filter((trace): trace is SelectionTrace => trace.kind === TRACE_MESSAGE_KINDS.selection) const frontiers = traces.filter((trace): trace is FrontierTrace => trace.kind === TRACE_MESSAGE_KINDS.frontier) @@ -33,6 +36,29 @@ describe('facultiesThreads — the root guard threads', () => { expect(frontiers.some((frontier) => frontier.status === 'ready')).toBe(true) }) + describe('the tui_* vocabulary is guarded at ingress', () => { + test('blocks a malformed tui_command detail: candidate present, none enabled', () => { + const { selections, frontiers } = runType(TUI_COMMAND, { line: 42 }) + expect(selections.some((s) => s.selected.type === TUI_COMMAND)).toBe(false) + const deadlock = frontiers.find((frontier) => frontier.status === 'deadlock') + expect(deadlock?.candidates.map((c) => c.type)).toContain(TUI_COMMAND) + expect(deadlock?.enabled).toEqual([]) + }) + + test('lets a well-formed tui_command select', () => { + const { selections, frontiers } = runType(TUI_COMMAND, { line: '/space new docs' }) + expect(selections.some((s) => s.selected.type === TUI_COMMAND)).toBe(true) + expect(frontiers.some((frontier) => frontier.status === 'ready')).toBe(true) + }) + + test('blocks a malformed tui_select detail and lets a well-formed one select', () => { + const blocked = runType(TUI_SELECT, { option: 7 }) + expect(blocked.selections.some((s) => s.selected.type === TUI_SELECT)).toBe(false) + const selected = runType(TUI_SELECT, { option: 'docs' }) + expect(selected.selections.some((s) => s.selected.type === TUI_SELECT)).toBe(true) + }) + }) + // The review's follow-up 4: a schema without properties.type.const is a // wiring defect — the guard generator must throw, not produce a guard that // can never match. From 391a48b1c293ce11a2792281d054a44b51a2eaa9 Mon Sep 17 00:00:00 2001 From: Edward Irby Date: Thu, 24 Sep 2026 00:24:30 -0700 Subject: [PATCH 06/55] =?UTF-8?q?feat:=20attach-or-start=20=E2=80=94=20the?= =?UTF-8?q?=20bare=20behavioral=20command?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Slice 4: bare `behavioral` (no subcommand) is the attach-or-start entry. A live instance (pidfile held) is attached to over /instance.sock with "attached to running instance "; a free or stale home starts the foreground instance — engine + socket host + TUI in one process, the shell supervising. The instance own TUI rides the socket like every other client (no in-process fast path); SIGINT/SIGTERM terminate the engine and remove the pidfile and socket (the daemon-door discipline). The router gains a lazy `default` entry; --help and subcommands still never load the composition graph. Two-process specs drive the real lifecycle across Bun.spawn children: attach notice, cross-process trigger, no second instance, stale-pidfile reap, SIGTERM cleanup. File IO is Bun-native (Bun.file/Bun.write/Bun.file().delete()); node:fs only where Bun has no equivalent (mkdir, socket-file existence). --- bin/behavioral.ts | 7 ++ src/cli/attach-or-start.ts | 90 ++++++++++++++++++ src/cli/attach.ts | 87 ++++++++++++++++++ src/cli/cli.ts | 12 ++- src/cli/socket-host.ts | 16 ++-- src/cli/tests/attach-or-start.spec.ts | 107 ++++++++++++++++++++++ src/cli/tests/cli.spec.ts | 35 ++++++- src/cli/tests/fixtures/attach-instance.ts | 51 +++++++++++ src/cli/tests/socket-host.spec.ts | 14 +-- src/faculties/instance-lock.ts | 25 +++-- src/faculties/tests/instance-lock.spec.ts | 36 ++++---- 11 files changed, 436 insertions(+), 44 deletions(-) create mode 100644 src/cli/attach-or-start.ts create mode 100644 src/cli/attach.ts create mode 100644 src/cli/tests/attach-or-start.spec.ts create mode 100644 src/cli/tests/fixtures/attach-instance.ts diff --git a/bin/behavioral.ts b/bin/behavioral.ts index 0af6a594a..ea77d60da 100755 --- a/bin/behavioral.ts +++ b/bin/behavioral.ts @@ -21,6 +21,13 @@ export const runCli = makeCliRouter({ await init(args) }, }, + // The bare command is attach-or-start: attach to a running instance over + // /instance.sock, or start the foreground instance. Lazy: --help and + // the subcommands must not load the composition graph until invoked. + default: async () => { + const { attachOrStart } = await import('../src/cli/attach-or-start.ts') + await attachOrStart() + }, }) await runCli(Bun.argv) diff --git a/src/cli/attach-or-start.ts b/src/cli/attach-or-start.ts new file mode 100644 index 000000000..eaf1ef82b --- /dev/null +++ b/src/cli/attach-or-start.ts @@ -0,0 +1,90 @@ +import { behavioralHome } from '../faculties/behavioral-home.ts' +import { acquireInstanceLock } from '../faculties/instance-lock.ts' +import { attachTui } from './attach.ts' +import type { HostRuntime } from './serve.ts' +import { createSocketHost, instanceSocketPath } from './socket-host.ts' + +/** + * The composition default: the runtime graph from `/config.ts`. Loaded + * lazily — an attaching process never pays for the composition graph. + */ +const defaultCreateRuntime = async (): Promise => { + const { bProgram } = await import('./b-program.ts') + const { loadConfig } = await import('./load-config.ts') + return bProgram(await loadConfig()) +} + +export type AttachOrStartOptions = { + /** The single `` root; defaults to `behavioralHome()`. */ + home?: string + /** The TUI's line source; defaults to stdin. */ + input?: NodeJS.ReadableStream + /** The TUI's terminal writer; defaults to stdout. */ + write?: (text: string) => void + /** The runtime factory; defaults to `bProgram(await loadConfig())`. */ + createRuntime?: () => HostRuntime | Promise +} + +export type AttachOrStartResult = { + attached: boolean + instanceId?: string +} + +/** + * The bare `behavioral` entry — attach-or-start, never second-instance. + * + * @remarks + * A live instance (pidfile held) is attached to over `/instance.sock` + * with the notice `attached to running instance `; a free or stale home is + * started fresh: engine + unix-socket host + TUI in one foreground process. + * The instance's own TUI rides the socket like every other client. In the + * start mode, SIGINT/SIGTERM terminate the engine and clean up the pidfile and + * socket — the daemon-door discipline. + * + * @public + */ +export const attachOrStart = async ({ + home = behavioralHome(), + input = process.stdin, + write = (text: string): void => { + process.stdout.write(text) + }, + createRuntime = defaultCreateRuntime, +}: AttachOrStartOptions = {}): Promise => { + const lock = await acquireInstanceLock({ home }) + if (!lock.acquired) { + // Attach: the TUI is a second client on the local lane. + const result = await attachTui({ + socketPath: instanceSocketPath(home), + input, + write, + onAttach: (id) => write(`attached to running instance ${id}\n`), + }) + return { attached: true, instanceId: result.instanceId } + } + + // Start: foreground engine + socket host + TUI (a socket client like every + // other client — no in-process fast path). + const runtime = await createRuntime() + const host = await createSocketHost({ runtime, home }) + let cleaned = false + const cleanup = async (): Promise => { + if (cleaned) return + cleaned = true + await host.close() + runtime.terminate() + await lock.release() + } + const onSignal = (): void => { + void cleanup().then(() => process.exit(0)) + } + process.on('SIGINT', onSignal) + process.on('SIGTERM', onSignal) + + runtime.start() + const result = await attachTui({ socketPath: host.path, input, write }) + await cleanup() + process.off('SIGINT', onSignal) + process.off('SIGTERM', onSignal) + return { attached: false, instanceId: result.instanceId } +} diff --git a/src/cli/attach.ts b/src/cli/attach.ts new file mode 100644 index 000000000..3259999e9 --- /dev/null +++ b/src/cli/attach.ts @@ -0,0 +1,87 @@ +import type { Trace } from '../behavioral/behavioral.types.ts' +import { createTui, TRACE_KIND_COLORS } from './tui.ts' + +/** How the attach loop ended. */ +export type AttachResult = { + /** The engine's self-minted instance id, learned from the trace wire. */ + instanceId?: string + reason: 'stdin-ended' | 'socket-closed' +} + +/** + * Run the TUI as a client of the running instance over its unix socket — the + * one client path for both an attaching process and the started instance's own + * TUI (no in-process fast path: every client is a client). + * + * @remarks + * Ingress: each prompt line resolves to a `tui_command` event sent as a + * JSON-RPC `trigger` request; the engine's guard threads validate it. Egress: + * `trace` notifications render (kind-colored) lines; `ui_*` selections render + * when the interface pack lands — today the TUI renders trace/log lines only. + * + * @public + */ +export const attachTui = async ({ + socketPath, + input = process.stdin, + write = (text: string): void => { + process.stdout.write(text) + }, + onAttach, +}: { + socketPath: string + input?: NodeJS.ReadableStream + write?: (text: string) => void + /** Called once, with the instance id from the first received trace. */ + onAttach?: (instanceId: string) => void +}): Promise => { + const tui = createTui({ input, write }) + const socket = new WebSocket(`ws+unix://${socketPath}`) + let instanceId: string | undefined + let nextId = 1 + + const result = await new Promise((resolve) => { + let settled = false + const finish = (reason: AttachResult['reason']): void => { + if (settled) return + settled = true + try { + socket.close() + } catch { + // Never opened — nothing to close. + } + resolve({ reason, instanceId }) + } + socket.addEventListener('open', () => { + // The ingress loop starts only when the carrier is open. + void (async () => { + for (;;) { + const event = await tui.prompt('behavioral> ') + socket.send(JSON.stringify({ jsonrpc: '2.0', id: nextId++, method: 'trigger', params: { event } })) + } + })().catch(() => finish('stdin-ended')) + }) + socket.addEventListener('message', (ev) => { + let frame: { method?: string; params?: unknown } + try { + frame = JSON.parse(String(ev.data)) + } catch { + return + } + if (frame.method !== 'trace') return + const trace = frame.params as Trace + if (instanceId === undefined && typeof trace?.instanceId === 'string') { + instanceId = trace.instanceId + onAttach?.(instanceId) + } + // MINIMAL: trace lines render as compact JSON; richer rendering rides + // the ui_* producers slice (upgrade path: a per-kind line formatter). + tui.emit(JSON.stringify(trace), TRACE_KIND_COLORS[trace.kind]) + }) + socket.addEventListener('close', () => finish('socket-closed')) + socket.addEventListener('error', () => finish('socket-closed')) + }) + + tui.close() + return result +} diff --git a/src/cli/cli.ts b/src/cli/cli.ts index aaf6cb13d..b21b68860 100644 --- a/src/cli/cli.ts +++ b/src/cli/cli.ts @@ -69,10 +69,15 @@ type CliHandlerConfig = { run: (input: TInput, flags: CliFlags) => Promise | TOutput } +/** Options the router's default (no-subcommand) entry receives. */ +export type CliDefaultEntry = (args: string[]) => Promise + type CliRouterConfig = { name: string description: string commands: Record Promise> + /** Runs for a bare invocation (no subcommand) — e.g. attach-or-start. */ + default?: CliDefaultEntry } const buildUsage = ({ name, help }: { name: string; help: string }): string => @@ -314,7 +319,7 @@ export const defineScript = async ({ } export const makeCliRouter = - ({ name, description, commands }: CliRouterConfig) => + ({ name, description, commands, default: commandsDefault }: CliRouterConfig) => async (argv: string[]): Promise => { const command = argv[2] const args = argv.slice(3) @@ -336,6 +341,11 @@ export const makeCliRouter = process.exit(0) } + if (!command && commandsDefault) { + await commandsDefault(args) + return + } + if (!command || command === '--help' || command === '-h') { console.error(`Usage: ${name} [options] ${name} --schema # Discover input schema diff --git a/src/cli/socket-host.ts b/src/cli/socket-host.ts index 670d07695..9a2aee8e6 100644 --- a/src/cli/socket-host.ts +++ b/src/cli/socket-host.ts @@ -1,4 +1,3 @@ -import { rmSync } from 'node:fs' import { join } from 'node:path' import type { ServerWebSocket } from 'bun' import { behavioralHome } from '../faculties/behavioral-home.ts' @@ -43,16 +42,19 @@ export type SocketHost = { * * @public */ -export const createSocketHost = ({ +export const createSocketHost = async ({ runtime, home = behavioralHome(), }: { runtime: HostRuntime home?: string -}): SocketHost => { +}): Promise => { const path = instanceSocketPath(home) - // A socket file left by a dead instance cannot be bound again — remove it. - rmSync(path, { force: true }) + // A socket file left by a dead instance cannot be bound again — remove it + // before the bind (ENOENT means there was nothing to reap). + await Bun.file(path) + .delete() + .catch(() => {}) const clients = new Set>() const frame = (method: string, params: unknown): string => JSON.stringify({ jsonrpc: '2.0', method, params }) @@ -131,7 +133,9 @@ export const createSocketHost = ({ path, close: async () => { await server.stop(true) - rmSync(path, { force: true }) + await Bun.file(path) + .delete() + .catch(() => {}) }, } } diff --git a/src/cli/tests/attach-or-start.spec.ts b/src/cli/tests/attach-or-start.spec.ts new file mode 100644 index 000000000..3b5ec64c5 --- /dev/null +++ b/src/cli/tests/attach-or-start.spec.ts @@ -0,0 +1,107 @@ +import { afterAll, describe, expect, test } from 'bun:test' +import { existsSync, mkdirSync, rmSync } from 'node:fs' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { instancePidfilePath } from '../../faculties/instance-lock.ts' +import { instanceSocketPath } from '../socket-host.ts' + +const homes: string[] = [] +const tempHome = (): string => { + const home = join(tmpdir(), `behavioral-attach-${Date.now()}-${Math.random().toString(36).slice(2)}`) + homes.push(home) + mkdirSync(home, { recursive: true }) + return home +} + +afterAll(() => { + for (const home of homes) rmSync(home, { recursive: true, force: true }) +}) + +/** Poll until `pred` holds or the timeout elapses. */ +const eventually = async (pred: () => boolean | Promise, what: string, timeoutMs = 20_000): Promise => { + const start = Date.now() + while (!(await pred())) { + if (Date.now() - start > timeoutMs) throw new Error(`timed out waiting for ${what}`) + await Bun.sleep(50) + } +} + +/** Incrementally drain a subprocess stream into a string. */ +const collect = (stream: ReadableStream): { text: () => string } => { + const decoder = new TextDecoder() + let acc = '' + void (async () => { + try { + for await (const chunk of stream as unknown as AsyncIterable) + acc += decoder.decode(chunk, { stream: true }) + } catch { + // Stream torn down with the process. + } + })() + return { text: () => acc } +} + +/** The two-process fixture: mode 'start' boots the foreground instance; mode 'attach' attaches. */ +const spawnLifecycleProcess = (mode: 'start' | 'attach', home: string) => + Bun.spawn({ + cmd: ['bun', 'run', join(import.meta.dir, 'fixtures', 'attach-instance.ts'), mode], + env: { ...process.env, BEHAVIORAL_HOME: home }, + stdin: 'pipe', + stdout: 'pipe', + stderr: 'pipe', + }) + +const pidOf = async (home: string): Promise => + (JSON.parse(await Bun.file(instancePidfilePath(home)).text()) as { pid: number }).pid + +describe('attachOrStart — the two-process lifecycle', () => { + test('a bare start owns the home; an attacher reuses it; SIGTERM cleans up', async () => { + const home = tempHome() + const instance = spawnLifecycleProcess('start', home) + const instanceOut = collect(instance.stdout) + + // The instance is up: the socket file exists and the pidfile names its pid. + const socketPath = instanceSocketPath(home) + await eventually(() => existsSync(socketPath), 'instance socket') + const instancePid = await pidOf(home) + expect(instancePid).toBeGreaterThan(0) + + // Second process: attach-or-start must attach, never spawn a second instance. + const attacher = spawnLifecycleProcess('attach', home) + const attacherOut = collect(attacher.stdout) + attacher.stdin.write('/kick\n') + await eventually(() => attacherOut.text().includes('attached to running instance'), 'attach notice') + + // No second instance: the pidfile still names the first instance's pid. + expect(await pidOf(home)).toBe(instancePid) + + // The trigger crossed processes: the instance selected the tui_command event. + await eventually(() => instanceOut.text().includes('tui_command'), 'trigger selection trace on the instance') + + // The attacher's stdin ends → it detaches; the instance keeps running. + attacher.stdin.end() + const attacherCode = await attacher.exited + expect(attacherCode).toBe(0) + expect(existsSync(socketPath)).toBe(true) + expect(await pidOf(home)).toBe(instancePid) + + // SIGTERM: terminate + pidfile/socket cleanup (the daemon-door discipline). + instance.kill('SIGTERM') + const instanceCode = await instance.exited + expect(instanceCode).toBe(0) + await eventually(() => !existsSync(socketPath), 'socket removal') + expect(existsSync(instancePidfilePath(home))).toBe(false) + }, 40_000) + + test('a stale pidfile is reaped and the instance starts fresh', async () => { + const home = tempHome() + await Bun.write(instancePidfilePath(home), JSON.stringify({ pid: 999_999_999 })) + const instance = spawnLifecycleProcess('start', home) + await eventually(() => existsSync(instanceSocketPath(home)), 'instance socket after stale-pidfile reap') + const pid = await pidOf(home) + expect(pid).toBeGreaterThan(1) + expect(pid).not.toBe(999_999_999) + instance.kill('SIGTERM') + await instance.exited + }, 40_000) +}) diff --git a/src/cli/tests/cli.spec.ts b/src/cli/tests/cli.spec.ts index 3bba0cefc..9ee6a927d 100644 --- a/src/cli/tests/cli.spec.ts +++ b/src/cli/tests/cli.spec.ts @@ -63,15 +63,19 @@ describe('Router-level flags (subprocess)', () => { expect(stderr).toContain('--version') }) - test('no args exits 1 and prints usage', async () => { + test('no args runs the attach-or-start default (a closed stdin detaches cleanly)', async () => { const proc = Bun.spawn(['bun', 'bin/behavioral.ts'], { stdout: 'pipe', stderr: 'pipe', cwd: path.resolve(import.meta.dir, '../../..'), + env: { ...process.env, BEHAVIORAL_HOME: path.join('/tmp', `behavioral-cli-default-${Date.now()}`) }, }) - expect(await proc.exited).toBe(1) + // No running instance → the bare command starts a foreground instance; the + // closed stdin (non-TTY) ends the TUI and detaches cleanly. + const code = await proc.exited + expect(code).toBe(0) const stderr = await new Response(proc.stderr).text() - expect(stderr).toContain('Commands') + expect(stderr).not.toContain('Commands') }) test('--schema lists all commands', async () => { @@ -311,3 +315,28 @@ describe('makeCli', () => { expect(stderr).toContain('--help') }) }) + +describe('makeCliRouter — the default (no-subcommand) entry', () => { + test('a bare invocation runs the default handler with the remaining args', async () => { + const proc = Bun.spawn( + [ + 'bun', + '-e', + `import { makeCliRouter } from '${cliPath}'; + const router = makeCliRouter({ + name: 'behavioral', + description: 'test', + commands: { serve: async () => {} }, + default: async (args) => { + process.stdout.write('default ran:' + JSON.stringify(args)); + }, + }); + await router(['behavioral']);`, + ], + { stdout: 'pipe', stderr: 'pipe' }, + ) + expect(await proc.exited).toBe(0) + const output = await new Response(proc.stdout).text() + expect(output).toContain('default ran:[]') + }) +}) diff --git a/src/cli/tests/fixtures/attach-instance.ts b/src/cli/tests/fixtures/attach-instance.ts new file mode 100644 index 000000000..b2b9eb23d --- /dev/null +++ b/src/cli/tests/fixtures/attach-instance.ts @@ -0,0 +1,51 @@ +import { TRACE_MESSAGE_KINDS } from '../../../behavioral/behavioral.constants.ts' +import type { SelectionTrace, Trace } from '../../../behavioral/behavioral.types.ts' +import { attachOrStart } from '../../attach-or-start.ts' +import type { HostRuntime } from '../../serve.ts' + +/** + * The two-process spec's child entry — the real attach-or-start lifecycle with + * the composition graph swapped for an echo runtime: each trigger is answered + * with a selection trace, so a trigger landing in this process is visible on + * stdout through the socket TUI. + * + * @remarks + * The runtime is a seam here (mocking the composition graph); the lifecycle + * under test — lock, socket host, socket TUI client, signals, cleanup — is + * fully real, including the cross-process wire. + */ +const echoRuntime = (): HostRuntime => { + // The engine self-mints the instance id (the host is the identity authority). + const instanceId = Bun.randomUUIDv7() + const listeners = new Set<(trace: Trace) => void>() + const emit = (trace: Trace): void => { + for (const listener of listeners) listener(trace) + } + const base = { instanceId, sessionId: instanceId } + return { + trigger: (event) => { + const trace: SelectionTrace = { + kind: TRACE_MESSAGE_KINDS.selection, + timestamp: Date.now(), + step: 1, + ...base, + selected: { priority: 0, type: event.type, detail: event.detail }, + } + emit(trace) + }, + useTrace: (l) => { + // The real engine's useTrace is a multi-subscriber subject. + listeners.add(l) + return () => { + listeners.delete(l) + } + }, + start: () => { + emit({ kind: TRACE_MESSAGE_KINDS.idle, timestamp: Date.now(), step: 1, ...base }) + }, + terminate: () => {}, + } +} + +const mode = process.argv[2] === 'attach' ? 'attach' : 'start' +await attachOrStart(mode === 'start' ? { createRuntime: () => echoRuntime() } : {}) diff --git a/src/cli/tests/socket-host.spec.ts b/src/cli/tests/socket-host.spec.ts index 4e4db7ce2..1b8d881a3 100644 --- a/src/cli/tests/socket-host.spec.ts +++ b/src/cli/tests/socket-host.spec.ts @@ -110,7 +110,7 @@ describe('createSocketHost', () => { test('a trigger request lands as an engine event and answers accepted', async () => { const home = tempHome() const fake = fakeRuntime() - const host = createSocketHost({ runtime: fake.runtime, home }) + const host = await createSocketHost({ runtime: fake.runtime, home }) const client = await attachClient(host.path) client.send({ type: 'trigger', detail: { event: { type: 'kick' } } }) type ResponseFrame = { id: number; result?: unknown; error?: unknown } @@ -127,7 +127,7 @@ describe('createSocketHost', () => { test('redacted traces fan back out to every connected client', async () => { const home = tempHome() const fake = fakeRuntime() - const host = createSocketHost({ runtime: fake.runtime, home }) + const host = await createSocketHost({ runtime: fake.runtime, home }) const first = await attachClient(host.path) const second = await attachClient(host.path) fake.emit(traceOf(TRACE_MESSAGE_KINDS.idle)) @@ -149,7 +149,7 @@ describe('createSocketHost', () => { test('a ui_* selection is pushed to clients as its own notification', async () => { const home = tempHome() const fake = fakeRuntime() - const host = createSocketHost({ runtime: fake.runtime, home }) + const host = await createSocketHost({ runtime: fake.runtime, home }) const client = await attachClient(host.path) fake.emit(selectionOf({ type: 'ui_render', detail: { id: 'r1', target: 'main' } })) const frame = await client.waitFor<{ method: string; params: JsonObject }>( @@ -164,7 +164,7 @@ describe('createSocketHost', () => { test('an unknown method is answered with a JSON-RPC error', async () => { const home = tempHome() const fake = fakeRuntime() - const host = createSocketHost({ runtime: fake.runtime, home }) + const host = await createSocketHost({ runtime: fake.runtime, home }) const client = await attachClient(host.path) client.sendRaw('{"jsonrpc":"2.0","id":2,"method":"nope"}') const frame = await client.waitFor<{ id: number; error?: { code: number } }>( @@ -180,7 +180,7 @@ describe('createSocketHost', () => { const home = tempHome() const path = instanceSocketPath(home) const fake = fakeRuntime() - const host = createSocketHost({ runtime: fake.runtime, home }) + const host = await createSocketHost({ runtime: fake.runtime, home }) expect(existsSync(path)).toBe(true) await host.close() expect(existsSync(path)).toBe(false) @@ -190,7 +190,7 @@ describe('createSocketHost', () => { const home = tempHome() writeFileSync(instanceSocketPath(home), 'garbage from a crashed instance') const fake = fakeRuntime() - const host = createSocketHost({ runtime: fake.runtime, home }) + const host = await createSocketHost({ runtime: fake.runtime, home }) const client = await attachClient(host.path) client.send({ type: 'trigger', detail: { event: { type: 'kick' } } }) await client.waitFor((frame) => (frame as { id?: number }).id === 1, 'trigger response') @@ -201,7 +201,7 @@ describe('createSocketHost', () => { test('a plain HTTP request on the carrier is refused with 426', async () => { const home = tempHome() const fake = fakeRuntime() - const host = createSocketHost({ runtime: fake.runtime, home }) + const host = await createSocketHost({ runtime: fake.runtime, home }) const response = await fetch('http://localhost/', { unix: host.path }) expect(response.status).toBe(426) await host.close() diff --git a/src/faculties/instance-lock.ts b/src/faculties/instance-lock.ts index d13e43a20..c509f2a10 100644 --- a/src/faculties/instance-lock.ts +++ b/src/faculties/instance-lock.ts @@ -1,4 +1,3 @@ -import { existsSync, readFileSync, rmSync, writeFileSync } from 'node:fs' import { join } from 'node:path' /** @@ -26,12 +25,20 @@ const pidAlive = (pid: number): boolean => { } } +/** Delete the pidfile if present (Bun file IO; missing file is a no-op). */ +const removePidfile = async (path: string): Promise => { + // ENOENT means already gone — force semantics. + await Bun.file(path) + .delete() + .catch(() => {}) +} + /** * The acquired instance lock — `release()` removes the pidfile. * * @public */ -export type InstanceLock = { acquired: true; release: () => void } +export type InstanceLock = { acquired: true; release: () => Promise } /** * The lock is held: a live instance owns the home; `pid` is its pid. @@ -51,18 +58,18 @@ export type InstanceLockHeld = { acquired: false; pid: number } * * @public */ -export const acquireInstanceLock = ({ +export const acquireInstanceLock = async ({ home, pid = process.pid, }: { home: string pid?: number -}): InstanceLock | InstanceLockHeld => { +}): Promise => { const path = instancePidfilePath(home) - if (existsSync(path)) { + if (await Bun.file(path).exists()) { let holder: number | undefined try { - const parsed: unknown = JSON.parse(readFileSync(path, 'utf8')) + const parsed: unknown = await Bun.file(path).json() if (typeof parsed === 'object' && parsed !== null && typeof (parsed as { pid?: unknown }).pid === 'number') { holder = (parsed as { pid: number }).pid } @@ -70,8 +77,8 @@ export const acquireInstanceLock = ({ // Unreadable pidfile: treat as stale, reaped below. } if (holder !== undefined && pidAlive(holder)) return { acquired: false, pid: holder } - rmSync(path, { force: true }) + await removePidfile(path) } - writeFileSync(path, JSON.stringify({ pid })) - return { acquired: true, release: () => rmSync(path, { force: true }) } + await Bun.write(path, JSON.stringify({ pid })) + return { acquired: true, release: () => removePidfile(path) } } diff --git a/src/faculties/tests/instance-lock.spec.ts b/src/faculties/tests/instance-lock.spec.ts index 955fefb05..6d7f07a1c 100644 --- a/src/faculties/tests/instance-lock.spec.ts +++ b/src/faculties/tests/instance-lock.spec.ts @@ -1,5 +1,5 @@ import { describe, expect, test } from 'bun:test' -import { existsSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs' +import { mkdtempSync, rmSync } from 'node:fs' import { tmpdir } from 'node:os' import { join } from 'node:path' import { acquireInstanceLock, type InstanceLock, instancePidfilePath } from '../instance-lock.ts' @@ -14,24 +14,24 @@ const acquired = (lock: InstanceLock | { acquired: false; pid: number }): Instan } describe('acquireInstanceLock', () => { - test('acquiring a free home writes the pidfile with the given pid', () => { + test('acquiring a free home writes the pidfile with the given pid', async () => { const home = tempHome() try { - const lock = acquired(acquireInstanceLock({ home, pid: 4242 })) + const lock = acquired(await acquireInstanceLock({ home, pid: 4242 })) const path = instancePidfilePath(home) - expect(existsSync(path)).toBe(true) - expect(JSON.parse(readFileSync(path, 'utf8'))).toEqual({ pid: 4242 }) - lock.release() + expect(await Bun.file(path).exists()).toBe(true) + expect(await Bun.file(path).json()).toEqual({ pid: 4242 }) + await lock.release() } finally { rmSync(home, { recursive: true, force: true }) } }) - test('a pidfile held by a live pid blocks acquisition and reports the pid', () => { + test('a pidfile held by a live pid blocks acquisition and reports the pid', async () => { const home = tempHome() try { - acquired(acquireInstanceLock({ home, pid: process.pid })) - const second = acquireInstanceLock({ home, pid: 1 }) + await acquireInstanceLock({ home, pid: process.pid }) + const second = await acquireInstanceLock({ home, pid: 1 }) expect(second.acquired).toBe(false) if (!second.acquired) expect(second.pid).toBe(process.pid) } finally { @@ -39,7 +39,7 @@ describe('acquireInstanceLock', () => { } }) - test('a stale pidfile (pid no longer alive) is reaped and acquisition succeeds', () => { + test('a stale pidfile (pid no longer alive) is reaped and acquisition succeeds', async () => { const home = tempHome() const path = instancePidfilePath(home) // A pid that cannot exist: high fixed value, no process holds it. EPERM @@ -51,18 +51,18 @@ describe('acquireInstanceLock', () => { } catch (error) { expect((error as NodeJS.ErrnoException).code).toBe('ESRCH') } - writeFileSync(path, JSON.stringify({ pid: deadPid })) - const lock = acquired(acquireInstanceLock({ home, pid: process.pid })) - expect(JSON.parse(readFileSync(path, 'utf8'))).toEqual({ pid: process.pid }) - lock.release() + await Bun.write(path, JSON.stringify({ pid: deadPid })) + const lock = acquired(await acquireInstanceLock({ home, pid: process.pid })) + expect(await Bun.file(path).json()).toEqual({ pid: process.pid }) + await lock.release() }) - test('release removes the pidfile', () => { + test('release removes the pidfile', async () => { const home = tempHome() try { - const lock = acquired(acquireInstanceLock({ home, pid: process.pid })) - lock.release() - expect(existsSync(instancePidfilePath(home))).toBe(false) + const lock = acquired(await acquireInstanceLock({ home, pid: process.pid })) + await lock.release() + expect(await Bun.file(instancePidfilePath(home)).exists()).toBe(false) } finally { rmSync(home, { recursive: true, force: true }) } From 8af653531095e750a9429a1e490e990f82299ee9 Mon Sep 17 00:00:00 2001 From: Edward Irby Date: Thu, 24 Sep 2026 00:31:29 -0700 Subject: [PATCH 07/55] feat: static GUI serving on the instance socket + --dev MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Slice 5: the host single listener now serves the bundled controller GUI at /.behavioral/connect.js alongside the WebSocket upgrade — the browser carrier and the TUI carrier are two clients of one Bun.serve (Carriers/H). bundleController is promoted from the controller fixture to src/controller/ (one home for the bundle; the fixture re-exports it). In prod the AOT bundle is built once and cached; --dev (start-time only, attachers cannot flip a running instance) rebuilds per request for source editing. Engine and wire are identical in both modes. The router default forwards argv so the bin entry recognizes the flag. --- bin/behavioral.ts | 8 ++- src/cli/attach-or-start.ts | 9 ++- src/cli/socket-host.ts | 27 +++++++-- src/cli/tests/attach-or-start.spec.ts | 77 +++++++++++++++++++++++++ src/cli/tests/socket-host-gui.spec.ts | 70 +++++++++++++++++++++++ src/controller/bundle-controller.ts | 81 +++++++++++++++++++++++++++ 6 files changed, 262 insertions(+), 10 deletions(-) create mode 100644 src/cli/tests/socket-host-gui.spec.ts create mode 100644 src/controller/bundle-controller.ts diff --git a/bin/behavioral.ts b/bin/behavioral.ts index ea77d60da..2ba6b4aca 100755 --- a/bin/behavioral.ts +++ b/bin/behavioral.ts @@ -22,11 +22,13 @@ export const runCli = makeCliRouter({ }, }, // The bare command is attach-or-start: attach to a running instance over - // /instance.sock, or start the foreground instance. Lazy: --help and + // /instance.sock, or start the foreground instance. --dev is a + // start-time flag: it configures the instance THIS process starts; an + // attaching process cannot flip a running instance. Lazy: --help and // the subcommands must not load the composition graph until invoked. - default: async () => { + default: async (args: string[]) => { const { attachOrStart } = await import('../src/cli/attach-or-start.ts') - await attachOrStart() + await attachOrStart({ dev: args.includes('--dev') }) }, }) diff --git a/src/cli/attach-or-start.ts b/src/cli/attach-or-start.ts index eaf1ef82b..3cb9d2a2a 100644 --- a/src/cli/attach-or-start.ts +++ b/src/cli/attach-or-start.ts @@ -17,6 +17,12 @@ const defaultCreateRuntime = async (): Promise => { export type AttachOrStartOptions = { /** The single `` root; defaults to `behavioralHome()`. */ home?: string + /** + * Start-time only: serve the controller GUI with per-request rebundling. + * An attaching process cannot flip a running instance — the flag reaches + * the instance the caller starts, never one it attaches to. + */ + dev?: boolean /** The TUI's line source; defaults to stdin. */ input?: NodeJS.ReadableStream /** The TUI's terminal writer; defaults to stdout. */ @@ -50,6 +56,7 @@ export const attachOrStart = async ({ process.stdout.write(text) }, createRuntime = defaultCreateRuntime, + dev = false, }: AttachOrStartOptions = {}): Promise => { const lock = await acquireInstanceLock({ home }) if (!lock.acquired) { @@ -66,7 +73,7 @@ export const attachOrStart = async ({ // Start: foreground engine + socket host + TUI (a socket client like every // other client — no in-process fast path). const runtime = await createRuntime() - const host = await createSocketHost({ runtime, home }) + const host = await createSocketHost({ runtime, home, dev }) let cleaned = false const cleanup = async (): Promise => { if (cleaned) return diff --git a/src/cli/socket-host.ts b/src/cli/socket-host.ts index 9a2aee8e6..74ff9db2b 100644 --- a/src/cli/socket-host.ts +++ b/src/cli/socket-host.ts @@ -1,5 +1,6 @@ import { join } from 'node:path' import type { ServerWebSocket } from 'bun' +import { bundleController, CONNECT_BEHAVIORAL_ROUTE } from '../controller/bundle-controller.ts' import { behavioralHome } from '../faculties/behavioral-home.ts' import type { JsonRpcMessage } from './json-rpc.ts' import { dispatchToRuntime, type HostRuntime, wireRuntimeEgress } from './serve.ts' @@ -33,9 +34,15 @@ export type SocketHost = { /** * Start the attach lane: a unix-socket `Bun.serve` over the shared host * dispatcher, with redacted traces and `ui_*` selections fanning out to every - * connected client. + * connected client. The same listener serves the bundled controller GUI at + * {@link CONNECT_BEHAVIORAL_ROUTE} — the browser's carrier and the TUI's + * carrier are two clients of one host (Carriers/H). * * @remarks + * With `dev: true` the controller bundle is rebuilt per request (the Bun + * fullstack-dev-server surface lands when the GUI entry exists); without it + * the production bundle is built once and cached. Engine/wire are identical + * in both modes — `--dev` gates the GUI-serving surface only. * Unlike {@link createHost}, this does NOT call `runtime.start()` — the * foreground entry composes the runtime, the socket host, and its clients, * then starts the composition itself. @@ -45,9 +52,12 @@ export type SocketHost = { export const createSocketHost = async ({ runtime, home = behavioralHome(), + dev = false, }: { runtime: HostRuntime home?: string + /** Rebuild the controller bundle per request instead of caching it. */ + dev?: boolean }): Promise => { const path = instanceSocketPath(home) // A socket file left by a dead instance cannot be bound again — remove it @@ -112,12 +122,17 @@ export const createSocketHost = async ({ clients.delete(ws) }, }, - fetch: (_req, server) => - // Later slices serve the controller GUI here (Carriers/H); today the - // unix carrier is WebSocket-only. - server.upgrade(_req) + fetch: async (req, server) => { + const url = new URL(req.url) + if (url.pathname === CONNECT_BEHAVIORAL_ROUTE) { + // Prod: one AOT bundle, cached. Dev: rebundle per request. + const routes = await bundleController({ dev }) + return routes[CONNECT_BEHAVIORAL_ROUTE] ?? new Response(null, { status: 404 }) + } + return server.upgrade(req) ? undefined - : new Response('behavioral instance socket — a WebSocket upgrade is required\n', { status: 426 }), + : new Response('behavioral instance socket — a WebSocket upgrade is required\n', { status: 426 }) + }, }) // Egress: one redaction pass, the JSONL log, then fan out to every client. diff --git a/src/cli/tests/attach-or-start.spec.ts b/src/cli/tests/attach-or-start.spec.ts index 3b5ec64c5..4848f2ab8 100644 --- a/src/cli/tests/attach-or-start.spec.ts +++ b/src/cli/tests/attach-or-start.spec.ts @@ -2,7 +2,12 @@ import { afterAll, describe, expect, test } from 'bun:test' import { existsSync, mkdirSync, rmSync } from 'node:fs' import { tmpdir } from 'node:os' import { join } from 'node:path' +import { Readable } from 'node:stream' +import { TRACE_MESSAGE_KINDS } from '../../behavioral/behavioral.constants.ts' +import type { Trace } from '../../behavioral/behavioral.types.ts' +import { CONNECT_BEHAVIORAL_ROUTE } from '../../controller/bundle-controller.ts' import { instancePidfilePath } from '../../faculties/instance-lock.ts' +import type { HostRuntime } from '../serve.ts' import { instanceSocketPath } from '../socket-host.ts' const homes: string[] = [] @@ -105,3 +110,75 @@ describe('attachOrStart — the two-process lifecycle', () => { await instance.exited }, 40_000) }) + +/** A still-open readable whose buffered contents readline consumes. */ +const scriptInput = (lines: string[]): Readable => { + const stream = new Readable({ read() {} }) + for (const line of lines) stream.push(line) + return stream +} + +/** The echo runtime of the child fixture, in-process: selections echo traces. */ +const echoRuntime = (): HostRuntime => { + const instanceId = Bun.randomUUIDv7() + const listeners = new Set<(trace: Trace) => void>() + const emit = (trace: Trace): void => { + for (const listener of listeners) listener(trace) + } + const base = { instanceId, sessionId: instanceId } + return { + trigger: (event) => + emit({ + kind: TRACE_MESSAGE_KINDS.selection, + timestamp: Date.now(), + step: 1, + ...base, + selected: { priority: 0, type: event.type, detail: event.detail }, + }), + useTrace: (l) => { + listeners.add(l) + return () => { + listeners.delete(l) + } + }, + start: () => emit({ kind: TRACE_MESSAGE_KINDS.idle, timestamp: Date.now(), step: 1, ...base }), + terminate: () => {}, + } +} + +describe('attachOrStart — the --dev flag', () => { + test('start-time only: dev: true serves the dev bundle; the attach path has no dev knob', async () => { + const { attachOrStart } = await import('../attach-or-start.ts') + const home = tempHome() + // The input stays open until the assertions are done. + const input = scriptInput(['/kick\n']) + const pending = attachOrStart({ home, input, write: () => {}, createRuntime: echoRuntime, dev: true }) + await eventually(() => existsSync(instanceSocketPath(home)), 'started instance socket') + // The started instance serves the controller bundle on the same listener. + const response = await fetch(`http://localhost${CONNECT_BEHAVIORAL_ROUTE}`, { unix: instanceSocketPath(home) }) + expect(response.status).toBe(200) + expect(await response.text()).toContain('ui_render') + // Stdin ends → clean detach with pidfile/socket cleanup. + input.push(null) + const result = await pending + expect(result.attached).toBe(false) + expect(existsSync(instanceSocketPath(home))).toBe(false) + }, 30_000) + + test('an attacher cannot flip a running instance into dev mode (start-time only)', async () => { + const { attachOrStart } = await import('../attach-or-start.ts') + const home = tempHome() + await Bun.write(join(home, 'instance.pid'), JSON.stringify({ pid: process.pid })) + // A live pidfile → attach path: no dev option is even accepted to reach a + // runtime — attaching starts nothing and reconfigures nothing. + const result = await attachOrStart({ + home, + input: scriptInput([]), + write: () => {}, + createRuntime: () => { + throw new Error('attach must never start a runtime') + }, + }) + expect(result.attached).toBe(true) + }) +}) diff --git a/src/cli/tests/socket-host-gui.spec.ts b/src/cli/tests/socket-host-gui.spec.ts new file mode 100644 index 000000000..9e8f63d0e --- /dev/null +++ b/src/cli/tests/socket-host-gui.spec.ts @@ -0,0 +1,70 @@ +import { afterAll, describe, expect, test } from 'bun:test' +import { mkdtempSync, rmSync } from 'node:fs' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import type { Trace } from '../../behavioral/behavioral.types.ts' +import { CONNECT_BEHAVIORAL_ROUTE } from '../../controller/bundle-controller.ts' +import type { HostRuntime } from '../serve.ts' +import { createSocketHost } from '../socket-host.ts' + +const homes: string[] = [] +const tempHome = (): string => { + const home = mkdtempSync(join(tmpdir(), 'behavioral-socket-gui-')) + homes.push(home) + return home +} + +afterAll(() => { + for (const home of homes) rmSync(home, { recursive: true, force: true }) +}) + +const fakeRuntime = (): HostRuntime => { + const listeners = new Set<(trace: Trace) => void>() + return { + trigger: () => {}, + useTrace: (l) => { + listeners.add(l) + return () => { + listeners.delete(l) + } + }, + start: () => {}, + terminate: () => {}, + } +} + +describe('createSocketHost — the GUI carrier', () => { + test('serves the bundled controller runtime at the connect route', async () => { + const host = await createSocketHost({ runtime: fakeRuntime(), home: tempHome() }) + const response = await fetch(`http://localhost${CONNECT_BEHAVIORAL_ROUTE}`, { unix: host.path }) + expect(response.status).toBe(200) + expect(response.headers.get('content-type')).toBe('text/javascript;charset=utf-8') + // The gzipped bundle decodes to real controller runtime: the ui_render + // handler wire type and the transport's WebSocket reconnect are present. + const body = await response.text() + expect(body).toContain('ui_render') + expect(body).toContain('WebSocket') + await host.close() + }, 30_000) + + test('non-connect paths stay on the 426 WebSocket-only carrier', async () => { + const host = await createSocketHost({ runtime: fakeRuntime(), home: tempHome() }) + const response = await fetch('http://localhost/other', { unix: host.path }) + expect(response.status).toBe(426) + await host.close() + }) + + test('dev mode rebuilds the bundle per request; prod mode caches it', async () => { + const host = await createSocketHost({ runtime: fakeRuntime(), home: tempHome(), dev: true }) + const first = await fetch(`http://localhost${CONNECT_BEHAVIORAL_ROUTE}`, { unix: host.path }) + expect(first.status).toBe(200) + const second = await fetch(`http://localhost${CONNECT_BEHAVIORAL_ROUTE}`, { unix: host.path }) + expect(second.status).toBe(200) + // Both responses decode to the controller runtime (rebundled or cached — + // the per-request rebuild is exercised by the bundle spec; here the dev + // host serves valid, current bundles on every request). + expect(await first.text()).toContain('ui_render') + expect(await second.text()).toContain('ui_render') + await host.close() + }, 30_000) +}) diff --git a/src/controller/bundle-controller.ts b/src/controller/bundle-controller.ts new file mode 100644 index 000000000..13d6f29e4 --- /dev/null +++ b/src/controller/bundle-controller.ts @@ -0,0 +1,81 @@ +/** + * Bundles the controller runtime into a gzipped module served at the connect + * route. The bundled module reads `modules` query params, loads any extension + * modules, constructs a {@link Controller}, and connects. + */ + +/** HTTP route where the bundled controller JS is served. */ +export const CONNECT_BEHAVIORAL_ROUTE = '/.behavioral/connect.js' + +/** + * Virtual entrypoint path for Bun.build. Must match a key in the `files` map. + * Bun transpiles the `.ts` extension natively, and virtual files from the `files` + * option take priority over disk — no actual file needs to exist at this path. + */ +const VIRTUAL_ENTRY = '/.behavioral/connect.ts' + +/** + * Bundles the controller runtime into a gzipped module served at the connect + * route. The bundled module reads `modules` query params, loads any extension + * modules, constructs a {@link Controller}, and connects. + * + * With `dev: true` the bundle is built unminified per call — the caller + * rebundles on every request instead of caching the production artifact. + */ +export const bundleController = async ({ dev = false }: { dev?: boolean } = {}) => { + const controllerEntry = Bun.resolveSync('../controller.ts', import.meta.dir) + const entrySource = ` +import { Controller } from ${JSON.stringify(controllerEntry)} + +const params = new URL(import.meta.url).searchParams + +// Load optional extension modules specified as comma-separated paths. +// Each module must export: +// - key: the b-trigger pair string this extension handles (e.g. "click:my_action") +// - default: a ControllerExtension function +const modulePaths = (params.get('modules') ?? '').split(',').map(function (s) { return s.trim() }).filter(Boolean) +const extEntries = await Promise.all( + modulePaths.map(async function (path) { + const mod = await import(path) + if (typeof mod.key !== 'string') { + throw new Error( + 'Extension module "' + path + '" is missing a string key export. ' + + 'Each extension module must export a key string (e.g. "click:my_action").', + ) + } + if (typeof mod.default !== 'function') { + throw new Error( + 'Extension module "' + path + '" has invalid default export. Expected a function, got ' + typeof mod.default + '.', + ) + } + return [mod.key, mod.default] + }), +) +const extensions = new Map(extEntries) + +const controller = new Controller({ extensions }) +controller.connect() +` + const { outputs, logs, success } = await Bun.build({ + entrypoints: [VIRTUAL_ENTRY], + files: { + [VIRTUAL_ENTRY]: entrySource, + }, + minify: !dev, + target: 'browser', + }) + if (!success) { + throw new AggregateError(logs, 'Failed to build behavioral controller runtime') + } + const artifact = outputs[0]! + const content = await artifact.text() + const compressed = Bun.gzipSync(content) + return { + [CONNECT_BEHAVIORAL_ROUTE]: new Response(compressed as BodyInit, { + headers: new Headers({ + 'content-type': artifact.type, + 'content-encoding': 'gzip', + }), + }), + } +} From cd5c20fc80f729981670b5e3f01bf265079a6bdd Mon Sep 17 00:00:00 2001 From: Edward Irby Date: Thu, 24 Sep 2026 00:32:11 -0700 Subject: [PATCH 08/55] =?UTF-8?q?refactor:=20one=20bundle=20home=20?= =?UTF-8?q?=E2=80=94=20the=20fixture=20re-exports=20the=20production=20bun?= =?UTF-8?q?dler?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .../tests/fixtures/bundle-controller.ts | 79 +------------------ 1 file changed, 3 insertions(+), 76 deletions(-) diff --git a/src/controller/tests/fixtures/bundle-controller.ts b/src/controller/tests/fixtures/bundle-controller.ts index 575f49531..726e82566 100644 --- a/src/controller/tests/fixtures/bundle-controller.ts +++ b/src/controller/tests/fixtures/bundle-controller.ts @@ -1,78 +1,5 @@ /** - * Bundles the controller runtime into a gzipped module served at the connect - * route. The bundled module reads `modules` query params, loads any extension - * modules, constructs a {@link Controller}, and connects. + * Re-export of the production bundler — the fixture server and the instance + * socket host share one bundle home (no forked copies). */ - -/** HTTP route where the bundled controller JS is served. */ -export const CONNECT_BEHAVIORAL_ROUTE = '/.behavioral/connect.js' - -/** - * Virtual entrypoint path for Bun.build. Must match a key in the `files` map. - * Bun transpiles the `.ts` extension natively, and virtual files from the `files` - * option take priority over disk — no actual file needs to exist at this path. - */ -const VIRTUAL_ENTRY = '/.behavioral/connect.ts' - -/** - * Bundles the controller runtime into a gzipped module served at the connect - * route. The bundled module reads `modules` query params, loads any extension - * modules, constructs a {@link Controller}, and connects. - */ -export const bundleController = async () => { - const controllerEntry = Bun.resolveSync('../../controller.ts', import.meta.dir) - const entrySource = ` -import { Controller } from ${JSON.stringify(controllerEntry)} - -const params = new URL(import.meta.url).searchParams - -// Load optional extension modules specified as comma-separated paths. -// Each module must export: -// - key: the b-trigger pair string this extension handles (e.g. "click:my_action") -// - default: a ControllerExtension function -const modulePaths = (params.get('modules') ?? '').split(',').map(function (s) { return s.trim() }).filter(Boolean) -const extEntries = await Promise.all( - modulePaths.map(async function (path) { - const mod = await import(path) - if (typeof mod.key !== 'string') { - throw new Error( - 'Extension module "' + path + '" is missing a string key export. ' + - 'Each extension module must export a key string (e.g. "click:my_action").', - ) - } - if (typeof mod.default !== 'function') { - throw new Error( - 'Extension module "' + path + '" has invalid default export. Expected a function, got ' + typeof mod.default + '.', - ) - } - return [mod.key, mod.default] - }), -) -const extensions = new Map(extEntries) - -const controller = new Controller({ extensions }) -controller.connect() -` - const { outputs, logs, success } = await Bun.build({ - entrypoints: [VIRTUAL_ENTRY], - files: { - [VIRTUAL_ENTRY]: entrySource, - }, - minify: true, - target: 'browser', - }) - if (!success) { - throw new AggregateError(logs, 'Failed to build behavioral controller runtime') - } - const artifact = outputs[0]! - const content = await artifact.text() - const compressed = Bun.gzipSync(content) - return { - [CONNECT_BEHAVIORAL_ROUTE]: new Response(compressed as BodyInit, { - headers: new Headers({ - 'content-type': artifact.type, - 'content-encoding': 'gzip', - }), - }), - } -} +export { bundleController, CONNECT_BEHAVIORAL_ROUTE } from '../../bundle-controller.ts' From 159038cbbae9bc56772ce2c895f56d061f965d7e Mon Sep 17 00:00:00 2001 From: Edward Irby Date: Fri, 25 Sep 2026 12:52:09 -0700 Subject: [PATCH 09/55] feat(shell): the generic fetch-based JSON-RPC 2.0 client MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Slice 1 of the mcp-into-shell handoff (.prompts/mcp-into-shell-security-faculty.md): the lightweight HTTP JSON-RPC client that replaces the @modelcontextprotocol SDK transport layer. - `send({ url, method, params, id, getAuthToken, fetch })` — one stateless HTTP POST per call; envelope is `{ jsonrpc, id, method, params }`. - Protocol-agnostic: no MCP knowledge, no `_meta`, no protocol versions — the MCP layering lives in the thread pack (later slice), not the client. - Auth is a seam: injectable async `getAuthToken`; a vended token rides a bearer header, absent means no header. The client does not know OAuth. - Errors-as-data: HTTP non-OK, JSON-RPC error payloads, malformed bodies, and network failures return `{ ok: false, error: { code, message } }` — no throw crosses a transport/protocol failure. - `fetch` injectable; specs script it to exercise the real request/decode paths (spec: shell/tests/rpc.client.spec.ts, 8 specs). --- src/faculties/shell/rpc.client.ts | 118 ++++++++++++++++++ src/faculties/shell/tests/rpc.client.spec.ts | 124 +++++++++++++++++++ 2 files changed, 242 insertions(+) create mode 100644 src/faculties/shell/rpc.client.ts create mode 100644 src/faculties/shell/tests/rpc.client.spec.ts diff --git a/src/faculties/shell/rpc.client.ts b/src/faculties/shell/rpc.client.ts new file mode 100644 index 000000000..f230cf7d3 --- /dev/null +++ b/src/faculties/shell/rpc.client.ts @@ -0,0 +1,118 @@ +/** + * The generic HTTP JSON-RPC 2.0 client — a `fetch`-based envelope carrier + * used by the shell faculty's `rpc` op. + * + * @remarks + * Protocol-agnostic by construction: the client knows nothing about MCP, + * `_meta`, protocol versions, or tool semantics — it carries the JSON-RPC + * envelope (`{ jsonrpc: "2.0", id, method, params }`) over one stateless + * HTTP POST per call. The MCP layering (request stamping, `server/discover`, + * `tools/call`, MRTR) lives in the remote-mcp thread pack, not here. + * + * Auth is a seam, not a capability: the client does not know about OAuth — + * it asks the injectable `getAuthToken` for a token and rides it as a + * bearer header when one is vended. (The credential seam's other end is the + * security faculty; wiring is the composition's, not this module's.) + * + * Errors-as-data (the op-runners law): HTTP non-OK, JSON-RPC error payloads, + * malformed responses, and network failures all return + * `{ ok: false, error: { code, message } }` — no throw crosses a + * transport/protocol failure. Only a caller-side programming error (an + * unserializable envelope) throws, and it never reaches this client. + * + * `fetch` is injectable for tests; the default is the platform global. + * + * @packageDocumentation + */ + +import type { JsonObject } from '../../behavioral/behavioral.types.ts' + +/** The outcome of one RPC call — the modified-B envelope, errors-as-data. */ +export type RpcResult = + | { ok: true; result: T } + | { ok: false; error: { code: number | string; message: string } } + +/** A token vendor — the auth seam's client-side end. `undefined` = no token. */ +export type GetAuthToken = () => Promise + +export type SendInput = { + url: string + method: string + params?: JsonObject + /** JSON-RPC correlation id. Defaults to a fresh UUID — stateless calls need one to match responses. */ + id?: string | number + /** Optional bearer-token vendor, consulted per call. */ + getAuthToken?: GetAuthToken + /** Injectable transport. Defaults to the platform `fetch`. */ + fetch?: typeof fetch +} + +/** Build the JSON-RPC 2.0 request envelope for one call. */ +const envelope = ({ method, params, id }: { method: string; params?: JsonObject; id: string | number }) => + ({ + jsonrpc: '2.0', + id, + method, + ...(params === undefined ? {} : { params }), + }) as JsonObject + +/** Decode one HTTP response body as a JSON-RPC 2.0 outcome. */ +const decodeBody = async ({ response, id }: { response: Response; id: string | number }): Promise => { + let parsed: unknown + try { + parsed = await response.json() + } catch { + return { ok: false, error: { code: 'invalid_response', message: 'response body is not JSON' } } + } + const body = parsed as { jsonrpc?: unknown; id?: unknown; result?: unknown; error?: unknown } + if (typeof body === 'object' && body !== null && body.jsonrpc === '2.0' && (body.id === id || body.id === null)) { + if (body.error !== undefined && typeof body.error === 'object' && body.error !== null) { + const err = body.error as { code?: unknown; message?: unknown } + return { + ok: false, + error: { + code: typeof err.code === 'number' || typeof err.code === 'string' ? err.code : 'invalid_response', + message: typeof err.message === 'string' ? err.message : 'JSON-RPC error (no message)', + }, + } + } + return { ok: true, result: (body.result ?? {}) as JsonObject } + } + return { ok: false, error: { code: 'invalid_response', message: 'response is not a JSON-RPC 2.0 response' } } +} + +/** + * Send one JSON-RPC 2.0 request over a single stateless HTTP POST. + * + * @returns the result branch on success; error data on HTTP failure, JSON-RPC + * error payloads, malformed responses, and network failures — never a throw. + */ +export const send = async ({ + url, + method, + params, + id = crypto.randomUUID(), + getAuthToken, + fetch: fetchImpl = fetch, +}: SendInput): Promise => { + const token = getAuthToken === undefined ? undefined : await getAuthToken().catch(() => undefined) + const headers: Record = { 'content-type': 'application/json', accept: 'application/json' } + if (token !== undefined) headers.authorization = `Bearer ${token}` + let response: Response + try { + response = await fetchImpl(url, { + method: 'POST', + headers, + body: JSON.stringify(envelope({ method, params, id })), + }) + } catch (err) { + return { + ok: false, + error: { code: 'network', message: err instanceof Error ? err.message : String(err) }, + } + } + if (!response.ok) { + return { ok: false, error: { code: response.status, message: `HTTP ${response.status} ${response.statusText}` } } + } + return decodeBody({ response, id }) +} diff --git a/src/faculties/shell/tests/rpc.client.spec.ts b/src/faculties/shell/tests/rpc.client.spec.ts new file mode 100644 index 000000000..f8739bb01 --- /dev/null +++ b/src/faculties/shell/tests/rpc.client.spec.ts @@ -0,0 +1,124 @@ +import { describe, expect, test } from 'bun:test' +import type { JsonObject } from '../../../behavioral/behavioral.types.ts' +import { send } from '../rpc.client.ts' + +/** + * RPC client specs — the fetch mock is the only double: the boundary under + * test IS the injected fetch, so a scripted `fetch` exercises the real + * request-building and response-decoding paths. + * + * @packageDocumentation + */ + +/** A scripted fetch: captures calls, returns queued responses. */ +const fetchMock = (responses: Array) => { + const calls: Array<{ url: string; init: RequestInit }> = [] + const impl = (async (url: string | URL, init?: RequestInit) => { + calls.push({ url: String(url), init: init ?? {} }) + const next = responses.shift() + if (next instanceof Error) throw next + if (next === undefined) throw new Error('no scripted response') + return next + }) as unknown as typeof fetch + return { impl, calls } +} + +const jsonResponse = (body: unknown, init?: ResponseInit): Response => + new Response(JSON.stringify(body), { status: 200, headers: { 'content-type': 'application/json' }, ...init }) + +const rpcResponse = (id: string | number, result: unknown): Response => jsonResponse({ jsonrpc: '2.0', id, result }) + +describe('rpc.client', () => { + test('POSTs a JSON-RPC 2.0 envelope with the method and params', async () => { + const params = { name: 'ping' } as JsonObject + const { impl, calls } = fetchMock([rpcResponse('r1', { pong: true })]) + const outcome = await send({ url: 'https://rpc.example/mcp', method: 'tools/list', params, id: 'r1', fetch: impl }) + expect(outcome.ok).toBe(true) + const call = calls[0]! + expect(call.url).toBe('https://rpc.example/mcp') + expect(call.init.method).toBe('POST') + const headers = new Headers(call.init.headers) + expect(headers.get('content-type')).toBe('application/json') + const body = JSON.parse(String(call.init.body)) as { jsonrpc: string; id: string; method: string; params: unknown } + expect(body).toEqual({ jsonrpc: '2.0', id: 'r1', method: 'tools/list', params }) + }) + + test('carries an Authorization bearer header when a token is provided', async () => { + const { impl, calls } = fetchMock([rpcResponse(1, {})]) + await send({ + url: 'https://rpc.example/mcp', + method: 'ping', + id: 1, + fetch: impl, + getAuthToken: async () => 'tok-123', + }) + const headers = new Headers(calls[0]!.init.headers) + expect(headers.get('authorization')).toBe('Bearer tok-123') + }) + + test('sends no Authorization header when no token is available', async () => { + const { impl, calls } = fetchMock([rpcResponse(1, {})]) + await send({ + url: 'https://rpc.example/mcp', + method: 'ping', + id: 1, + fetch: impl, + getAuthToken: async () => undefined, + }) + const headers = new Headers(calls[0]!.init.headers) + expect(headers.get('authorization')).toBeNull() + }) + + test('is protocol-agnostic: a plain echo method carries no MCP framing', async () => { + const { impl, calls } = fetchMock([rpcResponse(2, { echoed: true })]) + const outcome = await send({ + url: 'https://rpc.example/api', + method: 'math/add', + params: { a: 1, b: 2 } as JsonObject, + id: 2, + fetch: impl, + }) + expect(outcome.ok).toBe(true) + const body = JSON.parse(String(calls[0]!.init.body)) as Record + expect(Object.keys(body).sort()).toEqual(['id', 'jsonrpc', 'method', 'params']) + }) + + test('omits params from the envelope when none are given', async () => { + const { impl, calls } = fetchMock([rpcResponse(3, null)]) + await send({ url: 'https://rpc.example/api', method: 'ping', id: 3, fetch: impl }) + const body = JSON.parse(String(calls[0]!.init.body)) as Record + expect('params' in body).toBe(false) + }) + + test('HTTP non-OK is error data, not a throw', async () => { + const { impl } = fetchMock([new Response('nope', { status: 503 })]) + const outcome = await send({ url: 'https://rpc.example/mcp', method: 'tools/list', id: 'r2', fetch: impl }) + expect(outcome.ok).toBe(false) + if (!outcome.ok) { + expect(outcome.error.code).toBe(503) + expect(outcome.error.message).toContain('503') + } + }) + + test('a JSON-RPC error payload is error data carrying its code', async () => { + const { impl } = fetchMock([ + jsonResponse({ jsonrpc: '2.0', id: 'r3', error: { code: -32601, message: 'Method not found' } }), + ]) + const outcome = await send({ url: 'https://rpc.example/mcp', method: 'nope/x', id: 'r3', fetch: impl }) + expect(outcome.ok).toBe(false) + if (!outcome.ok) { + expect(outcome.error.code).toBe(-32601) + expect(outcome.error.message).toBe('Method not found') + } + }) + + test('a network failure is error data, not a throw', async () => { + const { impl } = fetchMock([new Error('connection refused')]) + const outcome = await send({ url: 'https://rpc.example/mcp', method: 'ping', id: 'r4', fetch: impl }) + expect(outcome.ok).toBe(false) + if (!outcome.ok) { + expect(outcome.error.code).toBe('network') + expect(outcome.error.message).toContain('connection refused') + } + }) +}) From 8059289a75482633ba784aa2043421874acda4e5 Mon Sep 17 00:00:00 2001 From: Edward Irby Date: Fri, 25 Sep 2026 12:59:29 -0700 Subject: [PATCH 10/55] =?UTF-8?q?feat(shell):=20the=20`rpc`=20op=20?= =?UTF-8?q?=E2=80=94=20a=20generic=20remote=20JSON-RPC=20transport=20varia?= =?UTF-8?q?nt?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Slice 2 of the mcp-into-shell handoff: remote tool execution as a third op beside `run` and `shell` — transport-shaped, not MCP-shaped. - `ShellRpcOpInput { op: 'rpc', url, method, params?, timeoutMs? }` joins the op-discriminated `ShellCallInput` union (anyOf branch, strict AJV). - `runOp` dispatches `rpc` to the Slice-1 client; `getAuthToken` is an injectable module seam returning undefined (the security faculty's credential round-trip lands in Slice 4). - Cancel rides the existing `active` map: an rpc `Execution` carries an AbortController; `shell_cancel` aborts the in-flight fetch (first stop wins — an abort is never mislabeled a remote error). - Errors-as-data at both layers: remote JSON-RPC errors and HTTP non-OK surface as `code: 'error'` + `remoteCode` (the remote discriminant, for retry policy); cancel/timeout as their own statuses. - Specs through the real faculty process boundary against a local JSON-RPC HTTP server (rpc-op.spec.ts, 6 specs); the client gains a signal seam with an abort spec (rpc.client.spec.ts, 9 specs). --- src/faculties/shell/faculty.ts | 111 ++++++++++--- src/faculties/shell/rpc.client.ts | 4 + src/faculties/shell/tests/rpc-op.spec.ts | 155 +++++++++++++++++++ src/faculties/shell/tests/rpc.client.spec.ts | 14 ++ src/faculties/shell/types.ts | 65 +++++++- 5 files changed, 327 insertions(+), 22 deletions(-) create mode 100644 src/faculties/shell/tests/rpc-op.spec.ts diff --git a/src/faculties/shell/faculty.ts b/src/faculties/shell/faculty.ts index c99edba28..7b77e95cf 100644 --- a/src/faculties/shell/faculty.ts +++ b/src/faculties/shell/faculty.ts @@ -48,11 +48,16 @@ import { ajv } from '../../behavioral/behavioral.types.ts' import { FACULTY_MESSAGE_KINDS } from '../faculties.constants.ts' import { type ShellRequestEvent, validateShellCancelEvent, validateShellRequestEvent } from '../faculties.types.ts' import { emit, wireInbound } from '../process-lane.ts' +import { type GetAuthToken, send as sendRpc } from './rpc.client.ts' import { + type RpcOpError, + type RpcOpSuccess, type ShellCallInput, ShellCallInputSchema, type ShellError, + type ShellOpResult, type ShellOptions, + type ShellRpcOpInput, type ShellStatus, type ShellSuccess, } from './types.ts' @@ -128,14 +133,16 @@ const CLAMPED_KEYS = ['timeoutMs', 'maxLines', 'maxCharacters', 'limit'] as cons const clampOptions = (input: ShellCallInput): { options: ShellOptions; clamped: string[] } => { const clamped: string[] = [] const options: ShellOptions = { - ...(input.format === undefined ? {} : { format: input.format }), - ...(input.offset === undefined ? {} : { offset: input.offset }), - ...(input.limit === undefined ? {} : { limit: input.limit }), - ...(input.timeoutMs === undefined ? {} : { timeoutMs: input.timeoutMs }), - ...(input.maxLines === undefined ? {} : { maxLines: input.maxLines }), - ...(input.maxCharacters === undefined ? {} : { maxCharacters: input.maxCharacters }), - ...(input.cwd === undefined ? {} : { cwd: input.cwd }), - ...(input.env === undefined ? {} : { env: input.env }), + // `in` narrowing: the knob fields exist per op branch (the rpc op carries + // only `timeoutMs`), so each read is guarded by its own presence check. + ...('format' in input && input.format !== undefined ? { format: input.format } : {}), + ...('offset' in input && input.offset !== undefined ? { offset: input.offset } : {}), + ...('limit' in input && input.limit !== undefined ? { limit: input.limit } : {}), + ...('timeoutMs' in input && input.timeoutMs !== undefined ? { timeoutMs: input.timeoutMs } : {}), + ...('maxLines' in input && input.maxLines !== undefined ? { maxLines: input.maxLines } : {}), + ...('maxCharacters' in input && input.maxCharacters !== undefined ? { maxCharacters: input.maxCharacters } : {}), + ...('cwd' in input && input.cwd !== undefined ? { cwd: input.cwd } : {}), + ...('env' in input && input.env !== undefined ? { env: input.env } : {}), // The stdin field exists only on the 'shell' branch; the worker channels it. ...(input.op === 'shell' && input.stdin !== undefined ? { stdin: input.stdin } : {}), } @@ -181,18 +188,27 @@ const channelPayload = async ({ } } +// --------------------------------------------------------------------------- +// Credential seam — the security faculty's end of the rpc op (Slice 4 wiring) +// --------------------------------------------------------------------------- + +/** + * The rpc op's token vendor. MINIMAL: the seam stub returns no token — the + * credential_request → credential_result round-trip through the security + * faculty lands in Slice 4 (the op itself never knows OAuth; the thread + * orchestrates the vending). + */ +const getAuthToken: GetAuthToken = async () => undefined + // --------------------------------------------------------------------------- // In-flight execution — enough state to stop it by correlation id // --------------------------------------------------------------------------- type StopReason = 'canceled' | 'timeout' | 'line_quota' | 'byte_quota' -type Execution = { - /** Process-group leader pid. `detached` makes the group id equal this pid. */ - pid: number - /** First stop signal wins, so a late cancel cannot relabel a timeout. */ - stopReason: StopReason | null -} +type Execution = + | { kind: 'process'; pid: number; stopReason: StopReason | null } + | { kind: 'rpc'; controller: AbortController; stopReason: StopReason | null } /** Executions currently running, keyed by correlation id. */ const active = new Map() @@ -223,11 +239,12 @@ const killGroup = ({ pid }: { pid: number }): void => { }, KILL_GRACE_MS) } -/** Record why an execution stopped and signal its group. First writer wins. */ +/** Stop one execution: signal a process group, or abort an rpc fetch. First writer wins. */ const stopExecution = ({ execution, reason }: { execution: Execution; reason: StopReason }): void => { if (execution.stopReason !== null) return execution.stopReason = reason - killGroup({ pid: execution.pid }) + if (execution.kind === 'process') killGroup({ pid: execution.pid }) + else execution.controller.abort() } // --------------------------------------------------------------------------- @@ -291,6 +308,59 @@ const pumpLines = async ({ /** Characters of stdout echoed into a JSON parse-failure message. */ const JSON_SNIPPET_CHARS = 200 +/** + * Run the `rpc` op — one generic remote JSON-RPC call, abortable by cancel + * or deadline through the fetch signal. The op is transport-shaped: it + * carries the envelope, nothing more (MCP semantics live in thread packs). + * + * @remarks + * Errors-as-data at both layers: the client never throws on transport or + * protocol failures, and a remote error payload surfaces as `code: 'error' + * + remoteCode` (the remote failure's own discriminant, for retry policy). + * A stop (cancel/timeout) wins over the outcome: the abort rejection is + * mapped to the stop's status, not mislabeled `error`. + */ +const runRpcOp = async ({ + id, + input, + options, +}: { + id: string + input: ShellRpcOpInput + options: ShellOptions +}): Promise => { + const started = performance.now() + const timeoutMs = options.timeoutMs ?? DEFAULT_TIMEOUT_MS + const controller = new AbortController() + const execution: Execution = { kind: 'rpc', controller, stopReason: null } + active.set(id, execution) + const deadline = setTimeout(() => stopExecution({ execution, reason: 'timeout' }), timeoutMs) + try { + const outcome = await sendRpc({ + url: input.url, + method: input.method, + ...(input.params === undefined ? {} : { params: input.params }), + getAuthToken, + signal: controller.signal, + }) + const durationMs = Math.round(performance.now() - started) + // First stop wins over the decoded outcome — the abort rejection is not + // mislabeled a remote error. + if (execution.stopReason === 'canceled') return { code: 'canceled', durationMs } + if (execution.stopReason === 'timeout') return { code: 'timeout', durationMs } + if (outcome.ok) return { output: outcome.result, durationMs } + return { + code: 'error', + durationMs, + message: outcome.error.message, + remoteCode: outcome.error.code, + } + } finally { + clearTimeout(deadline) + active.delete(id) + } +} + /** * Run one op and return its bounded result. * @@ -309,7 +379,8 @@ const runOp = async ({ id: string input: ShellCallInput options: ShellOptions -}): Promise => { +}): Promise => { + if (input.op === 'rpc') return runRpcOp({ id, input, options }) const started = performance.now() const format = options.format ?? 'paged' const offset = options.offset ?? DEFAULT_OFFSET @@ -357,7 +428,7 @@ const runOp = async ({ proc.stdin.write(input.op === 'run' ? input.script : SHELL_OP_WRAPPER) proc.stdin.end() - const execution: Execution = { pid: proc.pid, stopReason: null } + const execution: Execution = { kind: 'process', pid: proc.pid, stopReason: null } active.set(id, execution) const deadline = setTimeout(() => stopExecution({ execution, reason: 'timeout' }), timeoutMs) @@ -515,8 +586,8 @@ const postResult = ({ space, }: { id: string - payload?: ShellSuccess - error?: ShellError + payload?: ShellSuccess | RpcOpSuccess + error?: ShellError | RpcOpError space?: string }): void => { emit({ diff --git a/src/faculties/shell/rpc.client.ts b/src/faculties/shell/rpc.client.ts index f230cf7d3..5fd184ecb 100644 --- a/src/faculties/shell/rpc.client.ts +++ b/src/faculties/shell/rpc.client.ts @@ -45,6 +45,8 @@ export type SendInput = { getAuthToken?: GetAuthToken /** Injectable transport. Defaults to the platform `fetch`. */ fetch?: typeof fetch + /** Abort signal for the in-flight POST — cancellation rides the transport. */ + signal?: AbortSignal } /** Build the JSON-RPC 2.0 request envelope for one call. */ @@ -94,6 +96,7 @@ export const send = async ({ id = crypto.randomUUID(), getAuthToken, fetch: fetchImpl = fetch, + signal, }: SendInput): Promise => { const token = getAuthToken === undefined ? undefined : await getAuthToken().catch(() => undefined) const headers: Record = { 'content-type': 'application/json', accept: 'application/json' } @@ -104,6 +107,7 @@ export const send = async ({ method: 'POST', headers, body: JSON.stringify(envelope({ method, params, id })), + ...(signal === undefined ? {} : { signal }), }) } catch (err) { return { diff --git a/src/faculties/shell/tests/rpc-op.spec.ts b/src/faculties/shell/tests/rpc-op.spec.ts new file mode 100644 index 000000000..dedd0a597 --- /dev/null +++ b/src/faculties/shell/tests/rpc-op.spec.ts @@ -0,0 +1,155 @@ +import { afterEach, describe, expect, test } from 'bun:test' +import type { JsonObject } from '../../../behavioral/behavioral.types.ts' +import { FACULTY_MESSAGE_KINDS } from '../../faculties.constants.ts' +import { spawnFaculty } from '../../tests/faculty-harness.ts' + +/** + * RPC op specs — exercised through the REAL faculty process boundary + * speaking the behavioral event wire against a real local JSON-RPC HTTP + * server: `shell_request` (`op: 'rpc'`) in, one `shell_request_result` out. + * + * @packageDocumentation + */ + +type WireResult = { + id: string + ok: boolean + result?: { output?: JsonObject; durationMs?: number } + error?: { code?: string; message?: string; remoteCode?: number | string } + space?: string +} + +/** One JSON-RPC endpoint on a random localhost port; records requests. */ +const rpcServer = (handler?: (body: Record) => Response | Promise) => { + const requests: Array<{ url: string; body: Record; headers: Record }> = [] + const server = Bun.serve({ + port: 0, + fetch: async (request) => { + const body = (await request.json()) as Record + const headers: Record = {} + request.headers.forEach((value, key) => { + headers[key] = value + }) + requests.push({ url: request.url, body, headers }) + if (handler !== undefined) return handler(body) + return Response.json({ jsonrpc: '2.0', id: body.id, result: { echo: body } }) + }, + }) + return { + url: `http://localhost:${server.port}/mcp`, + requests, + close: () => server.stop(true), + } +} + +/** The faculty harness bound to the shell wire. */ +const spawnShellWorker = () => + spawnFaculty({ + file: 'shell/faculty.ts', + requestType: FACULTY_MESSAGE_KINDS.shell_request, + resultType: FACULTY_MESSAGE_KINDS.shell_request_result, + }) + +const servers: Array> = [] +const workers: Array> = [] +afterEach(() => { + for (const server of servers.splice(0)) server.close() + for (const worker of workers.splice(0)) worker.terminate() +}) + +const wire = (raw: Awaited['resultFor']>>): WireResult => + raw.detail as WireResult + +describe('shell rpc op', () => { + test('dispatches a generic remote JSON-RPC call and returns the output', async () => { + const server = rpcServer() + servers.push(server) + const worker = spawnShellWorker() + workers.push(worker) + worker.call({ id: 'rpc1', input: { op: 'rpc', url: server.url, method: 'tools/list', params: { page: 2 } } }) + const raw = await worker.resultFor('rpc1') + const result = wire(raw) + expect(result.ok).toBe(true) + const output = result.result?.output as { echo?: { method?: string } } + expect(output.echo?.method).toBe('tools/list') + // The generic envelope — no MCP framing. + const body = server.requests[0]?.body as Record + expect(body.method).toBe('tools/list') + expect(body.jsonrpc).toBe('2.0') + }) + + test('a remote JSON-RPC error payload is error data, not a throw', async () => { + const server = rpcServer((body) => + Response.json({ jsonrpc: '2.0', id: body.id, error: { code: -32601, message: 'nope' } }), + ) + servers.push(server) + const worker = spawnShellWorker() + workers.push(worker) + worker.call({ id: 'rpc2', input: { op: 'rpc', url: server.url, method: 'nope/x' } }) + const raw = await worker.resultFor('rpc2') + const result = wire(raw) + expect(result.ok).toBe(false) + expect(result.error?.code).toBe('error') + expect(result.error?.message).toBe('nope') + expect(result.error?.remoteCode).toBe(-32601) + }) + + test('an HTTP non-OK response is error data', async () => { + const server = rpcServer(() => new Response('down', { status: 503 })) + servers.push(server) + const worker = spawnShellWorker() + workers.push(worker) + worker.call({ id: 'rpc3', input: { op: 'rpc', url: server.url, method: 'ping' } }) + const raw = await worker.resultFor('rpc3') + const result = wire(raw) + expect(result.ok).toBe(false) + expect(result.error?.code).toBe('error') + expect(result.error?.remoteCode).toBe(503) + }) + + test('shell_cancel aborts the in-flight fetch — the result is canceled data', async () => { + let seen = 0 + const server = rpcServer(async () => { + seen += 1 + await Bun.sleep(5_000) + return Response.json({ jsonrpc: '2.0', id: 1, result: {} }) + }) + servers.push(server) + const worker = spawnShellWorker() + workers.push(worker) + worker.call({ id: 'rpc4', input: { op: 'rpc', url: server.url, method: 'slow/x' } }) + await Bun.sleep(100) + expect(seen).toBe(1) + worker.post({ type: FACULTY_MESSAGE_KINDS.shell_cancel, detail: { id: 'rpc4' } }) + const raw = await worker.resultFor('rpc4') + const result = wire(raw) + expect(result.ok).toBe(false) + expect(result.error?.code).toBe('canceled') + }) + + test('the deadline terminates a hung call — the result is timeout data', async () => { + const server = rpcServer(async () => { + await Bun.sleep(5_000) + return Response.json({ jsonrpc: '2.0', id: 1, result: {} }) + }) + servers.push(server) + const worker = spawnShellWorker() + workers.push(worker) + worker.call({ id: 'rpc5', input: { op: 'rpc', url: server.url, method: 'slow/x', timeoutMs: 200 } }) + const raw = await worker.resultFor('rpc5') + const result = wire(raw) + expect(result.ok).toBe(false) + expect(result.error?.code).toBe('timeout') + }) + + test('input that fails the rpc boundary is error data with the id intact', async () => { + const worker = spawnShellWorker() + workers.push(worker) + worker.call({ id: 'rpc6', input: { op: 'rpc', method: 'ping' } }) + const raw = await worker.resultFor('rpc6') + const result = wire(raw) + expect(result.ok).toBe(false) + expect(result.error?.code).toBe('error') + expect(result.error?.message).toContain('invalid input') + }) +}) diff --git a/src/faculties/shell/tests/rpc.client.spec.ts b/src/faculties/shell/tests/rpc.client.spec.ts index f8739bb01..d80ded633 100644 --- a/src/faculties/shell/tests/rpc.client.spec.ts +++ b/src/faculties/shell/tests/rpc.client.spec.ts @@ -121,4 +121,18 @@ describe('rpc.client', () => { expect(outcome.error.message).toContain('connection refused') } }) + + test('an aborted signal is error data — cancellation rides the transport', async () => { + const { impl } = fetchMock([new Error('The operation was aborted')]) + const controller = new AbortController() + const outcome = await send({ + url: 'https://rpc.example/mcp', + method: 'ping', + id: 'r5', + fetch: impl, + signal: controller.signal, + }) + expect(outcome.ok).toBe(false) + if (!outcome.ok) expect(outcome.error.code).toBe('network') + }) }) diff --git a/src/faculties/shell/types.ts b/src/faculties/shell/types.ts index 80bf42d23..3fe352458 100644 --- a/src/faculties/shell/types.ts +++ b/src/faculties/shell/types.ts @@ -25,6 +25,7 @@ */ import type { JSONSchemaType } from 'ajv' +import type { JsonObject } from '../../behavioral/behavioral.types.ts' /** Output representation for a completed execution. */ export type ShellFormat = 'paged' | 'json' | 'raw' @@ -98,8 +99,21 @@ export type ShellShellOpInput = { maxCharacters?: number } +/** The `'rpc'` op input — a generic remote JSON-RPC 2.0 call over HTTP POST. */ +export type ShellRpcOpInput = { + op: 'rpc' + /** The remote endpoint URL — one stateless POST per call. */ + url: string + /** The JSON-RPC method name (transport-shaped; protocol semantics live in thread packs). */ + method: string + /** The JSON-RPC params object, when the method takes one. */ + params?: JsonObject + /** Wall-clock deadline for the call. @default 30_000 */ + timeoutMs?: number +} + /** The `shell_request` event's `detail.input` — one discriminated shape per op. */ -export type ShellCallInput = ShellRunOpInput | ShellShellOpInput +export type ShellCallInput = ShellRunOpInput | ShellShellOpInput | ShellRpcOpInput // --------------------------------------------------------------------------- // Input boundary (shared knobs + per-op payloads; strict at every level) @@ -139,13 +153,26 @@ export const ShellShellOpInputSchema = { additionalProperties: false, } as unknown as JSONSchemaType +export const ShellRpcOpInputSchema: JSONSchemaType = { + type: 'object', + properties: { + op: { type: 'string', const: 'rpc' }, + url: { type: 'string', minLength: 1 }, + method: { type: 'string', minLength: 1 }, + params: { type: 'object', required: [], additionalProperties: true, nullable: true }, + timeoutMs: { type: 'integer', minimum: 1, nullable: true }, + }, + required: ['op', 'url', 'method'], + additionalProperties: false, +} + /** * The op-discriminated input boundary — `anyOf` branches (strict AJV rejects * union `type` arrays), `additionalProperties: false` at every level so the * op shapes cannot bleed into each other. */ export const ShellCallInputSchema = { - anyOf: [ShellRunOpInputSchema, ShellShellOpInputSchema], + anyOf: [ShellRunOpInputSchema, ShellShellOpInputSchema, ShellRpcOpInputSchema], } as unknown as JSONSchemaType // --------------------------------------------------------------------------- @@ -195,3 +222,37 @@ export type ShellError = { durationMs: number clamped?: string[] } + +// --------------------------------------------------------------------------- +// The `rpc` op's result envelope — transport outcomes, not process outcomes +// --------------------------------------------------------------------------- + +/** Terminal status of one rpc op — process-op statuses do not apply (no pid). */ +export type RpcStatus = 'canceled' | 'timeout' | 'error' + +/** The rpc success payload — the remote call's decoded `result` rides `output`. */ +export type RpcOpSuccess = { + output: JsonObject + /** Elapsed wall-clock time in milliseconds. */ + durationMs: number + /** Over-ceiling options that were clamped, as `' -> '`. */ + clamped?: string[] +} + +/** + * The rpc failure payload. `code` is the op's terminal status; a remote + * failure's own discriminant (HTTP status, JSON-RPC error code) rides + * `remoteCode` so retry policy can treat 5xx/timeouts differently from 4xx. + */ +export type RpcOpError = { + code: RpcStatus + /** Failure detail — the HTTP reason, JSON-RPC error message, or abort text. */ + message?: string + /** The remote failure's own code, when the call completed with an error response. */ + remoteCode?: number | string + durationMs: number + clamped?: string[] +} + +/** Every op runner's interior — one `code`-discriminated error branch over two payload families. */ +export type ShellOpResult = ShellSuccess | ShellError | RpcOpSuccess | RpcOpError From e12df6f2fef598ecb68587de0bcda8cbdde29a62 Mon Sep 17 00:00:00 2001 From: Edward Irby Date: Fri, 25 Sep 2026 13:39:15 -0700 Subject: [PATCH 11/55] =?UTF-8?q?feat(security):=20the=20credential-vendin?= =?UTF-8?q?g=20skeleton=20=E2=80=94=20a=20dedicated=20security=20faculty?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Slice 3 of the mcp-into-shell handoff: credentials/OAuth extracted from the mcp faculty into `src/faculties/security/` — cross-cutting and stateful, per the Direction/A ruling. - Wire kinds `credential_request` / `credential_result` / `credential_cancel` join the faculty event registry (schemas + validators in faculties.types.ts; the uniform WorkerResultDetail envelope; `credential_result` is the lane's only inbound kind — the pump re-enters results only). - The process vends fail-closed: broker env-data first (MCP_BROKER_URL + boot secret), then the keychain floor; absent -> error data naming the server. `credential_cancel` marks an in-flight vend (first writer wins). - keychain-oauth-provider MOVES from mcp/ to security/ and drops the SDK: plain types in security/types.ts (the issuer-stamp storage pattern, OAuthClientInformationContext, the discovery blob), a plain IssuerMismatchError and selectClientAuthMethod replace the SDK imports. tokens(ctx) now enforces the issuer binding its contract documented (a mismatched blob is treated as absent). - The keychain floor is `vendKeychainToken` (one home, shared with the mcp faculty's floor); broker env keys move to security/types.ts, re-exported by mcp/types.ts until deprecation. MINIMAL: expiry detection deferred. - Specs: provider suite (issuer binding, floor vend, auth selection) and the wire suite through the real process boundary (broker vends, absent-credential errors, cancel, space echo). 21 pass; tsc clean. --- src/faculties/faculties.constants.ts | 3 + src/faculties/faculties.types.ts | 61 ++++++ src/faculties/mcp/faculty.ts | 17 +- src/faculties/mcp/types.ts | 9 +- src/faculties/security/faculty.ts | 195 ++++++++++++++++++ .../keychain-oauth-provider.ts | 180 +++++++++++----- src/faculties/security/tests/faculty.spec.ts | 135 ++++++++++++ .../tests/keychain-oauth-provider.spec.ts | 80 ++++++- src/faculties/security/types.ts | 130 ++++++++++++ 9 files changed, 736 insertions(+), 74 deletions(-) create mode 100644 src/faculties/security/faculty.ts rename src/faculties/{mcp => security}/keychain-oauth-provider.ts (68%) create mode 100644 src/faculties/security/tests/faculty.spec.ts rename src/faculties/{mcp => security}/tests/keychain-oauth-provider.spec.ts (64%) create mode 100644 src/faculties/security/types.ts diff --git a/src/faculties/faculties.constants.ts b/src/faculties/faculties.constants.ts index 650b5f800..ca5f0e009 100644 --- a/src/faculties/faculties.constants.ts +++ b/src/faculties/faculties.constants.ts @@ -18,6 +18,9 @@ export const FACULTY_MESSAGE_KINDS = keyMirror( 'mcp_request', 'mcp_request_result', 'mcp_cancel', + 'credential_request', + 'credential_result', + 'credential_cancel', 'frontier_request', 'frontier_request_result', 'store_request', diff --git a/src/faculties/faculties.types.ts b/src/faculties/faculties.types.ts index e6944f79d..e7d0524c9 100644 --- a/src/faculties/faculties.types.ts +++ b/src/faculties/faculties.types.ts @@ -173,6 +173,27 @@ export type McpCancelEvent = { space?: string } +/** Security operations — credential vending for remote servers (broker first, keychain floor second). */ +export type SecurityRequestEvent = { + type: typeof FACULTY_MESSAGE_KINDS.credential_request + detail: { id: string; input: JsonObject } + space?: string +} + +export type SecurityRequestResultEvent = { + type: typeof FACULTY_MESSAGE_KINDS.credential_result + detail: WorkerResultDetail + space?: string +} + +// A vend is a quick broker/keychain read, but a down broker can hang — the +// async faculties keep their cancels (shell, response, mcp, security). +export type SecurityCancelEvent = { + type: typeof FACULTY_MESSAGE_KINDS.credential_cancel + detail: { id: string } + space?: string +} + /** Union of every event the router can move between ports. @public */ export type WorkerEvent = | SystemTwoRequestEvent @@ -187,6 +208,9 @@ export type WorkerEvent = | McpRequestEvent | McpRequestResultEvent | McpCancelEvent + | SecurityRequestEvent + | SecurityRequestResultEvent + | SecurityCancelEvent | FrontierRequestEvent | FrontierRequestResultEvent | StoreRequestEvent @@ -396,6 +420,40 @@ export const McpCancelEventSchema: JSONSchemaType = { additionalProperties: false, } +export const SecurityRequestEventSchema: JSONSchemaType = { + type: 'object', + properties: { + type: { type: 'string', const: FACULTY_MESSAGE_KINDS.credential_request }, + detail: { + type: 'object', + properties: { id: { type: 'string', minLength: 1 }, input: jsonObjectSchema }, + required: ['id', 'input'], + additionalProperties: false, + }, + space: { type: 'string', nullable: true }, + }, + required: ['type', 'detail'], + additionalProperties: false, +} + +export const SecurityRequestResultEventSchema = resultEventSchema(FACULTY_MESSAGE_KINDS.credential_result) + +export const SecurityCancelEventSchema: JSONSchemaType = { + type: 'object', + properties: { + type: { type: 'string', const: FACULTY_MESSAGE_KINDS.credential_cancel }, + detail: { + type: 'object', + properties: { id: { type: 'string', minLength: 1 } }, + required: ['id'], + additionalProperties: false, + }, + space: { type: 'string', nullable: true }, + }, + required: ['type', 'detail'], + additionalProperties: false, +} + export const FacultyErrorEventSchema: JSONSchemaType = { type: 'object', properties: { @@ -424,6 +482,9 @@ export const validateShellCancelEvent = ajv.compile(ShellCancelEventSchema) export const validateMcpRequestEvent = ajv.compile(McpRequestEventSchema) export const validateMcpRequestResultEvent = ajv.compile(McpRequestResultEventSchema) export const validateMcpCancelEvent = ajv.compile(McpCancelEventSchema) +export const validateSecurityRequestEvent = ajv.compile(SecurityRequestEventSchema) +export const validateSecurityRequestResultEvent = ajv.compile(SecurityRequestResultEventSchema) +export const validateSecurityCancelEvent = ajv.compile(SecurityCancelEventSchema) // No frontier cancel event: analyses are synchronous — nothing is in flight // to abort (the async faculties keep their cancels). export const FrontierRequestEventSchema: JSONSchemaType = { diff --git a/src/faculties/mcp/faculty.ts b/src/faculties/mcp/faculty.ts index 4fa5e9284..a70b0e3cc 100644 --- a/src/faculties/mcp/faculty.ts +++ b/src/faculties/mcp/faculty.ts @@ -56,7 +56,7 @@ import { validateMcpRequestEvent, } from '../faculties.types.ts' import { emit, envData, wireInbound } from '../process-lane.ts' -import { BunKeychain, tokensKey } from './keychain-oauth-provider.ts' +import { BunKeychain, vendKeychainToken } from '../security/keychain-oauth-provider.ts' import { MCP_BROKER_BOOT_SECRET_KEY, MCP_BROKER_URL_KEY, MCP_OP_INPUT_VALIDATORS } from './types.ts' // --------------------------------------------------------------------------- @@ -92,18 +92,9 @@ const brokerToken = async (): Promise => { } } -/** The keychain floor: read tokens persisted by prior BunKeychainOAuthProvider flows. */ -const keychainToken = async (serverUrl: string): Promise => { - try { - const raw = await keychain.get(tokensKey(serverUrl)) - if (raw === null) return undefined - const tokens = JSON.parse(raw) as { access_token?: string } - return typeof tokens.access_token === 'string' && tokens.access_token !== '' ? tokens.access_token : undefined - } catch { - // Corrupt blobs are absent tokens — fail-closed, never a throw. - return undefined - } -} +/** The keychain floor: read tokens persisted by prior grant flows (the security faculty's store). */ +const keychainToken = async (serverUrl: string): Promise => + vendKeychainToken({ serverUrl, keychain }) /** The per-server token: broker first, keychain floor second, absent last. */ const getToken = async (serverUrl: string): Promise => diff --git a/src/faculties/mcp/types.ts b/src/faculties/mcp/types.ts index 78d849cc9..3eff6f398 100644 --- a/src/faculties/mcp/types.ts +++ b/src/faculties/mcp/types.ts @@ -32,14 +32,11 @@ import { ajv, type JsonObject } from '../../behavioral/behavioral.types.ts' import type { McpOp } from '../faculties.types.ts' // --------------------------------------------------------------------------- -// Env-data — the worker's auth binding (the injection law one level down) +// Env-data — the vend's broker binding. One home: `security/types.ts` (the +// security faculty's); re-exported here until the mcp faculty's deprecation. // --------------------------------------------------------------------------- -/** Env-data key holding the taskbar broker's base URL (the Pattern-2 binding). */ -export const MCP_BROKER_URL_KEY = 'MCP_BROKER_URL' - -/** Env-data key holding the per-boot broker secret (never tool input). */ -export const MCP_BROKER_BOOT_SECRET_KEY = 'MCP_BROKER_BOOT_SECRET' +export { MCP_BROKER_BOOT_SECRET_KEY, MCP_BROKER_URL_KEY } from '../security/types.ts' // --------------------------------------------------------------------------- // Op inputs — one shape per op, no `mode` discriminator, no auth fields diff --git a/src/faculties/security/faculty.ts b/src/faculties/security/faculty.ts new file mode 100644 index 000000000..1e413ddd1 --- /dev/null +++ b/src/faculties/security/faculty.ts @@ -0,0 +1,195 @@ +/** + * Security faculty — the cross-cutting credential/policy faculty: vends one + * credential per `credential_request` event and returns a single terminal + * `credential_result` event. + * + * @remarks + * Spawned by URL (never imported) and speaks the behavioral event wire: + * `credential_request` / `credential_cancel` in, `credential_result` out, + * with any request `space` echoed on the result. `detail.input` is validated + * against the credential boundary (`security/types.ts`); per-call credentials + * never ride the wire — auth binds at this module's scope from env-data. + * + * The vend (fail-closed, errors-as-data — no throw crosses the wire): + * - broker env-data (`MCP_BROKER_URL` + `MCP_BROKER_BOOT_SECRET`, seeded by + * the spawning composition root) → the broker's `request_access_token` + * endpoint. MINIMAL: the broker slice has not landed; its request contract + * is thin (POST + boot-secret bearer) and fail-closed on any failure. Pin + * it when the broker exists. + * - No broker (or a down one) → the keychain floor: {@link vendKeychainToken} + * reads the issuer-stamped token blob the provider's grant flows persist — + * the reconnect-per-turn posture (credentials persist in the keyring, the + * connection does not). + * - Neither yields a token → the result is error data (`code: 'error'`, + * message naming the server) — the consumer decides what an absent + * credential means for its call. + * + * `credential_cancel` stops an in-flight vend: the stop is recorded (first + * writer wins) and the settling vend reports `code: 'canceled'` instead of + * its outcome. `credential_result` is the lane's only inbound kind — the + * pump re-enters results only; requests/cancels stay outbound-only. + * + * The supervisory policy threads (token-readiness blocks, secret masking) + * are a follow-up slice — this skeleton vends credentials, nothing more. + * + * @packageDocumentation + */ + +import type { ValidateFunction } from 'ajv' +import { ajv, type JsonObject } from '../../behavioral/behavioral.types.ts' +import { FACULTY_MESSAGE_KINDS } from '../faculties.constants.ts' +import { + type SecurityCancelEvent, + type SecurityRequestEvent, + validateSecurityCancelEvent, + validateSecurityRequestEvent, +} from '../faculties.types.ts' +import { emit, envData, wireInbound } from '../process-lane.ts' +import { BunKeychain, vendKeychainToken } from './keychain-oauth-provider.ts' +import { MCP_BROKER_BOOT_SECRET_KEY, MCP_BROKER_URL_KEY, validateCredentialRequestInput } from './types.ts' + +// --------------------------------------------------------------------------- +// Auth binding — module scope, from boundary-legal data only +// --------------------------------------------------------------------------- + +const brokerUrl = envData(MCP_BROKER_URL_KEY) as string | undefined +const brokerBootSecret = envData(MCP_BROKER_BOOT_SECRET_KEY) as string | undefined +const keychain = BunKeychain() + +/** Fetch an access token from the taskbar broker (env-data binding). Fail-closed. */ +const brokerToken = async (): Promise => { + if (typeof brokerUrl !== 'string' || typeof brokerBootSecret !== 'string') return undefined + try { + const response = await fetch(new URL('request_access_token', brokerUrl), { + method: 'POST', + headers: { authorization: `Bearer ${brokerBootSecret}` }, + }) + if (!response.ok) return undefined + const data = (await response.json()) as { token?: string } + return typeof data.token === 'string' && data.token !== '' ? data.token : undefined + } catch { + // A down broker must not break the floor — fall through to the keychain. + return undefined + } +} + +/** The per-server vend: broker first, keychain floor second, absent last. */ +const vendCredential = async (serverUrl: string): Promise => + (await brokerToken()) ?? (await vendKeychainToken({ serverUrl, keychain })) + +// --------------------------------------------------------------------------- +// In-flight vends — enough state to stop one by correlation id +// --------------------------------------------------------------------------- + +type Vend = { + /** First stop wins, so a late cancel cannot relabel a settled vend. */ + stopReason: 'canceled' | null +} + +/** Vends in flight, keyed by correlation id. */ +const active = new Map() + +/** Record the stop. First writer wins. */ +const stopVend = ({ vend }: { vend: Vend }): void => { + if (vend.stopReason !== null) return + vend.stopReason = 'canceled' +} + +// --------------------------------------------------------------------------- +// Result envelope +// --------------------------------------------------------------------------- + +/** Post the single terminal result event (the uniform modified-B envelope). */ +const postResult = ({ + id, + token, + error, + space, +}: { + id: string + token?: string + error?: { code: string; message?: string } + space?: string +}): void => { + emit({ + type: FACULTY_MESSAGE_KINDS.credential_result, + detail: (error === undefined + ? { id, ok: true, result: { token } } + : { id, ok: false, error: error as JsonObject }) as JsonObject & { id: string }, + ...(space === undefined ? {} : { space }), + }) +} + +// --------------------------------------------------------------------------- +// Worker message loop +// --------------------------------------------------------------------------- + +/** Route one inbound event. */ +const handleInbound = async (message: unknown): Promise => { + if (validateSecurityCancelEvent(message)) { + // Cast: the TS7/ajv compiler defect means the type guard does not narrow. + const cancel = message as SecurityCancelEvent + const vend = active.get(cancel.detail.id) + if (vend !== undefined) stopVend({ vend }) + return + } + // Events failing the shared schema have no correlation id to report to and + // are dropped — the router only forwards schema-valid events, so this is + // defense in depth at the process boundary. + if (!validateSecurityRequestEvent(message)) return + const event = message as SecurityRequestEvent + const { id, input } = event.detail + + // Input that fails the boundary is error data, not a throw: the id is + // valid, so the caller learns why nothing was vended. + const validate = validateCredentialRequestInput as unknown as ValidateFunction + if (!validate(input)) { + postResult({ + id, + space: event.space, + error: { code: 'error', message: `invalid input: ${ajv.errorsText(validate.errors)}` }, + }) + return + } + const { serverUrl } = input as { serverUrl: string } + + const vend: Vend = { stopReason: null } + active.set(id, vend) + try { + const token = await vendCredential(serverUrl) + // A stop wins over the outcome: the (possibly already-vended) credential + // is discarded and the caller learns the request was canceled. + if (vend.stopReason === 'canceled') { + postResult({ id, space: event.space, error: { code: 'canceled' } }) + return + } + if (token === undefined) { + postResult({ + id, + space: event.space, + error: { code: 'error', message: `no credential available for ${serverUrl}` }, + }) + return + } + postResult({ id, space: event.space, token }) + } catch (err) { + // The vend is fail-closed by construction; this is the last-resort guard + // so no worker-side throw ever escapes as a crash. + postResult({ + id, + space: event.space, + error: { code: 'error', message: err instanceof Error ? err.message : String(err) }, + }) + } finally { + active.delete(id) + } +} + +// The wire is the behavioral event vocabulary, validated with the shared +// schemas — the trust boundary for anything crossing into this process. +if (import.meta.main) { + // Standalone (spawned process) — wire the stdio line lane. An in-process + // import (the composition's frontier embed) wires nothing: the host's + // stdin is never touched. + wireInbound((message) => handleInbound(message)) +} diff --git a/src/faculties/mcp/keychain-oauth-provider.ts b/src/faculties/security/keychain-oauth-provider.ts similarity index 68% rename from src/faculties/mcp/keychain-oauth-provider.ts rename to src/faculties/security/keychain-oauth-provider.ts index 8438db42f..0c7cfaae4 100644 --- a/src/faculties/mcp/keychain-oauth-provider.ts +++ b/src/faculties/security/keychain-oauth-provider.ts @@ -1,48 +1,93 @@ /** - * `BunKeychainOAuthProvider` — a v2 {@link OAuthClientProvider} that persists - * OAuth tokens and client information to the OS keychain (via - * {@link BunKeychain}), implementing the agentskills.io / MCP v2 issuer-binding - * and RFC 9207 `iss` validation shape. + * `BunKeychainOAuthProvider` — the security faculty's credential store: an + * issuer-binding OAuth provider that persists tokens and client information + * to the OS keychain (via {@link BunKeychain}), SDK-free (plain types from + * `security/types.ts` — the `@modelcontextprotocol/client` dependency is + * retired). * * @remarks * One provider per server-url, reused across process restarts: the keychain * persists, the connection doesn't, but a reconnect reads tokens back via * {@link BunKeychainOAuthProvider.tokens | tokens()}. * - * Backs the non-interactive `client_credentials` and `refresh_token` grants. - * The v2 SDK's `auth()` orchestrator (invoked by the transport on 401) does - * RFC 9728 discovery and the token exchange via `prepareTokenRequest` + - * `addClientAuthentication` + `clientInformation`; this provider supplies the - * grant parameters and credentials and persists the issuer-stamped results. + * Backs the non-interactive `client_credentials` and `refresh_token` grants: + * this provider supplies the grant parameters and credentials + * ({@link BunKeychainOAuthProvider.prepareTokenRequest | prepareTokenRequest}, + * {@link BunKeychainOAuthProvider.addClientAuthentication | addClientAuthentication}, + * {@link BunKeychainOAuthProvider.clientInformation | clientInformation}) and + * persists the issuer-stamped results. The HTTP exchange itself is the grant + * orchestrator's (a later slice rides these hooks); MINIMAL: nothing here + * calls the token endpoint yet. * * Issuer-binding: `clientInformation(ctx)` and `tokens(ctx)` key persisted - * blobs by the SDK-stamped `issuer`; a blob whose `issuer` does not match the - * resolved authorization server is treated as absent (the SDK's - * `discardIfIssuerMismatch` enforces this at the `auth()` layer). When - * `ctx === undefined` (the resource-server `token()` read, pre-discovery), - * the most-recently-saved blob for the server is returned, per the v2 - * adapter contract. + * blobs by the stamped `issuer`; a blob whose `issuer` does not match the + * resolved authorization server is treated as absent. When `ctx === + * undefined` (the pre-discovery per-request read), the most-recently-saved + * blob for the server is returned. * * MINIMAL: `validateResourceURL` enforces origin (scheme+host+port) binding * between the MCP server URL and a requested `resource` (RFC 8707). Upgrade - * path: full RFC 8707 + RFC 9207 `iss` validation lives in the SDK's `auth()` - * (`validateAuthorizationResponseIssuer`); this hook covers the - * resource-binding leg that `auth()` delegates to the provider. + * path: full RFC 8707 + RFC 9207 `iss` validation rides the grant + * orchestrator when it lands; this hook covers the resource-binding leg. * * @packageDocumentation */ import type { - AddClientAuthentication, + ClientAuthMethod, + OAuthClientInformation, OAuthClientInformationContext, - OAuthClientInformationMixed, OAuthClientMetadata, OAuthDiscoveryState, - OAuthTokens, StoredOAuthClientInformation, StoredOAuthTokens, -} from '@modelcontextprotocol/client' -import { IssuerMismatchError, selectClientAuthMethod } from '@modelcontextprotocol/client' +} from './types.ts' + +// --------------------------------------------------------------------------- +// Plain client-auth selection (the SDK-free replacement for the SDK helper) +// --------------------------------------------------------------------------- + +/** + * Selects the client-auth method for a token request: basic, then post, when + * a client secret is available and supported; `none` otherwise (public + * clients, or an AS that only offers `none`). Documented SDK priority order, + * reimplemented against plain types. + */ +export const selectClientAuthMethod = ( + clientInformation: OAuthClientInformation, + supportedMethods: string[], +): ClientAuthMethod => { + if (clientInformation.client_secret !== undefined) { + if (supportedMethods.includes('client_secret_basic')) return 'client_secret_basic' + if (supportedMethods.includes('client_secret_post')) return 'client_secret_post' + } + return 'none' +} + +/** + * Thrown when an authorization-server issuer identifier fails validation — + * the mix-up-attack guard (RFC 8414 §3.3 metadata echo / RFC 9207 `iss`). + * Fatal for a grant flow, never retryable-by-credential-invalidation. + */ +export class IssuerMismatchError extends Error { + /** Which check failed — metadata echo or authorization-response `iss`. */ + readonly kind: 'metadata' | 'authorization_response' + /** The issuer the client expected (from validated metadata / discovery input). */ + readonly expected: string | undefined + /** The issuer value that was received. Attacker-controllable on the response path. */ + readonly received: string | undefined + + constructor(kind: 'metadata' | 'authorization_response', expected: string | undefined, received: string | undefined) { + // The values are JSON-encoded to neutralize log-injection. + super( + `issuer mismatch (${kind}): expected ${JSON.stringify(expected ?? null)}, received ${JSON.stringify(received ?? null)}`, + ) + this.name = 'IssuerMismatchError' + this.kind = kind + this.expected = expected + this.received = received + } +} // --------------------------------------------------------------------------- // Config @@ -71,7 +116,7 @@ export type KeychainOAuthProviderOptions = { clientAuthentication?: 'client_secret_basic' | 'client_secret_post' | 'none' /** * The authorization server's `issuer` these credentials are registered with. - * Stamped onto stored client information for SEP-2352 issuer-binding. May be + * Stamped onto stored client information for issuer-binding. May be * omitted when the issuer is only known after discovery. */ expectedIssuer?: string @@ -92,12 +137,43 @@ const hostOf = (url: string): string => { } // The keychain names within the service — exported so the keychain-floor -// reader (the mcp worker) derives the same keys as the writer (the provider) -// from one source. +// reader (the security faculty) derives the same keys as the writer (this +// provider) from one source. export const tokensKey = (serverUrl: string): string => `${hostOf(serverUrl)}:tokens` const clientInfoKey = (serverUrl: string): string => `${hostOf(serverUrl)}:clientinfo` const discoveryKey = (serverUrl: string): string => `${hostOf(serverUrl)}:discovery` +// --------------------------------------------------------------------------- +// The keychain floor — credential vending's last leg +// --------------------------------------------------------------------------- + +/** + * The keychain floor of credential vending: read the token slot for a server + * and vend its `access_token`, fail-closed. A missing slot, a corrupt blob, + * or an empty access token is an absent credential — never a throw. + * + * MINIMAL: token expiry is not detected (no `expires_in` bookkeeping, no + * refresh) — an expired token vends until the remote server rejects it. + * Upgrade path: expiry-aware reads + a refresh leg riding the provider's + * grant hooks when the grant orchestrator lands. + */ +export const vendKeychainToken = async ({ + serverUrl, + keychain, +}: { + serverUrl: string + keychain: Keychain +}): Promise => { + try { + const raw = await keychain.get(tokensKey(serverUrl)) + if (raw === null) return undefined + const tokens = JSON.parse(raw) as { access_token?: unknown } + return typeof tokens.access_token === 'string' && tokens.access_token !== '' ? tokens.access_token : undefined + } catch { + return undefined + } +} + // --------------------------------------------------------------------------- // Provider // --------------------------------------------------------------------------- @@ -134,7 +210,7 @@ export class BunKeychainOAuthProvider { this.#keychain = options.keychain ?? BunKeychain() } - // -- OAuthClientProvider: non-interactive basics ------------------------- + // -- grant-flow basics ---------------------------------------------------- get redirectUrl(): undefined { return undefined @@ -176,13 +252,13 @@ export class BunKeychainOAuthProvider { async clientInformation(ctx?: OAuthClientInformationContext): Promise { const cached = this.#cachedClientInfo ?? (await this.#loadClientInfo()) // Issuer-binding: a persisted blob bound to a different AS is treated as - // absent — the SDK will re-stamp from the resolved issuer (or fall back to - // the statically-configured credentials below). + // absent — the grant flow re-stamps from the resolved issuer (or falls + // back to the statically-configured credentials below). if (cached && this.#issuerMatches(cached.issuer, ctx?.issuer)) { return cached } // No usable persisted info — return the statically-configured credentials - // (unstamped); the SDK stamps + saves them on first auth. + // (unstamped); the grant flow stamps + saves them on first auth. if (this.#clientId) { return { client_id: this.#clientId, @@ -203,10 +279,13 @@ export class BunKeychainOAuthProvider { // -- tokens (issuer-keyed; most-recently-saved when ctx undefined) ------ - async tokens(_ctx?: OAuthClientInformationContext): Promise { - if (this.#cachedTokens) return this.#cachedTokens - this.#cachedTokens = await this.#loadTokens() - return this.#cachedTokens + async tokens(ctx?: OAuthClientInformationContext): Promise { + const tokens = this.#cachedTokens ?? (await this.#loadTokens()) + // Issuer-binding: a blob stamped with a different AS is not this AS's + // credential — absent, never vended to the wrong server. + if (tokens === undefined || !this.#issuerMatches(tokens.issuer, ctx?.issuer)) return undefined + this.#cachedTokens = tokens + return tokens } async saveTokens(tokens: StoredOAuthTokens, _ctx?: OAuthClientInformationContext): Promise { @@ -236,8 +315,8 @@ export class BunKeychainOAuthProvider { // -- token request ------------------------------------------------------ /** - * Builds the grant-specific token-request body. The v2 SDK's `fetchToken` - * calls this (plus {@link addClientAuthentication}) to perform the exchange. + * Builds the grant-specific token-request body — the grant orchestrator + * pairs this with {@link addClientAuthentication} for the exchange. */ prepareTokenRequest(scope?: string): URLSearchParams { const params = new URLSearchParams() @@ -257,10 +336,15 @@ export class BunKeychainOAuthProvider { /** * Adds client credentials to a token request per the configured method. - * Mirrors the SDK's `selectClientAuthMethod` default ordering when no - * method is configured. + * Mirrors {@link selectClientAuthMethod}'s default ordering when no method + * is configured. */ - addClientAuthentication: AddClientAuthentication = (headers, params, _url, _metadata) => { + addClientAuthentication = ( + headers: Headers, + params: URLSearchParams, + _url?: string | URL, + _metadata?: unknown, + ): void => { const method = this.#resolvedAuthMethod() switch (method) { case 'client_secret_basic': { @@ -336,12 +420,12 @@ export class BunKeychainOAuthProvider { #resolvedAuthMethod(): 'client_secret_basic' | 'client_secret_post' | 'none' { if (this.#clientAuthentication) return this.#clientAuthentication - // Default selection matches the SDK's selectClientAuthMethod priority. + // Default selection matches selectClientAuthMethod's priority. return selectClientAuthMethod( { client_id: this.#clientId, ...(this.#clientSecret ? { client_secret: this.#clientSecret } : {}), - } as OAuthClientInformationMixed, + }, ['client_secret_basic', 'client_secret_post', 'none'], ) } @@ -351,8 +435,9 @@ export class BunKeychainOAuthProvider { } #issuerMatches(stored: string | undefined, requested: string | undefined): boolean { - // ctx === undefined (resource-server read) → accept the most-recently-saved. + // ctx === undefined (pre-discovery read) → accept the most-recently-saved. if (requested === undefined) return true + // An unstamped blob is legacy state — bound to whatever AS asks. if (stored === undefined) return true return stored === requested } @@ -378,10 +463,8 @@ export class BunKeychainOAuthProvider { } } -export type { OAuthDiscoveryState, OAuthTokens, StoredOAuthClientInformation, StoredOAuthTokens } - /** - * OS-keychain abstraction for the MCP OAuth provider. + * OS-keychain abstraction for the security faculty's credential store. * * @remarks * `BunKeychain` wraps {@link Bun.secrets} (macOS Keychain / libsecret / @@ -392,7 +475,6 @@ export type { OAuthDiscoveryState, OAuthTokens, StoredOAuthClientInformation, St * * All values are JSON strings; the provider serializes `StoredOAuthTokens` / * `StoredOAuthClientInformation` / `OAuthDiscoveryState` blobs before storing. - * */ /** A name/value secret store keyed by `name` within a fixed `service`. */ @@ -405,7 +487,11 @@ export type Keychain = { delete(name: string): Promise } -/** The fixed keychain service label — unique to the behavioral MCP client. */ +/** + * The fixed keychain service label — historical: named for the mcp faculty + * the store originated in. Retained so previously-stored credentials remain + * readable across the move. + */ export const KEYCHAIN_SERVICE = 'behavioral.mcp' /** diff --git a/src/faculties/security/tests/faculty.spec.ts b/src/faculties/security/tests/faculty.spec.ts new file mode 100644 index 000000000..937c1f39a --- /dev/null +++ b/src/faculties/security/tests/faculty.spec.ts @@ -0,0 +1,135 @@ +import { afterEach, describe, expect, test } from 'bun:test' +import type { JsonObject } from '../../../behavioral/behavioral.types.ts' +import { FACULTY_MESSAGE_KINDS } from '../../faculties.constants.ts' +import { spawnFaculty } from '../../tests/faculty-harness.ts' + +/** + * Security faculty specs — exercised through the REAL faculty process + * boundary speaking the behavioral event wire: `credential_request` in, one + * `credential_result` out, `credential_cancel` for an in-flight vend. + * + * The credential vend is fail-closed by construction: broker env-data is + * absent unless the spec seeds it (a REAL loopback broker over `Bun.serve`, + * so the HTTP leg is genuine), and the keychain floor is empty in tests — + * a missing credential is error data, never a throw. + * + * @packageDocumentation + */ + +type WireResult = { + id: string + ok: boolean + result?: { token?: string } + error?: { code?: string; message?: string } + space?: string +} + +/** A loopback token broker — the env-data leg of the vend, over real HTTP. */ +const brokerServer = (handler?: (request: Request) => Response | Promise) => { + const requests: Array<{ url: string; auth?: string }> = [] + const server = Bun.serve({ + port: 0, + fetch: async (request) => { + requests.push({ url: request.url, auth: request.headers.get('authorization') ?? undefined }) + if (handler !== undefined) return handler(request) + return Response.json({ token: 'broker-tok' }) + }, + }) + return { + url: `http://localhost:${server.port}/`, + requests, + close: () => server.stop(true), + } +} + +/** The faculty harness bound to the security wire. */ +const spawnSecurityWorker = (env?: Record) => + spawnFaculty({ + file: 'security/faculty.ts', + requestType: FACULTY_MESSAGE_KINDS.credential_request, + resultType: FACULTY_MESSAGE_KINDS.credential_result, + env, + }) + +const SERVER_URL = 'https://mcp.example.com/mcp' + +const brokers: Array> = [] +const workers: Array> = [] +afterEach(() => { + for (const broker of brokers.splice(0)) broker.close() + for (const worker of workers.splice(0)) worker.terminate() +}) + +describe('security faculty — credential vending over the wire', () => { + test('a credential_request vends the broker token when broker env-data is bound', async () => { + const broker = brokerServer() + brokers.push(broker) + const worker = spawnSecurityWorker({ MCP_BROKER_URL: broker.url, MCP_BROKER_BOOT_SECRET: 'boot-secret' }) + workers.push(worker) + worker.call({ id: 'cred1', input: { serverUrl: SERVER_URL } } as JsonObject) + const raw = await worker.resultFor('cred1') + const result = raw.detail as unknown as WireResult + expect(result.ok).toBe(true) + expect(result.result?.token).toBe('broker-tok') + // The broker leg carries the boot secret as a bearer. + expect(broker.requests[0]?.auth).toBe('Bearer boot-secret') + }) + + test('no broker env-data and an empty keychain → the vend fails as error data', async () => { + const worker = spawnSecurityWorker() + workers.push(worker) + worker.call({ id: 'cred2', input: { serverUrl: SERVER_URL } } as JsonObject) + const raw = await worker.resultFor('cred2') + const result = raw.detail as unknown as WireResult + expect(result.ok).toBe(false) + expect(result.error?.code).toBe('error') + expect(result.error?.message).toContain(SERVER_URL) + }) + + test('a down broker falls through to the keychain floor — still error data when empty', async () => { + const broker = brokerServer(() => new Response('down', { status: 503 })) + brokers.push(broker) + const worker = spawnSecurityWorker({ MCP_BROKER_URL: broker.url, MCP_BROKER_BOOT_SECRET: 'boot-secret' }) + workers.push(worker) + worker.call({ id: 'cred3', input: { serverUrl: SERVER_URL } } as JsonObject) + const raw = await worker.resultFor('cred3') + const result = raw.detail as unknown as WireResult + expect(result.ok).toBe(false) + expect(result.error?.code).toBe('error') + }) + + test('input that fails the boundary is error data naming the failure', async () => { + const worker = spawnSecurityWorker() + workers.push(worker) + worker.call({ id: 'cred4', input: { serverUrl: '' } } as JsonObject) + const raw = await worker.resultFor('cred4') + const result = raw.detail as unknown as WireResult + expect(result.ok).toBe(false) + expect(result.error?.code).toBe('error') + expect(result.error?.message).toContain('invalid input') + }) + + test('credential_cancel stops an in-flight vend — the result is canceled data', async () => { + const broker = brokerServer(async () => { + await Bun.sleep(1_000) + return Response.json({ token: 'late-tok' }) + }) + brokers.push(broker) + const worker = spawnSecurityWorker({ MCP_BROKER_URL: broker.url, MCP_BROKER_BOOT_SECRET: 'boot-secret' }) + workers.push(worker) + worker.call({ id: 'cred5', input: { serverUrl: SERVER_URL } } as JsonObject) + worker.post({ type: FACULTY_MESSAGE_KINDS.credential_cancel, detail: { id: 'cred5' } } as never) + const raw = await worker.resultFor('cred5') + const result = raw.detail as unknown as WireResult + expect(result.ok).toBe(false) + expect(result.error?.code).toBe('canceled') + }) + + test('the request space is echoed on the result', async () => { + const worker = spawnSecurityWorker() + workers.push(worker) + worker.call({ id: 'cred6', input: { serverUrl: SERVER_URL } } as JsonObject, 'space-9') + const raw = await worker.resultFor('cred6') + expect(raw.space).toBe('space-9') + }) +}) diff --git a/src/faculties/mcp/tests/keychain-oauth-provider.spec.ts b/src/faculties/security/tests/keychain-oauth-provider.spec.ts similarity index 64% rename from src/faculties/mcp/tests/keychain-oauth-provider.spec.ts rename to src/faculties/security/tests/keychain-oauth-provider.spec.ts index ae1caf833..7ab0b9395 100644 --- a/src/faculties/mcp/tests/keychain-oauth-provider.spec.ts +++ b/src/faculties/security/tests/keychain-oauth-provider.spec.ts @@ -1,6 +1,12 @@ import { describe, expect, test } from 'bun:test' -import { IssuerMismatchError } from '@modelcontextprotocol/client' -import { BunKeychainOAuthProvider, InMemoryKeychain } from '../keychain-oauth-provider.ts' +import { + BunKeychainOAuthProvider, + InMemoryKeychain, + IssuerMismatchError, + selectClientAuthMethod, + tokensKey, + vendKeychainToken, +} from '../keychain-oauth-provider.ts' const SERVER_URL = 'https://mcp.example.com/mcp' const baseOptions = (keychain: ReturnType) => ({ @@ -16,7 +22,7 @@ describe('BunKeychainOAuthProvider — token round-trip across a reconnect', () test('saveTokens persists to the keychain; a new provider instance reads them back', async () => { const keychain = InMemoryKeychain() - // First process: the SDK exchanges and saves issuer-stamped tokens. + // First process: the grant exchange saves issuer-stamped tokens. const first = new BunKeychainOAuthProvider(baseOptions(keychain)) await first.saveTokens( { @@ -46,7 +52,7 @@ describe('BunKeychainOAuthProvider — token round-trip across a reconnect', () { access_token: 'atk-2', token_type: 'Bearer', issuer: 'https://as.example.com' }, { issuer: 'https://as.example.com' }, ) - // No ctx — the v2 adapter calls tokens() pre-discovery. + // No ctx — the per-request bearer read happens pre-discovery. const tokens = await provider.tokens() expect(tokens?.access_token).toBe('atk-2') }) @@ -58,7 +64,7 @@ describe('BunKeychainOAuthProvider — token round-trip across a reconnect', () grantType: 'refresh_token', initialRefreshToken: 'initial-rt', }) - // SDK refreshes and saves a rotated refresh token. + // The grant exchange refreshes and saves a rotated refresh token. await first.saveTokens( { access_token: 'atk-3', token_type: 'Bearer', refresh_token: 'rotated-rt', issuer: 'https://as.example.com' }, { issuer: 'https://as.example.com' }, @@ -80,11 +86,35 @@ describe('BunKeychainOAuthProvider — token round-trip across a reconnect', () }) describe('BunKeychainOAuthProvider — issuer binding', () => { + test('tokens treats a blob stamped with a different issuer as absent', async () => { + const keychain = InMemoryKeychain() + const provider = new BunKeychainOAuthProvider(baseOptions(keychain)) + await provider.saveTokens( + { access_token: 'atk-a', token_type: 'Bearer', issuer: 'https://as-a.example.com' }, + { issuer: 'https://as-a.example.com' }, + ) + // The resolved authorization server is B — the A-stamped blob is not + // B's credential and must not be vended (issuer-binding, SEP-2352). + expect(await provider.tokens({ issuer: 'https://as-b.example.com' })).toBeUndefined() + // Same issuer → returned. + expect((await provider.tokens({ issuer: 'https://as-a.example.com' }))?.access_token).toBe('atk-a') + }) + + test('tokens treats an unstamped blob as bound to whatever AS asks', async () => { + const keychain = InMemoryKeychain() + const provider = new BunKeychainOAuthProvider(baseOptions(keychain)) + await provider.saveTokens( + { access_token: 'atk-legacy', token_type: 'Bearer' }, + { issuer: 'https://as.example.com' }, + ) + expect((await provider.tokens({ issuer: 'https://other.example.com' }))?.access_token).toBe('atk-legacy') + }) + test('clientInformation returns issuer-bound persisted info only when the issuer matches', async () => { const keychain = InMemoryKeychain() const provider = new BunKeychainOAuthProvider(baseOptions(keychain)) - // SDK stamps issuer A and saves client info. + // The grant flow stamps issuer A and saves client info. await provider.saveClientInformation( { client_id: 'client-1', client_secret: 'secret-1', issuer: 'https://as-a.example.com' }, { issuer: 'https://as-a.example.com' }, @@ -95,8 +125,7 @@ describe('BunKeychainOAuthProvider — issuer binding', () => { expect(bound?.issuer).toBe('https://as-a.example.com') // Different issuer → persisted info is NOT returned; falls back to the - // statically-configured (unstamped) credentials. This is the issuer-binding - // that the old provider lacked. + // statically-configured (unstamped) credentials. const mismatched = await provider.clientInformation({ issuer: 'https://as-b.example.com' }) expect(mismatched?.issuer).toBeUndefined() expect(mismatched?.client_id).toBe('client-1') @@ -121,6 +150,41 @@ describe('BunKeychainOAuthProvider — issuer binding', () => { }) }) +describe('vendKeychainToken — the keychain floor of credential vending', () => { + test('vends the access_token from a stored issuer-stamped blob', async () => { + const keychain = InMemoryKeychain() + await keychain.set( + tokensKey(SERVER_URL), + JSON.stringify({ access_token: 'atk-9', token_type: 'Bearer', issuer: 'https://as.example.com' }), + ) + expect(await vendKeychainToken({ serverUrl: SERVER_URL, keychain })).toBe('atk-9') + }) + + test('treats a missing slot, a corrupt blob, and an empty access_token as absent', async () => { + const keychain = InMemoryKeychain() + expect(await vendKeychainToken({ serverUrl: SERVER_URL, keychain })).toBeUndefined() + await keychain.set(tokensKey(SERVER_URL), 'not-json') + expect(await vendKeychainToken({ serverUrl: SERVER_URL, keychain })).toBeUndefined() + await keychain.set(tokensKey(SERVER_URL), JSON.stringify({ access_token: '', token_type: 'Bearer' })) + expect(await vendKeychainToken({ serverUrl: SERVER_URL, keychain })).toBeUndefined() + }) +}) + +describe('selectClientAuthMethod — the plain client-auth selection', () => { + test('prefers basic, then post, when a client secret is available and supported', () => { + const info = { client_id: 'c', client_secret: 's' } + expect(selectClientAuthMethod(info, ['none', 'client_secret_post', 'client_secret_basic'])).toBe( + 'client_secret_basic', + ) + expect(selectClientAuthMethod(info, ['none', 'client_secret_post'])).toBe('client_secret_post') + }) + + test('falls back to none for a public client or an unsupported AS', () => { + expect(selectClientAuthMethod({ client_id: 'c' }, ['client_secret_basic', 'none'])).toBe('none') + expect(selectClientAuthMethod({ client_id: 'c', client_secret: 's' }, ['none'])).toBe('none') + }) +}) + describe('BunKeychainOAuthProvider — invalidateCredentials clears by scope', () => { test("invalidateCredentials('tokens') drops the token slot", async () => { const keychain = InMemoryKeychain() diff --git a/src/faculties/security/types.ts b/src/faculties/security/types.ts new file mode 100644 index 000000000..97af3ff23 --- /dev/null +++ b/src/faculties/security/types.ts @@ -0,0 +1,130 @@ +/** + * Types shared by the security faculty process (`security/faculty.ts`) and + * its event-wire consumers. + * + * @remarks + * The plain OAuth type set — the SDK-free equivalents of the shapes the + * former `@modelcontextprotocol/client` dependency defined, kept to what the + * security faculty's credential path touches (the issuer-stamping storage + * pattern, the client-auth method selection inputs, and the persisted + * discovery blob). Types are structural, not schema-derived: these values + * live inside JSON keychain blobs and JS objects, never on the wire raw. + * + * The wire itself is the behavioral event vocabulary (`credential_request` / + * `credential_cancel` in, one `credential_result` out) defined in + * `src/faculties/faculties.types.ts` — only the `detail.input` payload shape + * and the env-data keys live here. + * + * Also the one home for the broker env-data keys (the Pattern-2 binding the + * vend checks before the keychain floor); the mcp faculty re-exports them + * until its deprecation. + * + * @packageDocumentation + */ + +import type { JSONSchemaType } from 'ajv' +import { ajv, type JsonObject } from '../../behavioral/behavioral.types.ts' + +// --------------------------------------------------------------------------- +// Env-data — the vend's broker binding (the injection law one level down) +// --------------------------------------------------------------------------- + +/** Env-data key holding the taskbar broker's base URL (the Pattern-2 binding). */ +export const MCP_BROKER_URL_KEY = 'MCP_BROKER_URL' + +/** Env-data key holding the per-boot broker secret (never tool input). */ +export const MCP_BROKER_BOOT_SECRET_KEY = 'MCP_BROKER_BOOT_SECRET' + +// --------------------------------------------------------------------------- +// Plain OAuth types — the SDK-free storage/flow contract +// --------------------------------------------------------------------------- + +/** RFC 6749 §5.1 token response (the fields the grant flows consume). */ +export type OAuthTokens = { + access_token: string + id_token?: string + token_type: string + /** Seconds until expiry, as sent by the AS (coerced-number semantics). */ + expires_in?: number + scope?: string + refresh_token?: string +} + +/** + * {@linkcode OAuthTokens} as persisted — with the SDK-stamped + * authorization-server `issuer` so stored tokens are bound to the AS that + * issued them. The `issuer` is NOT part of the RFC 6749 wire response; the + * grant flow stamps it before `saveTokens` (SEP-2352 issuer-binding). + */ +export type StoredOAuthTokens = OAuthTokens & { issuer?: string } + +/** RFC 6749 §2.2 client identity — client identifiers are unique per AS. */ +export type OAuthClientInformation = { + client_id: string + client_secret?: string + client_id_issued_at?: number + client_secret_expires_at?: number +} + +/** + * {@linkcode OAuthClientInformation} as persisted — the same issuer-stamp + * pattern as {@linkcode StoredOAuthTokens}. + * + * MINIMAL: RFC 7591 DCR responses carrying the full registration metadata + * (redirect_uris, grant_types, …) survive storage verbatim as JSON blobs but + * are typed narrow here — no in-repo consumer reads those fields yet. + * Upgrade path: a full `OAuthClientInformationFull` variant when one does. + */ +export type StoredOAuthClientInformation = OAuthClientInformation & { issuer?: string } + +/** The client metadata a grant flow presents to the AS (RFC 7591 subset). */ +export type OAuthClientMetadata = { + redirect_uris: string[] + token_endpoint_auth_method?: string + grant_types?: string[] + response_types?: string[] + application_type?: string + client_name?: string + scope?: string +} + +/** + * Context passed to the issuer-keyed credential reads/writes — carries the + * resolved authorization-server `issuer` (from its validated metadata) as + * the binding key. Omitted on the pre-discovery per-request read, which + * returns the most-recently-saved blob. + */ +export type OAuthClientInformationContext = { issuer: string } + +/** A persisted authorization-server selection — blob-read, never re-derived. */ +export type OAuthDiscoveryState = { + authorizationServerUrl: string + resourceMetadataUrl?: string + resourceMetadata?: JsonObject + authorizationServerMetadata?: JsonObject +} + +/** Client-auth methods a token request can ride. */ +export type ClientAuthMethod = 'client_secret_basic' | 'client_secret_post' | 'none' + +// --------------------------------------------------------------------------- +// The credential op input boundary — the trust boundary for anything +// crossing into the security faculty process; strict. +// --------------------------------------------------------------------------- + +/** The `credential_request` event's `detail.input` — vend a token for a server. */ +export type CredentialRequestInput = { + /** The remote server URL the credential is for — keys the keychain slots. */ + serverUrl: string +} + +export const CredentialRequestInputSchema: JSONSchemaType = { + type: 'object', + properties: { + serverUrl: { type: 'string', minLength: 1, description: 'the remote server URL the credential is for' }, + }, + required: ['serverUrl'], + additionalProperties: false, +} + +export const validateCredentialRequestInput = ajv.compile(CredentialRequestInputSchema) From 5042228ba250cb425c8c90bc7fe1f50cde93c0e1 Mon Sep 17 00:00:00 2001 From: Edward Irby Date: Fri, 25 Sep 2026 13:49:08 -0700 Subject: [PATCH 12/55] =?UTF-8?q?feat(security):=20host-supplied=20ctx=20o?= =?UTF-8?q?n=20credential=5Frequest=20=E2=80=94=20the=20issuer-binding=20l?= =?UTF-8?q?ane?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Extends the Slice 3 skeleton with the out-of-band binding lane, modeled on the you.com MCP server's host-supplied override pattern: host-only parameters ride `detail.ctx` beside `input` (the model-facing arguments), never inside them — the thread pack that discovers the authorization server stamps the resolved `issuer`; the op never does. - `credential_request` events gain optional `detail.ctx` (`{ issuer?: string }`) — loose at the shared wire gate (faculties.types.ts), strict at the security faculty's boundary (`SecurityRequestContextSchema`); a ctx that fails its shape is error data like any invalid input. - The keychain floor (`vendKeychainToken`) takes the ctx issuer: a blob stamped for a different AS is treated as absent, never vended to the wrong server (the provider's issuer-match rule, now shared via `issuerMatches`). Unstamped legacy blobs and ctx-less reads keep the most-recently-saved contract. - Resolves the Slice 3 open question: the floor's issuer binding now has a comparison base (the caller's resolved AS issuer) instead of none. Specs: ctx-carrying requests validate and vend over the wire; a ctx failing its boundary is error data; the floor binds on ctx.issuer (mismatch -> absent, match -> vended, unstamped -> legacy-accepted). tsc clean, 114 targeted specs pass. --- src/faculties/faculties.types.ts | 11 +++++- src/faculties/security/faculty.ts | 31 +++++++++++++--- .../security/keychain-oauth-provider.ts | 37 +++++++++++++++---- src/faculties/security/tests/faculty.spec.ts | 29 +++++++++++++++ .../tests/keychain-oauth-provider.spec.ts | 18 +++++++++ src/faculties/security/types.ts | 27 ++++++++++++++ 6 files changed, 138 insertions(+), 15 deletions(-) diff --git a/src/faculties/faculties.types.ts b/src/faculties/faculties.types.ts index e7d0524c9..8dc40e916 100644 --- a/src/faculties/faculties.types.ts +++ b/src/faculties/faculties.types.ts @@ -176,7 +176,8 @@ export type McpCancelEvent = { /** Security operations — credential vending for remote servers (broker first, keychain floor second). */ export type SecurityRequestEvent = { type: typeof FACULTY_MESSAGE_KINDS.credential_request - detail: { id: string; input: JsonObject } + /** `ctx` is the optional host-supplied binding (e.g. the resolved AS issuer) — out-of-band, never a model-facing input field. */ + detail: { id: string; ctx?: JsonObject; input: JsonObject } space?: string } @@ -426,7 +427,13 @@ export const SecurityRequestEventSchema: JSONSchemaType = type: { type: 'string', const: FACULTY_MESSAGE_KINDS.credential_request }, detail: { type: 'object', - properties: { id: { type: 'string', minLength: 1 }, input: jsonObjectSchema }, + properties: { + id: { type: 'string', minLength: 1 }, + // The host-supplied binding lane — its strict shape is the security + // faculty's boundary (SecurityRequestContextSchema), not the wire's. + ctx: { type: 'object', required: [], additionalProperties: true, nullable: true }, + input: jsonObjectSchema, + }, required: ['id', 'input'], additionalProperties: false, }, diff --git a/src/faculties/security/faculty.ts b/src/faculties/security/faculty.ts index 1e413ddd1..cfcb16866 100644 --- a/src/faculties/security/faculty.ts +++ b/src/faculties/security/faculty.ts @@ -46,7 +46,12 @@ import { } from '../faculties.types.ts' import { emit, envData, wireInbound } from '../process-lane.ts' import { BunKeychain, vendKeychainToken } from './keychain-oauth-provider.ts' -import { MCP_BROKER_BOOT_SECRET_KEY, MCP_BROKER_URL_KEY, validateCredentialRequestInput } from './types.ts' +import { + MCP_BROKER_BOOT_SECRET_KEY, + MCP_BROKER_URL_KEY, + validateCredentialRequestInput, + validateSecurityRequestContext, +} from './types.ts' // --------------------------------------------------------------------------- // Auth binding — module scope, from boundary-legal data only @@ -74,8 +79,9 @@ const brokerToken = async (): Promise => { } /** The per-server vend: broker first, keychain floor second, absent last. */ -const vendCredential = async (serverUrl: string): Promise => - (await brokerToken()) ?? (await vendKeychainToken({ serverUrl, keychain })) +const vendCredential = async (serverUrl: string, issuer?: string): Promise => + (await brokerToken()) ?? + (await vendKeychainToken({ serverUrl, keychain, ...(issuer === undefined ? {} : { issuer }) })) // --------------------------------------------------------------------------- // In-flight vends — enough state to stop one by correlation id @@ -138,7 +144,7 @@ const handleInbound = async (message: unknown): Promise => { // defense in depth at the process boundary. if (!validateSecurityRequestEvent(message)) return const event = message as SecurityRequestEvent - const { id, input } = event.detail + const { id, input, ctx } = event.detail // Input that fails the boundary is error data, not a throw: the id is // valid, so the caller learns why nothing was vended. @@ -151,12 +157,27 @@ const handleInbound = async (message: unknown): Promise => { }) return } + // The host-supplied binding lane — its own strict boundary, same + // error-data rule (the strict shape is this faculty's, not the wire's). + let issuer: string | undefined + if (ctx !== undefined) { + const validateContext = validateSecurityRequestContext as unknown as ValidateFunction + if (!validateContext(ctx)) { + postResult({ + id, + space: event.space, + error: { code: 'error', message: `invalid input: ${ajv.errorsText(validateContext.errors)}` }, + }) + return + } + issuer = (ctx as { issuer?: string }).issuer + } const { serverUrl } = input as { serverUrl: string } const vend: Vend = { stopReason: null } active.set(id, vend) try { - const token = await vendCredential(serverUrl) + const token = await vendCredential(serverUrl, issuer) // A stop wins over the outcome: the (possibly already-vended) credential // is discarded and the caller learns the request was canceled. if (vend.stopReason === 'canceled') { diff --git a/src/faculties/security/keychain-oauth-provider.ts b/src/faculties/security/keychain-oauth-provider.ts index 0c7cfaae4..69d806784 100644 --- a/src/faculties/security/keychain-oauth-provider.ts +++ b/src/faculties/security/keychain-oauth-provider.ts @@ -143,6 +143,19 @@ export const tokensKey = (serverUrl: string): string => `${hostOf(serverUrl)}:to const clientInfoKey = (serverUrl: string): string => `${hostOf(serverUrl)}:clientinfo` const discoveryKey = (serverUrl: string): string => `${hostOf(serverUrl)}:discovery` +/** + * The issuer-binding rule, shared by the provider reads and the keychain + * floor: a blob stamped for a different authorization server is not this + * AS's credential — absent, never vended to the wrong server. An unstamped + * (legacy) blob binds to whatever AS asks; no ctx issuer (the pre-discovery + * read) accepts the most-recently-saved blob. + */ +export const issuerMatches = (stored: string | undefined, requested: string | undefined): boolean => { + if (requested === undefined) return true + if (stored === undefined) return true + return stored === requested +} + // --------------------------------------------------------------------------- // The keychain floor — credential vending's last leg // --------------------------------------------------------------------------- @@ -150,7 +163,9 @@ const discoveryKey = (serverUrl: string): string => `${hostOf(serverUrl)}:discov /** * The keychain floor of credential vending: read the token slot for a server * and vend its `access_token`, fail-closed. A missing slot, a corrupt blob, - * or an empty access token is an absent credential — never a throw. + * an empty access token, or a blob bound to a different authorization server + * (`issuer`, when the caller supplies one from its resolved discovery) is an + * absent credential — never a throw. * * MINIMAL: token expiry is not detected (no `expires_in` bookkeeping, no * refresh) — an expired token vends until the remote server rejects it. @@ -160,15 +175,25 @@ const discoveryKey = (serverUrl: string): string => `${hostOf(serverUrl)}:discov export const vendKeychainToken = async ({ serverUrl, keychain, + issuer, }: { serverUrl: string keychain: Keychain + /** The caller's resolved AS issuer — binds the read (undefined = most-recently-saved). */ + issuer?: string }): Promise => { try { const raw = await keychain.get(tokensKey(serverUrl)) if (raw === null) return undefined - const tokens = JSON.parse(raw) as { access_token?: unknown } - return typeof tokens.access_token === 'string' && tokens.access_token !== '' ? tokens.access_token : undefined + const tokens = JSON.parse(raw) as { access_token?: unknown; issuer?: unknown } + const accessToken = + typeof tokens.access_token === 'string' && tokens.access_token !== '' ? tokens.access_token : undefined + if (accessToken === undefined) return undefined + const stamped = typeof tokens.issuer === 'string' && tokens.issuer !== '' ? tokens.issuer : undefined + // Issuer-binding: a blob bound to another AS must not be vended — the + // caller's resolved issuer is the only accepted binding. + if (!issuerMatches(stamped, issuer)) return undefined + return accessToken } catch { return undefined } @@ -435,11 +460,7 @@ export class BunKeychainOAuthProvider { } #issuerMatches(stored: string | undefined, requested: string | undefined): boolean { - // ctx === undefined (pre-discovery read) → accept the most-recently-saved. - if (requested === undefined) return true - // An unstamped blob is legacy state — bound to whatever AS asks. - if (stored === undefined) return true - return stored === requested + return issuerMatches(stored, requested) } async #loadTokens(): Promise { diff --git a/src/faculties/security/tests/faculty.spec.ts b/src/faculties/security/tests/faculty.spec.ts index 937c1f39a..d67140984 100644 --- a/src/faculties/security/tests/faculty.spec.ts +++ b/src/faculties/security/tests/faculty.spec.ts @@ -125,6 +125,35 @@ describe('security faculty — credential vending over the wire', () => { expect(result.error?.code).toBe('canceled') }) + test('a ctx-carrying request is valid on the wire and vends (the binding rides out-of-band)', async () => { + const broker = brokerServer() + brokers.push(broker) + const worker = spawnSecurityWorker({ MCP_BROKER_URL: broker.url, MCP_BROKER_BOOT_SECRET: 'boot-secret' }) + workers.push(worker) + // ctx rides beside input (the host-supplied override lane) — never a + // model-facing input field. + worker.call({ + id: 'cred7', + input: { serverUrl: SERVER_URL }, + ctx: { issuer: 'https://as.example.com' }, + } as JsonObject) + const raw = await worker.resultFor('cred7') + const result = raw.detail as unknown as WireResult + expect(result.ok).toBe(true) + expect(result.result?.token).toBe('broker-tok') + }) + + test('a ctx that fails its boundary is error data naming the failure', async () => { + const worker = spawnSecurityWorker() + workers.push(worker) + worker.call({ id: 'cred8', input: { serverUrl: SERVER_URL }, ctx: { issuer: 42 } } as JsonObject) + const raw = await worker.resultFor('cred8') + const result = raw.detail as unknown as WireResult + expect(result.ok).toBe(false) + expect(result.error?.code).toBe('error') + expect(result.error?.message).toContain('invalid input') + }) + test('the request space is echoed on the result', async () => { const worker = spawnSecurityWorker() workers.push(worker) diff --git a/src/faculties/security/tests/keychain-oauth-provider.spec.ts b/src/faculties/security/tests/keychain-oauth-provider.spec.ts index 7ab0b9395..5a22947ef 100644 --- a/src/faculties/security/tests/keychain-oauth-provider.spec.ts +++ b/src/faculties/security/tests/keychain-oauth-provider.spec.ts @@ -160,6 +160,24 @@ describe('vendKeychainToken — the keychain floor of credential vending', () => expect(await vendKeychainToken({ serverUrl: SERVER_URL, keychain })).toBe('atk-9') }) + test('a ctx issuer binds the floor: a blob stamped for another AS is treated as absent', async () => { + const keychain = InMemoryKeychain() + await keychain.set( + tokensKey(SERVER_URL), + JSON.stringify({ access_token: 'atk-9', token_type: 'Bearer', issuer: 'https://as-a.example.com' }), + ) + // The resolved AS is B — the A-stamped blob is not B's credential. + expect( + await vendKeychainToken({ serverUrl: SERVER_URL, keychain, issuer: 'https://as-b.example.com' }), + ).toBeUndefined() + // Matching issuer → vended. + expect(await vendKeychainToken({ serverUrl: SERVER_URL, keychain, issuer: 'https://as-a.example.com' })).toBe( + 'atk-9', + ) + // No ctx issuer (pre-discovery read) → the most-recently-saved blob vends. + expect(await vendKeychainToken({ serverUrl: SERVER_URL, keychain })).toBe('atk-9') + }) + test('treats a missing slot, a corrupt blob, and an empty access_token as absent', async () => { const keychain = InMemoryKeychain() expect(await vendKeychainToken({ serverUrl: SERVER_URL, keychain })).toBeUndefined() diff --git a/src/faculties/security/types.ts b/src/faculties/security/types.ts index 97af3ff23..929fb4364 100644 --- a/src/faculties/security/types.ts +++ b/src/faculties/security/types.ts @@ -128,3 +128,30 @@ export const CredentialRequestInputSchema: JSONSchemaType = { + type: 'object', + properties: { + issuer: { + type: 'string', + minLength: 1, + nullable: true, + description: 'the resolved authorization-server issuer binding', + }, + }, + required: [], + additionalProperties: false, +} + +export const validateSecurityRequestContext = ajv.compile(SecurityRequestContextSchema) From 4158e4af5462b78646d0009c79ecab32d61384a7 Mon Sep 17 00:00:00 2001 From: Edward Irby Date: Fri, 25 Sep 2026 14:25:15 -0700 Subject: [PATCH 13/55] =?UTF-8?q?feat(shell):=20the=20credential=20seam=20?= =?UTF-8?q?=E2=80=94=20vend-and-replay=20threads=20wire=20rpc=20to=20secur?= =?UTF-8?q?ity?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Slice 4 of the mcp-into-shell handoff: the rpc op's credential round-trip is thread-orchestrated — the op never knows OAuth and never talks to the security faculty directly. - `ShellRpcOpInput` gains the declarative auth seam: `auth: true` without a token short-circuits as typed `credential_required` (no unauthenticated remote call, first-writer loop bound) echoing the originating request — the replay capture payload, mirroring the mcp spine's authorization_required. The replayed call carries the vended bearer in `authToken` (thread-injected, never model input; the redaction floor now scrubs `authToken` values from traces). - `shell/rpc-auth.threads.ts` — the vend-and-replay spine: the requestor transforms a first-attempt credential_required into `credential_request { serverUrl }` with the original call riding `ctx.echo` (the you.com MCP host-supplied out-of-band lane); the replayer joins the vended `credential_result` and replays the shell_request with the bearer merged in. The security faculty echoes `ctx.echo` verbatim on the result — the join rides one round-trip, no store capture needed. - bProgram wires the security faculty (default-on, `security` override for env-data hosts), routes credential_request/cancel, and mounts the seam pack when shell+security are both on; `Faculty` gains 'security'. - MINIMAL: a failed vend stays inert (the caller already holds the typed credential_required error; the failure is visible in traces). No retry policy — the generic retry thread pattern covers it later. - Specs: op short-circuit + bearer-injection through the real process boundary; the spine's requestor/replayer/loop-bound/absent-credential gates against the real engine; the full composition end-to-end — real loopback broker, real JSON-RPC endpoint, vend, replay, bearer observed. tsc clean; full suite 675 green. --- src/cli/b-program.ts | 40 +++++- src/cli/tests/b-program.spec.ts | 71 +++++++++++ src/cli/trace-consumer.ts | 2 +- src/faculties.ts | 4 +- src/faculties/security/faculty.ts | 11 +- src/faculties/security/types.ts | 7 ++ src/faculties/shell/faculty.ts | 30 +++-- src/faculties/shell/rpc-auth.threads.ts | 115 ++++++++++++++++++ .../shell/tests/rpc-auth.threads.spec.ts | 105 ++++++++++++++++ src/faculties/shell/tests/rpc-op.spec.ts | 38 ++++++ src/faculties/shell/types.ts | 18 ++- 11 files changed, 427 insertions(+), 14 deletions(-) create mode 100644 src/faculties/shell/rpc-auth.threads.ts create mode 100644 src/faculties/shell/tests/rpc-auth.threads.spec.ts diff --git a/src/cli/b-program.ts b/src/cli/b-program.ts index 87dacb0a3..f36cca813 100644 --- a/src/cli/b-program.ts +++ b/src/cli/b-program.ts @@ -7,6 +7,9 @@ import { McpCancelEventSchema, McpRequestEventSchema, McpRequestResultEventSchema, + SecurityCancelEventSchema, + SecurityRequestEventSchema, + SecurityRequestResultEventSchema, ShellCancelEventSchema, ShellRequestEventSchema, ShellRequestResultEventSchema, @@ -17,6 +20,7 @@ import { import { handleFrontierMessage } from '../faculties/frontier/faculty.ts' import { mcpThreads } from '../faculties/mcp/threads.ts' import { bindEmit } from '../faculties/process-lane.ts' +import { rpcAuthThreads } from '../faculties/shell/rpc-auth.threads.ts' import { shellThreads } from '../faculties/shell/threads.ts' import { useFaculty } from '../faculties/use-faculty.ts' import type { Faculty } from '../faculties.ts' @@ -84,6 +88,7 @@ export const bProgram = ({ faculties, shell: shellOverride, store: storeOverride, + security: securityOverride, systemOne: systemOneOverride, systemTwo: systemTwoOverride, }: { @@ -93,6 +98,12 @@ export const bProgram = ({ shell?: ReturnType /** The store faculty override: a pre-curried useFaculty return (durable store). */ store?: ReturnType + /** + * The security faculty override: a pre-curried useFaculty return carrying + * env-data (the broker binding — spawned children see STARTUP env only, so + * hosts binding the broker mid-process pass it explicitly here). + */ + security?: ReturnType /** * The System One faculty override (e.g. `useSystemOne({ endpoint })`). No * default: with no override the faculty carries no endpoint, so it is simply @@ -106,7 +117,7 @@ export const bProgram = ({ */ systemTwo?: ReturnType }) => { - const enabled = new Set(faculties === undefined ? ['shell', 'store', 'mcp'] : faculties) + const enabled = new Set(faculties === undefined ? ['shell', 'store', 'mcp', 'security'] : faculties) const has = (faculty: Faculty): boolean => enabled.has(faculty) // ── The engine, in-process ──────────────────────────────────────────────── @@ -184,6 +195,22 @@ export const bProgram = ({ resultSchema: McpRequestResultEventSchema, })(facultyAddThreads) + // The security faculty: the cross-cutting credential/policy faculty — its + // vending leg serves shell (remote rpc), system-two endpoints, ATProto, + // and any future remote faculty. No threads of its own yet (the skeleton + // vends); the rpc auth seam's pack lives with the op it serves. + const security = + securityOverride === undefined + ? useFaculty({ + command: ['bun', 'run', 'security/faculty.ts'], + name: 'security', + threads: [], + requestSchema: SecurityRequestEventSchema, + cancelSchema: SecurityCancelEventSchema, + resultSchema: SecurityRequestResultEventSchema, + })(facultyAddThreads) + : securityOverride(facultyAddThreads) + // ── Routing: event type → faculty lane (the only faculty knowledge) ──────── // The root guard threads are always mounted, independent of the allow-list. @@ -234,6 +261,16 @@ export const bProgram = ({ gate: (event: BPEvent): boolean => mcp.invalidEventGate(event), }) } + if (has('security')) { + route([FACULTY_MESSAGE_KINDS.credential_request, FACULTY_MESSAGE_KINDS.credential_cancel], { + send: (event: BPEvent): void => security.send(event), + gate: (event: BPEvent): boolean => security.invalidEventGate(event), + }) + } + + // The rpc auth seam: the vend-and-replay spine requires the op (shell) and + // the vending leg (security) — the pack mounts only when both are on. + if (has('shell') && has('security')) facultyAddThreads(rpcAuthThreads) // ── The engine pump: traces out, gated events to their faculty lanes ───── @@ -279,6 +316,7 @@ export const bProgram = ({ shell.terminate() store.terminate() mcp.terminate() + security.terminate() systemOne?.terminate() systemTwo?.terminate() }, diff --git a/src/cli/tests/b-program.spec.ts b/src/cli/tests/b-program.spec.ts index 5ab5455a7..1872a6ea4 100644 --- a/src/cli/tests/b-program.spec.ts +++ b/src/cli/tests/b-program.spec.ts @@ -3,6 +3,9 @@ import { TRACE_MESSAGE_KINDS } from '../../behavioral/behavioral.constants.ts' import type { SelectionTrace, Trace } from '../../behavioral/behavioral.types.ts' import { FACULTY_MESSAGE_KINDS } from '../../faculties/faculties.constants.ts' import { + SecurityCancelEventSchema, + SecurityRequestEventSchema, + SecurityRequestResultEventSchema, ShellCancelEventSchema, ShellRequestEventSchema, ShellRequestResultEventSchema, @@ -434,4 +437,72 @@ describe('bProgram — the runtime composition', () => { await server.close() } }) + + test('the credential seam ships with shell+security: an auth rpc op vends, then replays with the bearer', async () => { + // The JSON-RPC endpoint requires a bearer; the broker vends one. + const seenAuth: Array = [] + const rpc = Bun.serve({ + port: 0, + fetch: async (request) => { + seenAuth.push(request.headers.get('authorization') ?? undefined) + if (!request.headers.has('authorization')) return new Response('unauthorized', { status: 401 }) + const body = (await request.json()) as { id?: unknown } + return Response.json({ jsonrpc: '2.0', id: body.id, result: { echoed: true } }) + }, + }) + const broker = Bun.serve({ + port: 0, + fetch: () => Response.json({ token: 'broker-tok-1' }), + }) + const brokerUrl = `http://localhost:${broker.port}/` + // Spawned children see STARTUP env only — the broker binding rides the + // security override's env-data (the shell/store override pattern). + const { runtime, traces } = startRuntime({ + security: useFaculty({ + command: ['bun', 'run', 'security/faculty.ts'], + name: 'security', + threads: [], + env: { MCP_BROKER_URL: brokerUrl, MCP_BROKER_BOOT_SECRET: 'boot-secret' }, + requestSchema: SecurityRequestEventSchema, + cancelSchema: SecurityCancelEventSchema, + resultSchema: SecurityRequestResultEventSchema, + }), + }) + try { + runtime.trigger({ + type: FACULTY_MESSAGE_KINDS.shell_request, + detail: { + id: 'rpc-auth-1', + input: { op: 'rpc', url: `http://localhost:${rpc.port}/mcp`, method: 'tools/list', auth: true }, + }, + }) + // The first attempt short-circuits as credential_required; the thread + // vends through the security faculty and replays with the token. + await waitForTraces(traces, (s) => + selectionsOf(s).some( + (t) => + t.selected.type === FACULTY_MESSAGE_KINDS.shell_request_result && + (t.selected.detail as { id?: string } | undefined)?.id === 'rpc-auth-1' && + (t.selected.detail as { ok?: boolean }).ok === true, + ), + ) + const results = selectionsOf(traces).filter( + (t) => + t.selected.type === FACULTY_MESSAGE_KINDS.shell_request_result && + (t.selected.detail as { id?: string }).id === 'rpc-auth-1', + ) + expect(results.length).toBe(2) + const first = results[0]?.selected.detail as { error?: { code?: string } } | undefined + expect(first?.error?.code).toBe('credential_required') + const final = results[1]?.selected.detail as { ok?: boolean; result?: { output?: { echoed?: unknown } } } + expect(final.ok).toBe(true) + expect(final.result?.output?.echoed).toBe(true) + // The security faculty vended from the broker; the remote saw the bearer. + expect(seenAuth[0]).toBe('Bearer broker-tok-1') + } finally { + runtime.terminate() + rpc.stop(true) + broker.stop(true) + } + }) }) diff --git a/src/cli/trace-consumer.ts b/src/cli/trace-consumer.ts index 0dd77d0bd..50b919ed7 100644 --- a/src/cli/trace-consumer.ts +++ b/src/cli/trace-consumer.ts @@ -40,7 +40,7 @@ const SENSITIVE_KEY = /(^|_)(TOKEN|SECRET|PASSWORD|PASSWD|CREDENTIAL|API_?KEY|PR /** Object field names whose string values are always redacted. */ const SENSITIVE_FIELD = - /^(authorization|proxy-authorization|cookie|set-cookie|x-api-key|api[_-]?key|access[_-]?token|refresh[_-]?token|secret|password|credential|token)s?$/i + /^(authorization|proxy-authorization|cookie|set-cookie|x-api-key|api[_-]?key|access[_-]?token|refresh[_-]?token|auth_?token|secret|password|credential|token)s?$/i /** * The redaction registry — secret VALUES. A key is in scope when it matches diff --git a/src/faculties.ts b/src/faculties.ts index a05c8b06f..188c115ab 100644 --- a/src/faculties.ts +++ b/src/faculties.ts @@ -13,11 +13,13 @@ */ /** The selectable capability faculties (the `bProgram` allow-list). */ -export type Faculty = 'shell' | 'store' | 'mcp' +export type Faculty = 'shell' | 'store' | 'mcp' | 'security' export * from './faculties/faculties.types.ts' export * from './faculties/mcp/threads.ts' export * from './faculties/mcp/types.ts' +export * from './faculties/security/types.ts' +export * from './faculties/shell/rpc-auth.threads.ts' export * from './faculties/shell/threads.ts' export * from './faculties/shell/types.ts' export * from './faculties/store/threads.ts' diff --git a/src/faculties/security/faculty.ts b/src/faculties/security/faculty.ts index cfcb16866..d49a23d8f 100644 --- a/src/faculties/security/faculty.ts +++ b/src/faculties/security/faculty.ts @@ -109,18 +109,21 @@ const stopVend = ({ vend }: { vend: Vend }): void => { const postResult = ({ id, token, + echo, error, space, }: { id: string token?: string + /** The ctx echo — the caller's out-of-band join payload, round-tripped verbatim. */ + echo?: JsonObject error?: { code: string; message?: string } space?: string }): void => { emit({ type: FACULTY_MESSAGE_KINDS.credential_result, detail: (error === undefined - ? { id, ok: true, result: { token } } + ? { id, ok: true, result: { token, ...(echo === undefined ? {} : { echo }) } } : { id, ok: false, error: error as JsonObject }) as JsonObject & { id: string }, ...(space === undefined ? {} : { space }), }) @@ -172,6 +175,10 @@ const handleInbound = async (message: unknown): Promise => { } issuer = (ctx as { issuer?: string }).issuer } + // The out-of-band echo rides ctx verbatim — the orchestrating thread's + // join payload (the you.com MCP `_meta` pattern: host-supplied data + // round-trips beside the model-facing arguments). + const echo = (ctx as { echo?: JsonObject } | undefined)?.echo const { serverUrl } = input as { serverUrl: string } const vend: Vend = { stopReason: null } @@ -192,7 +199,7 @@ const handleInbound = async (message: unknown): Promise => { }) return } - postResult({ id, space: event.space, token }) + postResult({ id, space: event.space, token, echo }) } catch (err) { // The vend is fail-closed by construction; this is the last-resort guard // so no worker-side throw ever escapes as a crash. diff --git a/src/faculties/security/types.ts b/src/faculties/security/types.ts index 929fb4364..7840167a3 100644 --- a/src/faculties/security/types.ts +++ b/src/faculties/security/types.ts @@ -138,6 +138,12 @@ export const validateCredentialRequestInput = ajv.compile(CredentialRequestInput export type SecurityRequestContext = { /** The resolved authorization-server `issuer` — binds the keychain read. */ issuer?: string + /** + * The caller's out-of-band join payload (e.g. the original request for the + * replaying thread) — echoed verbatim on the vended `credential_result`. + * Never model-facing; never interpreted by this faculty. + */ + echo?: JsonObject } export const SecurityRequestContextSchema: JSONSchemaType = { @@ -149,6 +155,7 @@ export const SecurityRequestContextSchema: JSONSchemaType undefined +const needsCredential = (input: ShellRpcOpInput): boolean => input.auth === true && input.authToken === undefined // --------------------------------------------------------------------------- // In-flight execution — enough state to stop it by correlation id @@ -330,6 +332,16 @@ const runRpcOp = async ({ options: ShellOptions }): Promise => { const started = performance.now() + // The declarative auth gate: no vended token, no call — the typed + // credential_required result is the thread pack's capture payload. + if (needsCredential(input)) { + return { + code: 'credential_required', + durationMs: 0, + message: `credential required for ${input.url}`, + request: { op: 'rpc', input }, + } + } const timeoutMs = options.timeoutMs ?? DEFAULT_TIMEOUT_MS const controller = new AbortController() const execution: Execution = { kind: 'rpc', controller, stopReason: null } @@ -340,7 +352,9 @@ const runRpcOp = async ({ url: input.url, method: input.method, ...(input.params === undefined ? {} : { params: input.params }), - getAuthToken, + // The replayed token rides the input (thread-injected); the op itself + // never knows OAuth. + getAuthToken: async () => input.authToken, signal: controller.signal, }) const durationMs = Math.round(performance.now() - started) diff --git a/src/faculties/shell/rpc-auth.threads.ts b/src/faculties/shell/rpc-auth.threads.ts new file mode 100644 index 000000000..454259d1b --- /dev/null +++ b/src/faculties/shell/rpc-auth.threads.ts @@ -0,0 +1,115 @@ +/** + * The rpc auth seam's thread pack — the vend-and-replay spine that wires the + * shell faculty's `rpc` op to the security faculty's credential vending. + * + * @remarks + * The declarative flow: an `rpc` op declared `auth: true` short-circuits (in + * the shell faculty) as a typed `credential_required` result echoing the + * originating request — the capture payload. This pack completes the + * round-trip: + * + * 1. **requestor** — a `credential_required` result (first attempt only: a + * replayed call already carries `authToken`) transforms into a + * `credential_request { serverUrl }` with the original call riding + * `ctx.echo` — the out-of-band lane (the you.com MCP pattern: host-supplied + * data rides `detail.ctx` beside `input`, never as a model-facing field). + * 2. **replayer** — the vended `credential_result` (token + echoed ctx) + * replays the original `shell_request` with the bearer merged into the + * input. The security faculty never talks to shell directly; this thread + * is the only join. + * + * The absent-credential branch is deliberately inert: the caller already + * holds the terminal `credential_required` error, and a replay has nothing + * to join on — the failed vend is visible as an unmatched `credential_result` + * in the frontier traces. MINIMAL: no retry/backoff policy here; the generic + * retry thread pattern covers it when needed. + * + * Requires shell + security — bProgram mounts it only when both are on. + * + * @packageDocumentation + */ + +import type { Thread } from '../../behavioral/behavioral.types.ts' +import { FACULTY_MESSAGE_KINDS } from '../faculties.constants.ts' + +// ── Vocabulary ─────────────────────────────────────────────────────────────── + +/** The correlation-suffix distinguishing a vend round-trip from its call. */ +export const CREDENTIAL_SUFFIX = '-cred' + +/** The gate schema for the shell result events the requestor consumes. */ +const RPC_AUTH_RESULT_DETAIL = { + type: 'object', + properties: { + id: { type: 'string', minLength: 1 }, + ok: { type: 'boolean' }, + result: { type: 'object' }, + error: { type: 'object' }, + }, + required: ['id', 'ok'], +} as const + +// ── Threads ─────────────────────────────────────────────────────────────────── + +/** + * requestor — a `credential_required` shell result (first attempt only: the + * gate requires no `authToken`, bounding the loop on a post-vend 401) + * requests a credential for the call's server URL, carrying the original + * request out-of-band in `ctx.echo` for the replay join. + */ +const credRequestor: Thread = { + label: 'rpc-auth/requestor', + rules: [ + { + transform: [ + { + type: FACULTY_MESSAGE_KINDS.shell_request_result, + query: + '. as $d | select($d.ok == false and $d.error.code? == "credential_required" and $d.error.request.input.auth == true and ($d.error.request.input.authToken == null)) | {id: ($d.id + "-cred"), input: {serverUrl: $d.error.request.input.url}, ctx: {echo: {id: $d.id, input: $d.error.request.input}}}', + target: FACULTY_MESSAGE_KINDS.credential_request, + detailSchema: RPC_AUTH_RESULT_DETAIL, + }, + ], + }, + ], +} + +/** + * replayer — a vended `credential_result` carrying the echoed request + * replays the original `shell_request` with the bearer merged into the + * input. The op proceeds with the token; the op itself never knew OAuth. + */ +const credReplayer: Thread = { + label: 'rpc-auth/replayer', + rules: [ + { + transform: [ + { + type: FACULTY_MESSAGE_KINDS.credential_result, + query: + '. as $d | select($d.ok == true and ($d.result.echo != null)) | {id: $d.result.echo.id, input: ($d.result.echo.input + {authToken: $d.result.token})}', + target: FACULTY_MESSAGE_KINDS.shell_request, + detailSchema: { + type: 'object', + properties: { + id: { type: 'string', minLength: 1 }, + ok: { type: 'boolean' }, + result: { + type: 'object', + properties: { token: { type: 'string' }, echo: { type: 'object' } }, + required: ['token', 'echo'], + }, + }, + required: ['id', 'ok', 'result'], + }, + }, + ], + }, + ], +} + +/** + * The rpc auth thread library — add to the program alongside the shell + * faculty; requires the security faculty for the vending leg. + */ +export const rpcAuthThreads: Thread[] = [credRequestor, credReplayer] diff --git a/src/faculties/shell/tests/rpc-auth.threads.spec.ts b/src/faculties/shell/tests/rpc-auth.threads.spec.ts new file mode 100644 index 000000000..1d639f7ba --- /dev/null +++ b/src/faculties/shell/tests/rpc-auth.threads.spec.ts @@ -0,0 +1,105 @@ +import { describe, expect, test } from 'bun:test' +import { TRACE_MESSAGE_KINDS } from '../../../behavioral/behavioral.constants.ts' +import { behavioral } from '../../../behavioral/behavioral.ts' +import type { BPEvent, JsonObject, SelectionTrace, Trace } from '../../../behavioral/behavioral.types.ts' +import { FACULTY_MESSAGE_KINDS } from '../../faculties.constants.ts' +import { rpcAuthThreads } from '../rpc-auth.threads.ts' + +/** + * The rpc auth seam's thread library against the real engine — the + * vend-and-replay spine: a typed `credential_required` shell result requests + * a credential (carrying the original call out-of-band in `ctx.echo`), and + * the vended `credential_result` replays the call with the token merged in. + * The op never knows OAuth; the thread orchestrates the cross-faculty + * round-trip. + */ + +type Selected = { type: string; detail: Record | undefined } + +const runProgram = (events: BPEvent[]): Selected[] => { + const program = behavioral() + const selected: Selected[] = [] + program.useTrace((trace: Trace) => { + if (trace.kind === TRACE_MESSAGE_KINDS.selection) + selected.push({ + type: (trace as SelectionTrace).selected.type, + detail: (trace as SelectionTrace).selected.detail as Record | undefined, + }) + }) + for (const thread of rpcAuthThreads) program.addThread(thread) + for (const event of events) + program.addThread({ label: `producer/${event.type}`, once: true, rules: [{ request: event }] }) + // addThread is inert — trigger admits one ingress event and runs one + // super-step; the second pump cascades transform re-entries. + program.trigger({ type: 'rpc_auth_pump', detail: {} }) + program.trigger({ type: 'rpc_auth_pump', detail: {} }) + return selected +} + +const credentialRequired = (id: string, url: string, extraInput: JsonObject = {}): BPEvent => ({ + type: FACULTY_MESSAGE_KINDS.shell_request_result, + detail: { + id, + ok: false, + error: { + code: 'credential_required', + durationMs: 1, + message: 'credential required', + request: { op: 'rpc', input: { op: 'rpc', url, method: 'tools/list', auth: true, ...extraInput } }, + }, + }, +}) + +const vended = (credId: string, echo: { id: string; input: JsonObject }): BPEvent => ({ + type: FACULTY_MESSAGE_KINDS.credential_result, + detail: { id: credId, ok: true, result: { token: 'vended-1', echo } }, +}) + +describe('rpc auth threads — the vend-and-replay spine', () => { + test('a credential_required result requests a credential carrying the original call in ctx.echo', () => { + const selected = runProgram([credentialRequired('c1', 'https://mcp.example.com/mcp')]) + const request = selected.find((s) => s.type === FACULTY_MESSAGE_KINDS.credential_request) + expect(request).toBeDefined() + const detail = request?.detail as { + id?: string + input?: { serverUrl?: string } + ctx?: { echo?: { id?: string; input?: Record } } + } + expect(detail.id).toBe('c1-cred') + expect(detail.input?.serverUrl).toBe('https://mcp.example.com/mcp') + expect(detail.ctx?.echo).toEqual({ + id: 'c1', + input: { op: 'rpc', url: 'https://mcp.example.com/mcp', auth: true, method: 'tools/list' }, + }) + }) + + test('the vended credential replays the call with the bearer merged in', () => { + const selected = runProgram([ + vended('c2-cred', { id: 'c2', input: { op: 'rpc', url: 'https://mcp.example.com/mcp', auth: true } }), + ]) + const replay = selected.find((s) => s.type === FACULTY_MESSAGE_KINDS.shell_request) + expect(replay).toBeDefined() + const detail = replay?.detail as { id?: string; input?: { authToken?: string; auth?: boolean; url?: string } } + expect(detail.id).toBe('c2') + expect(detail.input?.authToken).toBe('vended-1') + expect(detail.input?.auth).toBe(true) + expect(detail.input?.url).toBe('https://mcp.example.com/mcp') + }) + + test('an absent credential never replays — the caller keeps the credential_required error', () => { + const selected = runProgram([ + { + type: FACULTY_MESSAGE_KINDS.credential_result, + detail: { id: 'c3-cred', ok: false, error: { code: 'error', message: 'no credential' } }, + }, + ]) + expect(selected.some((s) => s.type === FACULTY_MESSAGE_KINDS.shell_request)).toBe(false) + }) + + test('a replayed call that fails again is not re-captured — the loop is bounded', () => { + // The replayed request carries the token; its credential_required-shaped + // failure (a 401 after vend) does not match the requestor's gate. + const selected = runProgram([credentialRequired('c4', 'https://mcp.example.com/mcp', { authToken: 'vended-1' })]) + expect(selected.some((s) => s.type === FACULTY_MESSAGE_KINDS.credential_request)).toBe(false) + }) +}) diff --git a/src/faculties/shell/tests/rpc-op.spec.ts b/src/faculties/shell/tests/rpc-op.spec.ts index dedd0a597..cd5634efc 100644 --- a/src/faculties/shell/tests/rpc-op.spec.ts +++ b/src/faculties/shell/tests/rpc-op.spec.ts @@ -152,4 +152,42 @@ describe('shell rpc op', () => { expect(result.error?.code).toBe('error') expect(result.error?.message).toContain('invalid input') }) + + test('an auth-declared rpc op without a token short-circuits as credential_required data', async () => { + let hits = 0 + const server = rpcServer(() => { + hits += 1 + return Response.json({ jsonrpc: '2.0', id: 1, result: {} }) + }) + servers.push(server) + const worker = spawnShellWorker() + workers.push(worker) + worker.call({ id: 'rpc8', input: { op: 'rpc', url: server.url, method: 'tools/list', auth: true } }) + const raw = await worker.resultFor('rpc8') + const result = wire(raw) + expect(result.ok).toBe(false) + expect(result.error?.code).toBe('credential_required') + // The echo is the replay capture payload — the thread re-issues the call + // with the vended token; nothing reached the remote. + const error = result.error as { request?: { op?: string; input?: { url?: string; auth?: boolean } } } + expect(error.request?.op).toBe('rpc') + expect(error.request?.input?.url).toBe(server.url) + expect(error.request?.input?.auth).toBe(true) + expect(hits).toBe(0) + }) + + test('a token injected by the replay rides the fetch as a bearer header', async () => { + const server = rpcServer((body) => Response.json({ jsonrpc: '2.0', id: body.id, result: { ok: 1 } })) + servers.push(server) + const worker = spawnShellWorker() + workers.push(worker) + worker.call({ + id: 'rpc9', + input: { op: 'rpc', url: server.url, method: 'tools/list', auth: true, authToken: 'vended-tok' }, + }) + const raw = await worker.resultFor('rpc9') + const result = wire(raw) + expect(result.ok).toBe(true) + expect(server.requests[0]?.headers.authorization).toBe('Bearer vended-tok') + }) }) diff --git a/src/faculties/shell/types.ts b/src/faculties/shell/types.ts index 3fe352458..71cfd50af 100644 --- a/src/faculties/shell/types.ts +++ b/src/faculties/shell/types.ts @@ -108,6 +108,15 @@ export type ShellRpcOpInput = { method: string /** The JSON-RPC params object, when the method takes one. */ params?: JsonObject + /** + * Declare the call needs a vended credential: without a token the op + * short-circuits as typed `credential_required` (the thread pack's vend- + * and-replay capture payload) — it never calls the remote unauthenticated. + * The token itself rides `authToken`, injected by the replaying thread. + */ + auth?: boolean + /** The vended bearer token — set by the replaying thread, never model input. */ + authToken?: string /** Wall-clock deadline for the call. @default 30_000 */ timeoutMs?: number } @@ -160,6 +169,8 @@ export const ShellRpcOpInputSchema: JSONSchemaType = { url: { type: 'string', minLength: 1 }, method: { type: 'string', minLength: 1 }, params: { type: 'object', required: [], additionalProperties: true, nullable: true }, + auth: { type: 'boolean', nullable: true }, + authToken: { type: 'string', nullable: true }, timeoutMs: { type: 'integer', minimum: 1, nullable: true }, }, required: ['op', 'url', 'method'], @@ -228,7 +239,7 @@ export type ShellError = { // --------------------------------------------------------------------------- /** Terminal status of one rpc op — process-op statuses do not apply (no pid). */ -export type RpcStatus = 'canceled' | 'timeout' | 'error' +export type RpcStatus = 'canceled' | 'timeout' | 'error' | 'credential_required' /** The rpc success payload — the remote call's decoded `result` rides `output`. */ export type RpcOpSuccess = { @@ -252,6 +263,11 @@ export type RpcOpError = { remoteCode?: number | string durationMs: number clamped?: string[] + /** + * The originating request, echoed only on `credential_required` — the + * vend-and-replay capture payload (the thread's join via `ctx.echo`). + */ + request?: { op: 'rpc'; input: ShellRpcOpInput } } /** Every op runner's interior — one `code`-discriminated error branch over two payload families. */ From 7da427af5715651fa4bb873ab754375885b7e0c7 Mon Sep 17 00:00:00 2001 From: Edward Irby Date: Fri, 25 Sep 2026 14:47:01 -0700 Subject: [PATCH 14/55] =?UTF-8?q?feat(shell):=20the=20remote-mcp=20thread?= =?UTF-8?q?=20pack=20=E2=80=94=20deprecate=20the=20mcp=20faculty?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Slice 5 of the mcp-into-shell handoff: the MCP layering lives in `shell/remote-mcp.threads.ts` over the generic `rpc` op, and the mcp faculty — with the `@modelcontextprotocol/*` dependency — is gone. - Request stamping: every pack-issued rpc op carries the reserved `_meta` envelope (`io.modelcontextprotocol/protocolVersion` + `clientInfo` + `clientCapabilities`) and the `MCP-Protocol-Version` header (the op gains `headers`); results never carry the request-envelope keys. - Discovery: `remote_mcp_discover` -> `server/discover` -> `tools/list` -> the store registry (`remote-mcp` collection, keyed by server url, alongside skills/plugins) + `remote_mcp_discovered` surfacing. - Execution: `remote_mcp_call` -> `tools/call` -> `remote_mcp_call_result`. - MRTR (the 2026-07-28 contract): an `input_required` result (reserved `inputRequests`/`requestState`, at-least-one) surfaces `remote_mcp_elicitation`; the host answers with `remote_mcp_elicitation_response` and the pack retries on a FRESH request id with the answers + a byte-exact `requestState` echo, capped as a typed `round_cap` error. - Retry: deadline/network/5xx re-requests the op with the attempt advanced, bounded; `retry-after` rides as data. - Trust boundary: AJV validates ONLY the four trusted response shapes (exported as `REMOTE_MCP_*_SCHEMA`); a failing shape silently no-matches the acting transform — fail-closed, visible as an unmatched event. - The join lane generalizes: `shell_request` gains optional `detail.ctx`, echoed verbatim on the result (both branches) — the you.com MCP `_meta` pattern. All pack joins (source call, attempt, MRTR round) ride the echo, stateless per transform. The security faculty echoes ctx on failed vends too, so absent credentials surface as typed errors instead of pending callers. - Deprecation: `src/faculties/mcp/` deleted; the mcp_* wire kinds removed from the registry; bProgram's default faculties are shell/store/security; the Faculty union drops 'mcp'; `@modelcontextprotocol/client` + `@modelcontextprotocol/server` removed from package.json; the skill reference moves to `references/remote-mcp.md`; AGENTS.md and plan.md updated. - Specs: the pack against the real engine (stamping, discovery chain, execution, MRTR round-trip + cap, bounded retry, failed-vend surfacing), wire-kind parity in the registry spec, and the composition end-to-end — discovery registering tools through real routing against a loopback JSON-RPC server. tsc clean; full suite 671 green. --- AGENTS.md | 23 +- bun.lock | 28 +- package.json | 2 - skills/behavioral-tools/SKILL.md | 18 +- .../behavioral-tools/references/mcp-client.md | 66 --- .../behavioral-tools/references/remote-mcp.md | 68 +++ src/cli/b-program.ts | 31 +- src/cli/tests/b-program.spec.ts | 111 ++-- src/faculties.ts | 5 +- src/faculties/faculties.constants.ts | 3 - src/faculties/faculties.types.ts | 108 +--- src/faculties/mcp/faculty.ts | 369 ------------- src/faculties/mcp/tests/faculty.spec.ts | 174 ------ .../mcp/tests/fixtures/mcp-server-fixture.ts | 73 --- src/faculties/mcp/tests/threads.spec.ts | 122 ----- src/faculties/mcp/threads.ts | 130 ----- src/faculties/mcp/types.ts | 176 ------ src/faculties/security/faculty.ts | 26 +- src/faculties/shell/faculty.ts | 33 +- src/faculties/shell/remote-mcp.threads.ts | 512 ++++++++++++++++++ src/faculties/shell/rpc-auth.threads.ts | 12 +- src/faculties/shell/rpc.client.ts | 11 +- .../shell/tests/remote-mcp.threads.spec.ts | 359 ++++++++++++ .../shell/tests/rpc-auth.threads.spec.ts | 51 +- src/faculties/shell/tests/rpc-op.spec.ts | 66 ++- src/faculties/shell/types.ts | 3 + src/faculties/tests/faculties.types.spec.ts | 93 +--- src/faculties/use-faculty.ts | 2 +- 28 files changed, 1258 insertions(+), 1417 deletions(-) delete mode 100644 skills/behavioral-tools/references/mcp-client.md create mode 100644 skills/behavioral-tools/references/remote-mcp.md delete mode 100644 src/faculties/mcp/faculty.ts delete mode 100644 src/faculties/mcp/tests/faculty.spec.ts delete mode 100644 src/faculties/mcp/tests/fixtures/mcp-server-fixture.ts delete mode 100644 src/faculties/mcp/tests/threads.spec.ts delete mode 100644 src/faculties/mcp/threads.ts delete mode 100644 src/faculties/mcp/types.ts create mode 100644 src/faculties/shell/remote-mcp.threads.ts create mode 100644 src/faculties/shell/tests/remote-mcp.threads.spec.ts diff --git a/AGENTS.md b/AGENTS.md index 02ee24be8..364b15205 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -95,24 +95,29 @@ faculties), plus its `tests/`: - `system-one/` — TypeSafe/OpenRouter Decisions; `configSystemOne` + `useSystemOne({ endpoint })`, with 429/529 retry - `shell/` — bun-direct script execution — `run` op TS scripts via `bun run -`, - `shell` op Bun Shell commands through the wrapper; temp-file payloads over - ~100KB, deleted on every exit + `shell` op Bun Shell commands through the wrapper; `rpc` op generic remote + JSON-RPC (the remote-mcp layering is the thread pack, not the op); + temp-file payloads over ~100KB, deleted on every exit - `store/` — durable space-scoped persistence -- `mcp/` — remote MCP connections/sessions/auth; `keychain-oauth-provider.ts` - is the MCP OAuth `BunKeychain` over `Bun.secrets` plus the issuer-binding v2 - provider (faculty-only: nothing outside `mcp/` imports it) +- `security/` — the cross-cutting credential/policy faculty: + `keychain-oauth-provider.ts` is the issuer-bound OAuth `BunKeychain` over + `Bun.secrets` (SDK-free plain types in `security/types.ts`); the faculty + vends `credential_request` → `credential_result` (broker env-data first, + keychain floor second) — consumers are shell (remote MCP), system-two, ATProto - `frontier/` — the in-process embed — imported and driven by the composition; standalone spawns are a compatibility entry Each faculty owns its event types + input boundary; results echo the request `space`; op runners errors-as-data. **`src/tools/`** — deleted (fleet 0): the ICL conversion retired the CLI tool -fleet. mcp-client is the mcp faculty (`src/faculties/mcp/faculty.ts`); +fleet. Remote MCP is the remote-mcp thread pack over the shell faculty's +generic `rpc` op (`src/faculties/shell/remote-mcp.threads.ts` — the retired +`mcp` faculty's replacement; the official SDK dependency is gone); skill/plugin operations are the shell faculty's threads (`src/faculties/shell/threads.ts`) + recipes + store, taught by `skills/skill-conventions/`. **`src/faculties.ts`** — the faculties public surface (package export `./faculties`): the `Faculty` union, the wire types + JSON schemas/validators (`faculties.types.ts`), the override thread -threads (`shellThreads`, `mcpThreads`), their schemas/types, `useFaculty`, and the +threads (`shellThreads`), their schemas/types, `useFaculty`, and the System One/Two config surface (`configSystemOne`/`useSystemOne`, `configSystemTwo`/`useSystemTwo`) — what a `config.ts` imports to compose. (`facultiesThreads`, the default root threads, is internal.) The runtime composition itself is `src/cli/b-program.ts`. @@ -141,7 +146,9 @@ traces out), `load-config.ts` (`/config.ts`), and `trace-consum **`src/utils/`** — shared pure utilities. **`src/faculties//threads.ts`** — faculty threads: `shell/threads.ts` (the ICL threads — skill/plugin scans, catalog/manifest schema gates, links dispatchers -+ stored recipes) and `mcp/threads.ts` (the auth replay spine). Threads ship with ++ stored recipes), `shell/rpc-auth.threads.ts` (the credential vend-and-replay +spine), and `shell/remote-mcp.threads.ts` (the MCP layering over the rpc op). +Threads ship with their faculty; `bProgram` mounts the faculty's threads when the faculty and its required faculties are on — except `faculties.threads.ts`, the composition's **root guard threads**, always mounted regardless of the allow-list. The diff --git a/bun.lock b/bun.lock index 3a540fe36..44cf05564 100644 --- a/bun.lock +++ b/bun.lock @@ -11,8 +11,6 @@ "@biomejs/biome": "2.5.12", "@commitlint/cli": "21.2.2", "@commitlint/config-conventional": "21.2.2", - "@modelcontextprotocol/client": "^2.0.0", - "@modelcontextprotocol/server": "^2.0.0", "@types/bun": "^1.4.1", "@webref/css": "^8.7.4", "commitlint": "21.2.2", @@ -101,12 +99,6 @@ "@inquirer/external-editor": ["@inquirer/external-editor@1.0.3", "", { "dependencies": { "chardet": "^2.1.1", "iconv-lite": "^0.7.0" }, "peerDependencies": { "@types/node": ">=18" }, "optionalPeers": ["@types/node"] }, "sha512-RWbSrDiYmO4LbejWY7ttpxczuwQyZLBUyygsA9Nsv95hpzUWwnNTVQmAq3xuh7vNwCp07UTmE5i11XAEExx4RA=="], - "@modelcontextprotocol/client": ["@modelcontextprotocol/client@2.0.0", "", { "dependencies": { "@modelcontextprotocol/core": "2.0.0", "cross-spawn": "^7.0.5", "eventsource": "^3.0.2", "eventsource-parser": "^3.0.0", "jose": "^6.1.3", "pkce-challenge": "^5.0.0", "zod": "^4.2.0" } }, "sha512-8f1OghQ2rjzIOfqgUCP+8GiUWqRs89njoWLNqAe8kWmDePv3s1fZXseej+QXemssEuuOvLLmLO/kqM3IQHtISw=="], - - "@modelcontextprotocol/core": ["@modelcontextprotocol/core@2.0.0", "", { "dependencies": { "zod": "^4.2.0" } }, "sha512-pJCEwGG7Lfr/+PQp9ZTwKXNeO5wzbfKL7H3MYpCorM4oFBoQrdjnBgEoqG+RjhsvS1FKrDbKux+M1HhlnGWqcA=="], - - "@modelcontextprotocol/server": ["@modelcontextprotocol/server@2.0.0", "", { "dependencies": { "@modelcontextprotocol/core": "2.0.0", "zod": "^4.2.0" } }, "sha512-YhHWdHfpFMQfd0prsEnxKeS3Qz3ytIGmsS0sth4KDjnacIT7hxk6hXHkJ9KysxlkvTM+WZAtQbbcUhdoP4Hvtw=="], - "@simple-libs/child-process-utils": ["@simple-libs/child-process-utils@2.0.0", "", { "dependencies": { "@simple-libs/stream-utils": "^2.0.0" } }, "sha512-dvNoRKLijXnD0XoJAz94pbNuB5GQgDr55UhpSPhffDkTT0Cmcqh9jSCOtwfT2d4H6MI9E7c4SgtMuJXZ6F3c6A=="], "@simple-libs/stream-utils": ["@simple-libs/stream-utils@2.0.0", "", {}, "sha512-fCTuZK4QBa+39Oz9l4OGfJfz+GpwCp3AqO7Zch3to99xHPgstVsRFpeQ8LNd2o1Gv8raL2mCFwiaHh7bFSp5DQ=="], @@ -227,8 +219,6 @@ "cosmiconfig-typescript-loader": ["cosmiconfig-typescript-loader@6.3.0", "", { "dependencies": { "jiti": "2.6.1" }, "peerDependencies": { "@types/node": "*", "cosmiconfig": ">=9", "typescript": ">=5" } }, "sha512-Akr82WH1Wfqatyiqpj8HDkO2o2KmJRu1FhKfSNJP3K4IdXwHfEyL7MOb62i1AGQVLtIQM+iCE9CGOtrfhR+mmA=="], - "cross-spawn": ["cross-spawn@7.0.6", "", { "dependencies": { "path-key": "^3.1.0", "shebang-command": "^2.0.0", "which": "^2.0.1" } }, "sha512-uV2QOWP2nWzsy2aMp8aRibhi9dlzF5Hgh5SHaB9OiTGEyDTiJJyx0uy51QXdyWbtAHNua4XJzUKca3OzKUd3vA=="], - "css-tree": ["css-tree@3.2.1", "", { "dependencies": { "mdn-data": "2.27.1", "source-map-js": "^1.2.1" } }, "sha512-X7sjQzceUhu1u7Y/ylrRZFU2FS6LRiFVp6rKLPg23y3x3c3DOKAwuXGDp+PAGjh6CSnCjYeAul8pcT8bAl+lSA=="], "cz-conventional-changelog": ["cz-conventional-changelog@3.3.0", "", { "dependencies": { "chalk": "^2.4.1", "commitizen": "^4.0.3", "conventional-commit-types": "^3.0.0", "lodash.map": "^4.5.1", "longest": "^2.0.1", "word-wrap": "^1.0.3" }, "optionalDependencies": { "@commitlint/load": ">6.1.1" } }, "sha512-U466fIzU5U22eES5lTNiNbZ+d8dfcHcssH4o7QsdWaCcRs/feIPCxKYSWkYBNs5mny7MvEfwpTLWjvbm94hecw=="], @@ -253,10 +243,6 @@ "escape-string-regexp": ["escape-string-regexp@1.0.5", "", {}, "sha512-vbRorB5FUQWvla16U8R/qgaFIya2qGzwDrNmCZuYKrbdSUMG6I1ZCGQRefkRVhuOkIGVne7BQ35DSfo1qvJqFg=="], - "eventsource": ["eventsource@3.0.7", "", { "dependencies": { "eventsource-parser": "^3.0.1" } }, "sha512-CRT1WTyuQoD771GW56XEZFQ/ZoSfWid1alKGDYMmkt2yl8UXrVR4pspqWNEcqKvVIzg6PAltWjxcSSPrboA4iA=="], - - "eventsource-parser": ["eventsource-parser@3.1.1", "", {}, "sha512-EKN1vKAMcZ8MlYMpaNuxN6R9yakzH6uajHcHVTqWJzvu5pWw9DyhbP35HH8MVBQ+dZjAfDxk+A8NiR9KWaXiyQ=="], - "expand-tilde": ["expand-tilde@2.0.2", "", { "dependencies": { "homedir-polyfill": "^1.0.1" } }, "sha512-A5EmesHW6rfnZ9ysHQjPdJRni0SRar0tjtG5MNtm9n5TUvsYU8oozprtRD4AqHxcZWWlVuAmQo2nWKfN9oyjTw=="], "fast-deep-equal": ["fast-deep-equal@3.1.3", "", {}, "sha512-f3qQ9oQy9j2AhBe/H9VC91wLmKBCCU/gDOnKNAYG5hswO7BLKj09Hc5HYNz9cGI++xlpDCIgDaitVs03ATR84Q=="], @@ -337,8 +323,6 @@ "joi": ["joi@18.2.9", "", { "dependencies": { "@hapi/address": "^5.1.1", "@hapi/formula": "^3.0.2", "@hapi/hoek": "^11.0.7", "@hapi/pinpoint": "^2.0.1", "@hapi/tlds": "^1.1.1", "@hapi/topo": "^6.0.2", "@standard-schema/spec": "^1.1.0" } }, "sha512-2mD929bUVKUhOLQQEVhlf6EZ0Mlo0DeRb5MO7cViR9AXLtBauuccEtB1py9Ocxpo/P7ucnh442iY/iOwrh3IQw=="], - "jose": ["jose@6.2.12", "", {}, "sha512-9NiFmJEex0sy2Dk58j2UGBSHgUs2ypF9eZSu4L6vjOX3Dp96Sw1F3uL+H+D1sx02jZZdzUT0HgvCy59CuvXcWw=="], - "jq-wasm": ["jq-wasm@3.0.0-jq-1.8.2", "", {}, "sha512-jgWSEBJSd0lYR4Q5Fw8333MxQS5jCRI+g9KwAGL7yK1spwzTJy8C5uOS08wbpKE0Gz8oQYLlRpNLE/W70Ksp2g=="], "js-tokens": ["js-tokens@4.0.0", "", {}, "sha512-RdJUflcE3cUzKiMqQgsCu06FPu9UdIJO0beYbPhHN4k6apgJtifcoCtT9bcxOpYBtpD2kCM6Sbzg4CausW/PKQ=="], @@ -393,14 +377,10 @@ "path-is-absolute": ["path-is-absolute@1.0.1", "", {}, "sha512-AVbw3UJ2e9bq64vSaS9Am0fje1Pa8pbGqTTsmXfaIiMpnr5DlDhfJOuLj9Sf95ZPVDAUerDfEk88MPmPe7UCQg=="], - "path-key": ["path-key@3.1.1", "", {}, "sha512-ojmeN0qd+y0jszEtoY48r0Peq5dwMEkIlCOu6Q5f41lfkswXuKtYrhgoTpLnyIcHm24Uhqx+5Tqm2InSwLhE6Q=="], - "picocolors": ["picocolors@1.1.1", "", {}, "sha512-xceH2snhtb5M9liqDsmEw56le376mTZkEX/jEb/RxNFyegNul7eNslCXP9FDj/Lcu0X8KEyMceP2ntpaHrDEVA=="], "picomatch": ["picomatch@4.0.7", "", {}, "sha512-qcJu88Q2IWqJsDD529JKMdwGm/dvInW4HvQnRwiH9JtihJvzGOscDtHE3x1pBKeUOTysQ8kVmLnJ2kJu7yhcGA=="], - "pkce-challenge": ["pkce-challenge@5.0.1", "", {}, "sha512-wQ0b/W4Fr01qtpHlqSqspcj3EhBvimsdh0KlHhH8HRZnMsEa0ea2fTULOXOS9ccQr3om+GcGRk4e+isrZWV8qQ=="], - "prettier": ["prettier@3.9.9", "", { "bin": { "prettier": "bin/prettier.cjs" } }, "sha512-Z/CJHIkdujO/OtN7nXUii0Rf3VT5SRuhjBA82Xvu2XhBUgX3nhP67T0LHceBdQLex7OOFGTox+Q5Yg8Jk2Qivg=="], "readable-stream": ["readable-stream@3.6.2", "", { "dependencies": { "inherits": "^2.0.3", "string_decoder": "^1.1.1", "util-deprecate": "^1.0.1" } }, "sha512-9u/sniCrY3D5WdsERHzHE4G2YCXqoG5FTHUiCC4SIbr6XcLZBY05ya9EKjYek9O5xOAwjGq+1JdGBAS7Q9ScoA=="], @@ -423,10 +403,6 @@ "semver": ["semver@7.8.5", "", { "bin": { "semver": "bin/semver.js" } }, "sha512-Y7/KDsb8LjooZpwaqGyulO6DQlksgCncchHGk+sZIY4SBvUocMBEFH5Ur1fI4dV+Jvl0w6cjvucaIi40puRioA=="], - "shebang-command": ["shebang-command@2.0.0", "", { "dependencies": { "shebang-regex": "^3.0.0" } }, "sha512-kHxr2zZpYtdmrN1qDjrrX/Z1rR1kG8Dx+gkpK1G4eXmvXswmcE1hTWBWYUzlraYw1/yZp6YuDY77YtvbN0dmDA=="], - - "shebang-regex": ["shebang-regex@3.0.0", "", {}, "sha512-7++dFhtcx3353uBaq8DDR4NuxBetBzC7ZQOhmTQInHEd6bSrXdiEyzCvG07Z44UYdLShWUyXt5M/yhz8ekcb1A=="], - "signal-exit": ["signal-exit@3.0.7", "", {}, "sha512-wnD2ZE+l+SPC/uoS0vXeE9L1+0wuaMqKlfz9AMUo38JsyLSBWSFcHR1Rri62LZc12vLr1gb3jl7iwQhgwpAbGQ=="], "sort-scripts": ["sort-scripts@1.0.1", "", {}, "sha512-58eys3wXg05rI51Gg/90Uvc0id0aboGLSzHm4nFvuD0MofSg/y8cyJ7ZqYuZ1eyj6AA8XwFTGaXA+6tApsMv4w=="], @@ -467,7 +443,7 @@ "wcwidth": ["wcwidth@1.0.1", "", { "dependencies": { "defaults": "^1.0.3" } }, "sha512-XHPEwS0q6TaxcvG85+8EYkbiCux2XtWG2mkc47Ng2A77BQu9+DqIOJldST4HgPkuea7dvKSj5VgX3P1d4rW8Tg=="], - "which": ["which@2.0.2", "", { "dependencies": { "isexe": "^2.0.0" }, "bin": { "node-which": "./bin/node-which" } }, "sha512-BLI3Tl1TW3Pvl70l3yq3Y64i+awpwXqsGBYWkkqMtnbXgrMD+yj7rhW0kuEDxzJaYXGjEW5ogapKNMEKNMjibA=="], + "which": ["which@1.3.1", "", { "dependencies": { "isexe": "^2.0.0" }, "bin": { "which": "./bin/which" } }, "sha512-HxJdYWq1MTIQbJ3nw0cqssHoTNU267KlrDuGZ1WYlxDStUtKUhOaJmh112/TZmHxxUfuJqPXSOm7tDyas0OSIQ=="], "word-wrap": ["word-wrap@1.2.5", "", {}, "sha512-BN22B5eaMMI9UMtjrGd5g5eCYPpCPDUy0FJXbYsaT5zYxjFOckS53SQDE3pWkVoWpHXVb3BrYcEN4Twa55B5cA=="], @@ -489,8 +465,6 @@ "global-prefix/ini": ["ini@1.3.8", "", {}, "sha512-JV/yugV2uzW5iMRSiZAyDtQd+nxtUnjeLt0acNdw98kKLrvuRVyB80tsREOE7yvGVgalhZ6RNXCmEHkUKBKxew=="], - "global-prefix/which": ["which@1.3.1", "", { "dependencies": { "isexe": "^2.0.0" }, "bin": { "which": "./bin/which" } }, "sha512-HxJdYWq1MTIQbJ3nw0cqssHoTNU267KlrDuGZ1WYlxDStUtKUhOaJmh112/TZmHxxUfuJqPXSOm7tDyas0OSIQ=="], - "import-fresh/resolve-from": ["resolve-from@4.0.0", "", {}, "sha512-pb/MYmXstAkysRFx8piNI1tGFNQIFA3vkE3Gq4EuA1dF6gHp/+vgZqsCGJapvy8N3Q+4o7FwvquPJcnZ7RYy4g=="], "inquirer/chalk": ["chalk@4.1.2", "", { "dependencies": { "ansi-styles": "^4.1.0", "supports-color": "^7.1.0" } }, "sha512-oKnbhFyRIXpUuez8iBMmyEa4nbj4IOQyuhc/wy9kY7/WVPcwIO9VA668Pu8RkO7+0G76SLROeyw9CpQ061i4mA=="], diff --git a/package.json b/package.json index ca659898c..51a173507 100644 --- a/package.json +++ b/package.json @@ -73,8 +73,6 @@ "@biomejs/biome": "2.5.12", "@commitlint/cli": "21.2.2", "@commitlint/config-conventional": "21.2.2", - "@modelcontextprotocol/client": "^2.0.0", - "@modelcontextprotocol/server": "^2.0.0", "@types/bun": "^1.4.1", "@webref/css": "^8.7.4", "commitlint": "21.2.2", diff --git a/skills/behavioral-tools/SKILL.md b/skills/behavioral-tools/SKILL.md index d44ebc330..501f7bfd0 100644 --- a/skills/behavioral-tools/SKILL.md +++ b/skills/behavioral-tools/SKILL.md @@ -1,6 +1,6 @@ --- name: behavioral-tools -description: Remote MCP operations for the behavioral agent via the mcp faculty — the mcp_request/mcp_request_result/mcp_cancel event wire (seven ops, typed authorization_required results, the auth replay spine) plus the auth/broker binding rules. The CLI tool fleet is retired: skills and plugins run through the skill-conventions skill (threads + recipes + store), git and raw shell belong to the shell faculty, HTML validation belongs to the controller floors + the classifier story , TypeScript LSP is a future faculty (TS 7.1 stable API). +description: Remote MCP operations for the behavioral agent via the remote-mcp thread pack — the MCP layering over the shell faculty's generic `rpc` op (the 2026-07-28 stateless era: stamped `_meta` envelope, discovery, tools/call, the MRTR elicitation loop, bounded retry), with auth via the credential seam. The CLI tool fleet is retired: skills and plugins run through the skill-conventions skill (threads + recipes + store), git and raw shell belong to the shell faculty, HTML validation belongs to the controller floors + the classifier story , TypeScript LSP is a future faculty (TS 7.1 stable API). license: ISC compatibility: Requires bun and the behavioral CLI allowed-tools: Bash @@ -12,12 +12,11 @@ Reference for the behavioral agent's compiled operator surfaces. As of the ICL conversion, **every compiled surface is a faculty or the shell** — the CLI tool fleet is retired: -- **Remote MCP** is the mcp faculty: requests ride the behavioral - event wire (`mcp_request` / `mcp_request_result` / `mcp_cancel`), seven - ops, typed `authorization_required` results with request echo, auth bound - at the faculty's module scope (broker env-data + keychain floor — per-call - credentials are retired). See - [references/mcp-client.md](references/mcp-client.md). +- **Remote MCP** is the remote-mcp thread pack over the shell faculty's + generic `rpc` op (the 2026-07-28 stateless era: stamped `_meta` envelope, + discovery, tools/call, the MRTR elicitation loop, bounded retry; auth via + the credential seam). See + [references/remote-mcp.md](references/remote-mcp.md). - **Skills and plugins** (discovery, reading, frontmatter validation, link extraction/validation) run through threads + the shell faculty (`bun run -`) + the store — taught by the **skill-conventions** skill. @@ -29,5 +28,6 @@ the CLI tool fleet is retired: ## Module references -- [mcp-client](references/mcp-client.md) — the mcp faculty: wire, - ops, typed results, auth binding, the replay spine, composing. +- [remote-mcp](references/remote-mcp.md) — the remote-mcp thread pack: + the rpc op layering, trusted response shapes, the ctx join lane, + MRTR, composing. diff --git a/skills/behavioral-tools/references/mcp-client.md b/skills/behavioral-tools/references/mcp-client.md deleted file mode 100644 index ff5409ea6..000000000 --- a/skills/behavioral-tools/references/mcp-client.md +++ /dev/null @@ -1,66 +0,0 @@ -# mcp-client — the remote MCP faculty - -Remote MCP server operations are a spawned **faculty**, not CLI fleet tools: -`src/faculties/mcp/faculty.ts` (a spawned Bun process) holds the connections, -and the engine speaks to it over the behavioral event wire. The -`src/faculties/mcp/threads.ts` thread spine orchestrates cross-turn auth -replay. - -## The wire - -| Event | Detail | Direction | -|-------|--------|-----------| -| `mcp_request` | `{ id, op, input }` | program → faculty | -| `mcp_request_result` | `{ id, result }` | faculty → program | -| `mcp_cancel` | `{ id }` | program → faculty | - -The seven ops (`detail.op`): `discover`, `list-tools`, `call-tool`, -`list-prompts`, `get-prompt`, `list-resources`, `read-resource`. Each op -input carries the server `url` plus the op's own fields (`tool`/`args` for -call-tool, `name` for get-prompt, `uri` for read-resource) and an optional -`timeoutMs` wall-clock deadline for the whole call. - -## The result envelope - -`detail` is the uniform faculty result envelope — errors-as-data, never a -throw (`postResult` in `src/faculties/mcp/faculty.ts`): - -```json -{ "id": "…", "ok": true, "result": { "started": …, "output": … } } -{ "id": "…", "ok": false, "error": { "code": "…", "started": … } } -``` - -- `ok: true` — `result.output` carries the remote MCP data (loose; consumers - gate with their own `detailSchema`). -- `error.code = "authorization_required"` — the call hit a 401; `message` - holds the reason and `request` echoes `{ op, input }` (the replay spine's - capture payload). -- `error.code = "timeout" / "canceled"` — the two stop doors: the input - `timeoutMs` (default 30s) or an `mcp_cancel` mid-flight. -- `error.code = "error"` — invalid op input (the message names the AJV - errors) or a failed call/connection. - -## Auth - -Per-call input credentials are **retired** — the wire carries the server -URL only. Auth binds at the faculty's module scope: broker env-data -(`MCP_BROKER_URL` + `MCP_BROKER_BOOT_SECRET`, seeded by the spawning host) -with the OS-keychain floor beneath it. Neither yields a token → the call -goes unauthenticated → the server's 401 → typed `authorization_required`. - -## The auth replay spine (threads) - -An `authorization_required` result is captured in the store (`mcp-calls`, -keyed by call id, value = the echoed request), surfaced to the host as -`mcp_authorization_required { id, reason }` (the shell's "authorize X" -prompt), and after the host re-enters `mcp_authorization_granted { id }`, -the captured request is replayed and the capture deleted. Successful calls -never touch the store. - -## Composing - -Threads request `mcp_request` events like any other faculty; the composition -spawns it by default (`['bun', 'run', 'mcp/faculty.ts']` over stdio -lines — same wire, one JSON event per line). Schema-reflect the op inputs via -`src/faculties/mcp/types.ts` (`MCP_*_OP_INPUT_SCHEMA`) when model-facing -context is needed. diff --git a/skills/behavioral-tools/references/remote-mcp.md b/skills/behavioral-tools/references/remote-mcp.md new file mode 100644 index 000000000..3dbc3c8ce --- /dev/null +++ b/skills/behavioral-tools/references/remote-mcp.md @@ -0,0 +1,68 @@ +# remote-mcp — the remote MCP thread pack + +Remote MCP is no longer a faculty. It is a **thread pack** over the shell +faculty's generic `rpc` op (`src/faculties/shell/remote-mcp.threads.ts`): +the op is transport-shaped (one stateless HTTP JSON-RPC POST per call), and +this pack is where "MCP" lives — the protocol envelope, discovery, tool +execution, the multi-round-trip elicitation loop, and bounded retry. + +The pack speaks the MCP 2026-07-28 stateless era: no handshake, no session +id — every request carries the protocol stamp in-band (`_meta` envelope: +`io.modelcontextprotocol/protocolVersion` + `clientInfo` + +`clientCapabilities`, plus the `MCP-Protocol-Version` header), and +server→client interactions arrive **in-band** as `input_required` results +(no server→client JSON-RPC channel on this revision). + +## The events + +| Event | Detail | Direction | +|-------|--------|-----------| +| `remote_mcp_discover` | `{ id, input: { url } }` | program → pack (host/config ingress) | +| `remote_mcp_discovered` | `{ id, ok, input?: { url, tools } }` or `{ id, ok: false, error }` | pack → program | +| `remote_mcp_call` | `{ id, input: { url, tool, args } }` | program → pack | +| `remote_mcp_call_result` | `{ id, ok: true, result }` or `{ id, ok: false, error }` | pack → program | +| `remote_mcp_elicitation` | `{ id, input: { url, tool, args, round, inputRequests, requestState } }` | pack → program (host surfaces) | +| `remote_mcp_elicitation_response` | the elicitation detail echoed + `inputResponses` | host → pack (ingress) | + +All of the pack's remote work rides `shell_request` events (`op: 'rpc'`) +and comes back on `shell_request_result`; registration rides +`store_request` (the `remote-mcp` collection, keyed by server URL — the +tools sit alongside the skills/plugins tenants in the shell registry). + +## The join lane: `ctx` + +Cross-event state rides the shell wire's `detail.ctx` — the out-of-band +lane beside `input` (the you.com MCP `_meta` pattern: host-supplied, +round-tripped verbatim, never a model-facing field). The pack stamps +`ctx.echo { source, url, leg, round, attempt }` on every op it issues; the +shell faculty echoes `ctx` on the result; the pack's transforms join on it. +Auth rides the credential seam (`shell/rpc-auth.threads.ts`): a remote 401 +challenge maps to the typed `credential_required`, and the seam vends +(broker first, keychain floor second — issuer-bound via `ctx.issuer`) and +replays the call with the vended bearer. + +## Trusted response shapes + +The pack AJV-validates ONLY the four responses it acts on +(`server/discover`, `tools/list`, `tools/call`, `InputRequiredResult`) — +exported from `remote-mcp.threads.ts` as `REMOTE_MCP_*_SCHEMA`. A response +failing its trusted shape silently no-matches the acting transform (the +result stays visible as an unmatched event in the frontier traces) — +fail-closed, not silently-wrong. + +## Multi-round-trip (MRTR) + +A `tools/call` result carrying the reserved `inputRequests` / `requestState` +members (at-least-one) is an `input_required` answer: the pack surfaces +`remote_mcp_elicitation`, the host answers with +`remote_mcp_elicitation_response` (echo + bare `inputResponses`), and the +pack retries `tools/call` with the answers plus a byte-exact `requestState` +echo, on a fresh request id, up to `REMOTE_MCP_MAX_ROUNDS` — the cap +exhausts as a typed `round_cap` error on `remote_mcp_call_result`. + +## Composition + +`bProgram` mounts the pack when **shell + security + store** are all on +(executor + vending leg + registry). The retired `mcp` faculty's replay +spine is gone; its capture-on-auth-required pattern lives on in the +credential seam. diff --git a/src/cli/b-program.ts b/src/cli/b-program.ts index f36cca813..fd47eb562 100644 --- a/src/cli/b-program.ts +++ b/src/cli/b-program.ts @@ -4,9 +4,6 @@ import type { BPEvent, JsonObject, SelectionTrace, Thread, Trace } from '../beha import { FACULTY_MESSAGE_KINDS } from '../faculties/faculties.constants.ts' import { eventGuardEntries, facultiesThreads, guardThreads } from '../faculties/faculties.threads.ts' import { - McpCancelEventSchema, - McpRequestEventSchema, - McpRequestResultEventSchema, SecurityCancelEventSchema, SecurityRequestEventSchema, SecurityRequestResultEventSchema, @@ -18,8 +15,8 @@ import { validateFrontierRequestEvent, } from '../faculties/faculties.types.ts' import { handleFrontierMessage } from '../faculties/frontier/faculty.ts' -import { mcpThreads } from '../faculties/mcp/threads.ts' import { bindEmit } from '../faculties/process-lane.ts' +import { remoteMcpThreads } from '../faculties/shell/remote-mcp.threads.ts' import { rpcAuthThreads } from '../faculties/shell/rpc-auth.threads.ts' import { shellThreads } from '../faculties/shell/threads.ts' import { useFaculty } from '../faculties/use-faculty.ts' @@ -32,7 +29,7 @@ import type { Faculty } from '../faculties.ts' * what makes this safe on the main thread). Frontier is the in-process embed: * its analysis dispatch is imported and driven directly, its emit lane bound * to the composition's reenter. The capability faculties — shell, store, and - * mcp as default processes; system One/Two as endpoint-carrying overrides — + * security as default processes; system One/Two as endpoint-carrying overrides — * are Bun.spawn PROCESSES speaking the unchanged wire over stdio lines — * per-space isolatable, abort-able, head-of-line-free — wired by the useFaculty * primitive. @@ -42,7 +39,7 @@ import type { Faculty } from '../faculties.ts' * This was the engine transport's trailing step; it is the composition's * now. * - * `faculties` is the allow-list (unset = shell/store/mcp on); `shell` and + * `faculties` is the allow-list (unset = shell/store/security on); `shell` and * `store` are the two default-faculty instance overrides (sandboxed shell, * durable store), and `systemOne`/`systemTwo` are endpoint-carrying overrides * with no default — all pre-curried useFaculty returns for host-constructed @@ -117,7 +114,7 @@ export const bProgram = ({ */ systemTwo?: ReturnType }) => { - const enabled = new Set(faculties === undefined ? ['shell', 'store', 'mcp', 'security'] : faculties) + const enabled = new Set(faculties === undefined ? ['shell', 'store', 'security'] : faculties) const has = (faculty: Faculty): boolean => enabled.has(faculty) // ── The engine, in-process ──────────────────────────────────────────────── @@ -185,16 +182,6 @@ export const bProgram = ({ })(facultyAddThreads) : storeOverride(facultyAddThreads) - const mcp = useFaculty({ - command: ['bun', 'run', 'mcp/faculty.ts'], - name: 'mcp', - // The spine requires mcp + store. - threads: has('mcp') && has('store') ? mcpThreads : [], - requestSchema: McpRequestEventSchema, - cancelSchema: McpCancelEventSchema, - resultSchema: McpRequestResultEventSchema, - })(facultyAddThreads) - // The security faculty: the cross-cutting credential/policy faculty — its // vending leg serves shell (remote rpc), system-two endpoints, ATProto, // and any future remote faculty. No threads of its own yet (the skeleton @@ -255,12 +242,6 @@ export const bProgram = ({ gate: (event: BPEvent): boolean => store.invalidEventGate(event), }) } - if (has('mcp')) { - route([FACULTY_MESSAGE_KINDS.mcp_request, FACULTY_MESSAGE_KINDS.mcp_cancel], { - send: (event: BPEvent): void => mcp.send(event), - gate: (event: BPEvent): boolean => mcp.invalidEventGate(event), - }) - } if (has('security')) { route([FACULTY_MESSAGE_KINDS.credential_request, FACULTY_MESSAGE_KINDS.credential_cancel], { send: (event: BPEvent): void => security.send(event), @@ -271,6 +252,9 @@ export const bProgram = ({ // The rpc auth seam: the vend-and-replay spine requires the op (shell) and // the vending leg (security) — the pack mounts only when both are on. if (has('shell') && has('security')) facultyAddThreads(rpcAuthThreads) + // The remote-mcp pack: the MCP layering over the rpc op — requires the + // executor (shell), the vending leg (security), and the registry (store). + if (has('shell') && has('security') && has('store')) facultyAddThreads(remoteMcpThreads) // ── The engine pump: traces out, gated events to their faculty lanes ───── @@ -315,7 +299,6 @@ export const bProgram = ({ bindEmit(null) shell.terminate() store.terminate() - mcp.terminate() security.terminate() systemOne?.terminate() systemTwo?.terminate() diff --git a/src/cli/tests/b-program.spec.ts b/src/cli/tests/b-program.spec.ts index 1872a6ea4..44f693792 100644 --- a/src/cli/tests/b-program.spec.ts +++ b/src/cli/tests/b-program.spec.ts @@ -10,7 +10,11 @@ import { ShellRequestEventSchema, ShellRequestResultEventSchema, } from '../../faculties/faculties.types.ts' -import { startMcpServer } from '../../faculties/mcp/tests/fixtures/mcp-server-fixture.ts' +import { + REMOTE_MCP_EVENT_TYPES, + REMOTE_MCP_PROTOCOL_VERSION, + REMOTE_MCP_STORE_COLLECTION, +} from '../../faculties/shell/remote-mcp.threads.ts' import { useSystemOne } from '../../faculties/system-one/config.ts' import { startDecisionsServer } from '../../faculties/system-one/tests/fixtures/decisions-server.ts' import { useSystemTwo } from '../../faculties/system-two/config.ts' @@ -21,7 +25,7 @@ import { bProgram } from '../b-program.ts' /** * bProgram — the runtime composition — through its REAL surface: the * hook spawns every faculty itself (engine + frontier router-owned, always - * on; mcp/shell/responses/store default-on, pruned by the `faculties` + * on; shell/responses/store default-on, pruned by the `faculties` * allow-list). The host attaches ingress and observation through the * returned handle — `runtime.trigger(...)` and `runtime.useTrace(...)`. * `shell` is the one instance-level override: the pre-curried useFaculty @@ -36,7 +40,7 @@ import { bProgram } from '../b-program.ts' * * The default threads are faculty-shipped: the shell threads * (shell/threads.ts — skill/plugin scans + links) mounts with shell+store - * on; the mcp spine (mcp.threads.ts) mounts with store+mcp on. + * on; the remote-mcp pack mounts with shell+security+store on. */ const selectionsOf = (traces: Trace[]): SelectionTrace[] => @@ -129,22 +133,6 @@ describe('bProgram — the runtime composition', () => { } }) - test('the mcp spine ships with the mcp faculty: granted ingress fires the store get', async () => { - const { runtime, traces } = startRuntime() - try { - // No capture exists, so the get returns nothing and the spine waits — - // but the GET itself is the observable: the spine is mounted. - runtime.trigger({ type: 'mcp_authorization_granted', detail: { id: 'none' } }) - await waitForTraces(traces, (s) => storeRequest(s, 'get', 'mcp-calls') !== undefined) - const get = storeRequest(selectionsOf(traces), 'get', 'mcp-calls') - expect((get?.selected.detail as { input?: { collection?: string } } | undefined)?.input?.collection).toBe( - 'mcp-calls', - ) - } finally { - runtime.terminate() - } - }) - test('the faculties allow-list prunes faculties: without shell, no route — a triggered shell_request is never answered', async () => { const { runtime, traces } = startRuntime({ faculties: ['store'] }) try { @@ -174,31 +162,6 @@ describe('bProgram — the runtime composition', () => { } }) - test('the mcp faculty responds through the composition (real loopback server)', async () => { - const { runtime, traces } = startRuntime() - const server = await startMcpServer() - const loopback = Bun.serve({ port: 0, fetch: (req) => server.fetch(req.url, req) }) - try { - // Drive the mcp faculty via the spine's replay path: granted → get → - // (empty capture) → nothing. Instead, assert faculty presence through - // a direct trigger-shaped caller: the composition mounts the spine, - // and the spine's auth-retry fires the get — already covered above. - // Here: the honest direct check — the fixture loopback round-trip is - // covered by the mcp worker spec; composition-level assertion is the - // spine mount (previous test). This test pins: the composition does - // not crash when mcp is default-on with a live server present. - runtime.trigger({ type: 'mcp_authorization_required_probe', detail: {} }) - await Bun.sleep(200) - const types = new Set(selectionsOf(traces).map((t) => t.selected.type)) - expect(types.has(FACULTY_MESSAGE_KINDS.mcp_request)).toBe(false) // no capture → no replay - expect(true).toBe(true) - } finally { - runtime.terminate() - loopback.stop(true) - await server.close() - } - }) - test('shell overrides the default faculty — a host-constructed shell takes the route', async () => { const hostShell = useFaculty({ command: ['bun', 'run', 'tests/fixtures/probe.proc.ts'], @@ -438,6 +401,66 @@ describe('bProgram — the runtime composition', () => { } }) + test('the remote-mcp pack ships with shell+security+store: discovery registers the tools', async () => { + // A plain JSON-RPC endpoint speaking server/discover + tools/list — the + // 2026-07-28 stateless era needs no handshake. + const rpc = Bun.serve({ + port: 0, + fetch: async (request) => { + const body = (await request.json()) as { id?: unknown; method?: string } + const method = body.method + if (method === 'server/discover') + return Response.json({ + jsonrpc: '2.0', + id: body.id, + result: { supportedVersions: ['2026-07-28'], capabilities: { tools: {} } }, + }) + if (method === 'tools/list') + return Response.json({ + jsonrpc: '2.0', + id: body.id, + result: { tools: [{ name: 'echo', description: 'echoes' }] }, + }) + return Response.json({ jsonrpc: '2.0', id: body.id, error: { code: -32601, message: 'nope' } }) + }, + }) + const { runtime, traces } = startRuntime() + try { + runtime.trigger({ + type: REMOTE_MCP_EVENT_TYPES.discover, + detail: { id: 'r1', input: { url: `http://localhost:${rpc.port}/mcp` } }, + }) + // The pack drives the generic rpc op: server/discover → tools/list → + // the store registry put (alongside the skills/plugins tenants). + await waitForTraces(traces, (s) => storeRequest(s, 'put', REMOTE_MCP_STORE_COLLECTION) !== undefined) + const put = storeRequest(selectionsOf(traces), 'put', REMOTE_MCP_STORE_COLLECTION) + const input = (put?.selected.detail as { input?: { key?: string; value?: { tools?: Array<{ name?: string }> } } }) + ?.input + expect(input?.key).toContain('localhost') + expect(input?.value?.tools?.[0]?.name).toBe('echo') + // The outcome surfaces to the host. + await waitForTraces(traces, (s) => + selectionsOf(s).some((t) => t.selected.type === REMOTE_MCP_EVENT_TYPES.discovered), + ) + const surfaced = selectionsOf(traces).find((t) => t.selected.type === REMOTE_MCP_EVENT_TYPES.discovered) + const surfacedDetail = surfaced?.selected.detail as { id?: string } | undefined + expect(surfacedDetail?.id).toBe('r1') + // The pack's issued ops carry the protocol stamp (observed on the result lane). + expect( + selectionsOf(traces).some( + (t) => + t.selected.type === FACULTY_MESSAGE_KINDS.shell_request && + (t.selected.detail as { input?: { headers?: Record } }).input?.headers?.[ + 'MCP-Protocol-Version' + ] === REMOTE_MCP_PROTOCOL_VERSION, + ), + ).toBe(true) + } finally { + runtime.terminate() + rpc.stop(true) + } + }) + test('the credential seam ships with shell+security: an auth rpc op vends, then replays with the bearer', async () => { // The JSON-RPC endpoint requires a bearer; the broker vends one. const seenAuth: Array = [] diff --git a/src/faculties.ts b/src/faculties.ts index 188c115ab..c8743b227 100644 --- a/src/faculties.ts +++ b/src/faculties.ts @@ -13,12 +13,11 @@ */ /** The selectable capability faculties (the `bProgram` allow-list). */ -export type Faculty = 'shell' | 'store' | 'mcp' | 'security' +export type Faculty = 'shell' | 'store' | 'security' export * from './faculties/faculties.types.ts' -export * from './faculties/mcp/threads.ts' -export * from './faculties/mcp/types.ts' export * from './faculties/security/types.ts' +export * from './faculties/shell/remote-mcp.threads.ts' export * from './faculties/shell/rpc-auth.threads.ts' export * from './faculties/shell/threads.ts' export * from './faculties/shell/types.ts' diff --git a/src/faculties/faculties.constants.ts b/src/faculties/faculties.constants.ts index ca5f0e009..f8629c40f 100644 --- a/src/faculties/faculties.constants.ts +++ b/src/faculties/faculties.constants.ts @@ -15,9 +15,6 @@ export const FACULTY_MESSAGE_KINDS = keyMirror( 'shell_request', 'shell_request_result', 'shell_cancel', - 'mcp_request', - 'mcp_request_result', - 'mcp_cancel', 'credential_request', 'credential_result', 'credential_cancel', diff --git a/src/faculties/faculties.types.ts b/src/faculties/faculties.types.ts index 8dc40e916..666bbaa90 100644 --- a/src/faculties/faculties.types.ts +++ b/src/faculties/faculties.types.ts @@ -61,7 +61,8 @@ export type SystemOneCancelEvent = { export type ShellRequestEvent = { type: typeof FACULTY_MESSAGE_KINDS.shell_request /** `label` is an optional trace annotation (logical names like 'skill-scan') — no routing weight. */ - detail: { id: string; label?: string; input: JsonObject } + /** `ctx` is the optional out-of-band join lane (the you.com MCP `_meta` pattern): orchestration state riding beside `input`, echoed verbatim on the result — never a model-facing field. */ + detail: { id: string; label?: string; ctx?: JsonObject; input: JsonObject } space?: string } @@ -82,14 +83,16 @@ export type ShellCancelEvent = { * two-branch shape (modified-B envelope, ruled 2026-09-21): the `ok` * discriminant sits at detail level beside the correlation id; `result` and * `error` are XOR branches (oneOf on the ok const). Faculty statuses ride as - * `error.code` (mcp's typed `authorization_required` included — first-class - * preserved, its request echo rides inside `error`); success payloads ride - * `result` verbatim. Uniform gate across every faculty: `select($d.ok)`. + * `error.code` (the shell rpc's typed `credential_required` included — + * first-class preserved, its request echo rides inside `error`); success + * payloads ride `result` verbatim. Uniform gate across every faculty: `select($d.ok)`. */ export type WorkerResultOk = { id: string ok: true result: JsonObject + /** The request's `ctx`, echoed verbatim by faculties that pass it through (shell). */ + ctx?: JsonObject } export type WorkerResultError = { @@ -97,6 +100,8 @@ export type WorkerResultError = { ok: false /** The faculty failure payload — code (the faculty status enum), message, and any diagnostics. */ error: { code: string; message?: string } & JsonObject + /** The request's `ctx`, echoed verbatim by faculties that pass it through (shell). */ + ctx?: JsonObject } /** The `detail` of every `*_result` event — one shape across all five faculties. */ @@ -141,38 +146,6 @@ export type StoreRequestResultEvent = { space?: string } -/** MCP operations — its own worker faculty, like frontier and store. */ -export type McpOp = - | 'discover' - | 'list-tools' - | 'call-tool' - | 'list-prompts' - | 'get-prompt' - | 'list-resources' - | 'read-resource' - -export type McpRequestEvent = { - type: typeof FACULTY_MESSAGE_KINDS.mcp_request - /** `op` selects the MCP client operation; the backing schema lives in `src/faculties/mcp/types.ts`. */ - detail: { id: string; op: McpOp; input: JsonObject } - space?: string -} - -export type McpRequestResultEvent = { - type: typeof FACULTY_MESSAGE_KINDS.mcp_request_result - detail: WorkerResultDetail - space?: string -} - -// Remote MCP calls can hang indefinitely (third-party servers) — the async -// faculties keep their cancels (shell, response, mcp; frontier/store ops are -// short-lived and have none). -export type McpCancelEvent = { - type: typeof FACULTY_MESSAGE_KINDS.mcp_cancel - detail: { id: string } - space?: string -} - /** Security operations — credential vending for remote servers (broker first, keychain floor second). */ export type SecurityRequestEvent = { type: typeof FACULTY_MESSAGE_KINDS.credential_request @@ -188,7 +161,7 @@ export type SecurityRequestResultEvent = { } // A vend is a quick broker/keychain read, but a down broker can hang — the -// async faculties keep their cancels (shell, response, mcp, security). +// async faculties keep their cancels (shell, response, security). export type SecurityCancelEvent = { type: typeof FACULTY_MESSAGE_KINDS.credential_cancel detail: { id: string } @@ -206,9 +179,6 @@ export type WorkerEvent = | ShellRequestEvent | ShellRequestResultEvent | ShellCancelEvent - | McpRequestEvent - | McpRequestResultEvent - | McpCancelEvent | SecurityRequestEvent | SecurityRequestResultEvent | SecurityCancelEvent @@ -228,6 +198,9 @@ const workerResultOkBranch = { id: { type: 'string', minLength: 1 }, ok: { type: 'boolean', const: true }, result: jsonObjectSchema, + // The out-of-band join lane — its strict shape is the requesting side's + // (the echo rides beside `ok`, the you.com MCP `_meta` pattern). + ctx: { type: 'object', required: [], additionalProperties: true, nullable: true }, }, required: ['id', 'ok', 'result'], additionalProperties: false, @@ -248,6 +221,7 @@ const workerResultErrorBranch = { // Faculty diagnostics ride along (request echoes, exit codes, stderr…). additionalProperties: true, }, + ctx: { type: 'object', required: [], additionalProperties: true, nullable: true }, }, required: ['id', 'ok', 'error'], additionalProperties: false, @@ -343,6 +317,8 @@ export const ShellRequestEventSchema: JSONSchemaType = { properties: { id: { type: 'string', minLength: 1 }, label: { type: 'string', nullable: true }, + // The out-of-band join lane — strict shape is the requesting side's. + ctx: { type: 'object', required: [], additionalProperties: true, nullable: true }, input: jsonObjectSchema, }, required: ['id', 'input'], @@ -372,55 +348,6 @@ export const ShellCancelEventSchema: JSONSchemaType = { additionalProperties: false, } -export const McpRequestEventSchema: JSONSchemaType = { - type: 'object', - properties: { - type: { type: 'string', const: FACULTY_MESSAGE_KINDS.mcp_request }, - detail: { - type: 'object', - properties: { - id: { type: 'string', minLength: 1 }, - op: { - type: 'string', - enum: [ - 'discover', - 'list-tools', - 'call-tool', - 'list-prompts', - 'get-prompt', - 'list-resources', - 'read-resource', - ], - }, - input: jsonObjectSchema, - }, - required: ['id', 'op', 'input'], - additionalProperties: false, - }, - space: { type: 'string', nullable: true }, - }, - required: ['type', 'detail'], - additionalProperties: false, -} - -export const McpRequestResultEventSchema = resultEventSchema(FACULTY_MESSAGE_KINDS.mcp_request_result) - -export const McpCancelEventSchema: JSONSchemaType = { - type: 'object', - properties: { - type: { type: 'string', const: FACULTY_MESSAGE_KINDS.mcp_cancel }, - detail: { - type: 'object', - properties: { id: { type: 'string', minLength: 1 } }, - required: ['id'], - additionalProperties: false, - }, - space: { type: 'string', nullable: true }, - }, - required: ['type', 'detail'], - additionalProperties: false, -} - export const SecurityRequestEventSchema: JSONSchemaType = { type: 'object', properties: { @@ -486,9 +413,6 @@ export const validateSystemOneCancelEvent = ajv.compile(SystemOneCancelEventSche export const validateShellRequestEvent = ajv.compile(ShellRequestEventSchema) export const validateShellRequestResultEvent = ajv.compile(ShellRequestResultEventSchema) export const validateShellCancelEvent = ajv.compile(ShellCancelEventSchema) -export const validateMcpRequestEvent = ajv.compile(McpRequestEventSchema) -export const validateMcpRequestResultEvent = ajv.compile(McpRequestResultEventSchema) -export const validateMcpCancelEvent = ajv.compile(McpCancelEventSchema) export const validateSecurityRequestEvent = ajv.compile(SecurityRequestEventSchema) export const validateSecurityRequestResultEvent = ajv.compile(SecurityRequestResultEventSchema) export const validateSecurityCancelEvent = ajv.compile(SecurityCancelEventSchema) diff --git a/src/faculties/mcp/faculty.ts b/src/faculties/mcp/faculty.ts deleted file mode 100644 index a70b0e3cc..000000000 --- a/src/faculties/mcp/faculty.ts +++ /dev/null @@ -1,369 +0,0 @@ -/** - * MCP client worker — executes one remote MCP operation per `mcp_request` - * event against its own per-call connection and returns a single terminal - * `mcp_request_result` event. - * - * @remarks - * Spawned by URL (never imported) and speaks the behavioral event wire: - * `mcp_request` / `mcp_cancel` in, `mcp_request_result` out, with any request - * `space` echoed on the result. `detail.input` is validated against the op - * input boundary (`mcp/types.ts`); per-call input credentials are - * RETIRED per the worker-conversion rulings — auth binds at this module's - * scope from env-data, never from the wire. - * - * Auth binding (the injection law one level down — functions cannot ride - * postMessage): - * - broker env-data (`MCP_BROKER_URL` + `MCP_BROKER_BOOT_SECRET`, seeded by - * the spawning composition root) → the provider fetches the broker's - * `request_access_token` endpoint. MINIMAL: the broker slice has not - * landed; its request contract is thin (POST + boot-secret bearer) and - * fail-closed on any failure. Pin it when the broker exists. - * - No broker → the keychain floor: tokens written by prior - * BunKeychainOAuthProvider flows are read per server URL (the - * reconnect-per-turn posture — credentials persist in the keyring, the - * connection does not). - * - Neither yields a token → the call goes unauthenticated, the server's 401 - * surfaces as typed `authorization_required`, and the result echoes the - * originating request — the replay spine's capture payload. - * - * One connection per call (cold-per-turn: sessions die with the worker); - * the envelope deadline (`input.timeoutMs`, default 30s) and `mcp_cancel` - * are the two stop doors — remote MCP calls are the one faculty where - * "hangs indefinitely" is a real third-party failure mode. - * - * MINIMAL: a stop closes the in-flight client but cannot abort the SDK's - * pending fetch (no signal seam on StreamableHTTPClientTransport) — the - * abandoned op settles into a settled race and its late rejection is - * absorbed. Upgrade path: a transport signal when the SDK grows one. - * - * @packageDocumentation - */ - -import { - type AuthProvider, - Client, - StreamableHTTPClientTransport, - UnauthorizedError, -} from '@modelcontextprotocol/client' -import type { ValidateFunction } from 'ajv' -import { ajv, type JsonObject } from '../../behavioral/behavioral.types.ts' -import { FACULTY_MESSAGE_KINDS } from '../faculties.constants.ts' -import { - type McpCancelEvent, - type McpOp, - type McpRequestEvent, - validateMcpCancelEvent, - validateMcpRequestEvent, -} from '../faculties.types.ts' -import { emit, envData, wireInbound } from '../process-lane.ts' -import { BunKeychain, vendKeychainToken } from '../security/keychain-oauth-provider.ts' -import { MCP_BROKER_BOOT_SECRET_KEY, MCP_BROKER_URL_KEY, MCP_OP_INPUT_VALIDATORS } from './types.ts' - -// --------------------------------------------------------------------------- -// Constants -// --------------------------------------------------------------------------- - -const DEFAULT_TIMEOUT_MS = 30_000 - -const CLIENT_INFO = { name: 'behavioral', version: '0.0.0' } - -// --------------------------------------------------------------------------- -// Auth binding — module scope, from boundary-legal data only -// --------------------------------------------------------------------------- - -const brokerUrl = envData(MCP_BROKER_URL_KEY) as string | undefined -const brokerBootSecret = envData(MCP_BROKER_BOOT_SECRET_KEY) as string | undefined -const keychain = BunKeychain() - -/** Fetch an access token from the taskbar broker (env-data binding). Fail-closed. */ -const brokerToken = async (): Promise => { - if (typeof brokerUrl !== 'string' || typeof brokerBootSecret !== 'string') return undefined - try { - const response = await fetch(new URL('request_access_token', brokerUrl), { - method: 'POST', - headers: { authorization: `Bearer ${brokerBootSecret}` }, - }) - if (!response.ok) return undefined - const data = (await response.json()) as { token?: string } - return typeof data.token === 'string' && data.token !== '' ? data.token : undefined - } catch { - // A down broker must not break the floor — fall through to the keychain. - return undefined - } -} - -/** The keychain floor: read tokens persisted by prior grant flows (the security faculty's store). */ -const keychainToken = async (serverUrl: string): Promise => - vendKeychainToken({ serverUrl, keychain }) - -/** The per-server token: broker first, keychain floor second, absent last. */ -const getToken = async (serverUrl: string): Promise => - (await brokerToken()) ?? keychainToken(serverUrl) - -// --------------------------------------------------------------------------- -// In-flight execution — enough state to stop it by correlation id -// --------------------------------------------------------------------------- - -type StopReason = 'canceled' | 'timeout' - -type Execution = { - /** First stop wins, so a late cancel cannot relabel a timeout. */ - stopReason: StopReason | null - /** Set once the operation opens its client, so a stop can close it. */ - client: Client | undefined - /** Resolves the stop race when the first stop fires (unused when the op wins). */ - onStop: ((reason: StopReason) => void) | undefined -} - -/** Executions in flight, keyed by correlation id. */ -const active = new Map() - -/** Record why an execution stopped, signal the race, and close the client. First writer wins. */ -const stopExecution = ({ execution, reason }: { execution: Execution; reason: StopReason }): void => { - if (execution.stopReason !== null) return - execution.stopReason = reason - // Best-effort close: the pending op rejects into the settled race's catch. - void execution.client?.close().catch(() => undefined) - execution.onStop?.(reason) -} - -// --------------------------------------------------------------------------- -// Per-call session — one connection per op, owned by this execution -// --------------------------------------------------------------------------- - -/** A broker/keychain-bound provider for one server URL — wire credentials never. */ -const providerFor = (serverUrl: string): AuthProvider => ({ - token: () => getToken(serverUrl), -}) - -/** Run one op over its own connection, closing the client afterward. */ -const runOperation = async ({ - op, - input, - url, - execution, -}: { - op: McpOp - input: JsonObject - url: string - execution: Execution -}): Promise => { - const client = new Client(CLIENT_INFO) - execution.client = client - const transport = new StreamableHTTPClientTransport(new URL(url), { - authProvider: providerFor(url), - }) - await client.connect(transport) - try { - return await runOp({ op, input, client }) - } finally { - execution.client = undefined - try { - await client.close() - } catch { - // Best-effort — a close failure must not mask the operation result. - } - } -} - -const runOp = async ({ op, input, client }: { op: McpOp; input: JsonObject; client: Client }): Promise => { - switch (op) { - case 'call-tool': { - const tool = input as unknown as { tool: string; args: Record } - return (await client.callTool({ name: tool.tool, arguments: tool.args })) as unknown as JsonObject - } - case 'list-tools': - return { tools: (await client.listTools()).tools } as unknown as JsonObject - case 'list-prompts': - return { prompts: (await client.listPrompts()).prompts } as unknown as JsonObject - case 'get-prompt': { - const prompt = input as unknown as { name: string; args?: Record } - return { - messages: (await client.getPrompt({ name: prompt.name, arguments: prompt.args })).messages, - } as unknown as JsonObject - } - case 'list-resources': - return { resources: (await client.listResources()).resources } as unknown as JsonObject - case 'read-resource': { - const resource = input as unknown as { uri: string } - return { contents: (await client.readResource({ uri: resource.uri })).contents } as unknown as JsonObject - } - case 'discover': { - const [tools, prompts, resources] = await Promise.allSettled([ - client.listTools(), - client.listPrompts(), - client.listResources(), - ]) - return { - tools: tools.status === 'fulfilled' ? tools.value.tools : [], - prompts: prompts.status === 'fulfilled' ? prompts.value.prompts : [], - resources: resources.status === 'fulfilled' ? resources.value.resources : [], - } as unknown as JsonObject - } - } -} - -// --------------------------------------------------------------------------- -// Request execution — race the op against the first stop; no throw escapes -// --------------------------------------------------------------------------- - -type RunOutcome = { output: JsonObject } | { stop: StopReason } | { fail: unknown } - -const runRequest = async ({ - op, - input, - url, - timeoutMs, - execution, -}: { - op: McpOp - input: JsonObject - url: string - timeoutMs?: number - execution: Execution -}): Promise => { - // The stop promise is resolver-shaped: no polling, no interval to leak. - const stopped = new Promise((resolve) => { - execution.onStop = (reason) => resolve({ stop: reason }) - }) - const deadline = setTimeout(() => stopExecution({ execution, reason: 'timeout' }), timeoutMs ?? DEFAULT_TIMEOUT_MS) - - // The op settles as data (never rejects), so the race has no unhandled loser. - const operation = runOperation({ op, input, url, execution }) - .then((output) => ({ output }) as RunOutcome) - .catch((fail: unknown) => ({ fail }) as RunOutcome) - - try { - return await Promise.race([operation, stopped]) - } finally { - clearTimeout(deadline) - } -} - -// --------------------------------------------------------------------------- -// Result envelope -// --------------------------------------------------------------------------- - -const postResult = ({ - id, - payload, - error, - space, -}: { - id: string - payload?: JsonObject - error?: { code: string; message?: string } & JsonObject - space?: string -}): void => { - emit({ - type: FACULTY_MESSAGE_KINDS.mcp_request_result, - detail: (error === undefined - ? { id, ok: true, result: payload ?? {} } - : { id, ok: false, error: error as JsonObject }) as JsonObject & { id: string }, - ...(space === undefined ? {} : { space }), - }) -} - -/** The success interior — status rides along as plain diagnostics. */ -const successInterior = ({ started }: { started: number }): JsonObject => ({ - status: 'completed', - durationMs: Math.round(performance.now() - started), -}) - -/** The error interior — the terminal status rides as `code`. */ -const errorInterior = ({ code, started }: { code: string; started: number }): { code: string; durationMs: number } => ({ - code, - durationMs: Math.round(performance.now() - started), -}) - -const failMessage = (fail: unknown): string => (fail instanceof Error ? fail.message : String(fail)) - -// --------------------------------------------------------------------------- -// Worker message loop -// --------------------------------------------------------------------------- - -const handleInbound = async (message: unknown): Promise => { - if (validateMcpCancelEvent(message)) { - // Cast: the TS7/ajv compiler defect means the type guard does not narrow. - const cancel = message as McpCancelEvent - const execution = active.get(cancel.detail.id) - if (execution !== undefined) stopExecution({ execution, reason: 'canceled' }) - return - } - // Events failing the shared schema have no correlation id to report to and - // are dropped — the router only forwards schema-valid events, so this is - // defense in depth at the process boundary. - if (!validateMcpRequestEvent(message)) return - const event = message as McpRequestEvent - const { id, op, input } = event.detail - const started = performance.now() - - // Input that fails the boundary is error data, not a throw: the id is - // valid, so the caller learns why nothing ran. - const validateOpInput = MCP_OP_INPUT_VALIDATORS[op] as unknown as ValidateFunction - if (!validateOpInput(input)) { - postResult({ - id, - space: event.space, - error: { - ...errorInterior({ code: 'error', started }), - message: `invalid input: ${ajv.errorsText(validateOpInput.errors)}`, - }, - }) - return - } - - const execution: Execution = { stopReason: null, client: undefined, onStop: undefined } - active.set(id, execution) - try { - const { url, timeoutMs } = input as { url: string; timeoutMs?: number } - const outcome = await runRequest({ op, input, url, timeoutMs, execution }) - - if ('output' in outcome) { - postResult({ - id, - space: event.space, - payload: { ...successInterior({ started }), output: outcome.output }, - }) - return - } - if ('stop' in outcome) { - postResult({ - id, - space: event.space, - error: { ...errorInterior({ code: outcome.stop, started }) }, - }) - return - } - if (UnauthorizedError.isInstance(outcome.fail)) { - postResult({ - id, - space: event.space, - error: { - ...errorInterior({ code: 'authorization_required', started }), - message: failMessage(outcome.fail), - // The replay spine's capture payload — the store never saw the - // request; this echo is the only carrier. - request: { op, input }, - }, - }) - return - } - postResult({ - id, - space: event.space, - error: { ...errorInterior({ code: 'error', started }), message: failMessage(outcome.fail) }, - }) - } finally { - active.delete(id) - } -} - -// The wire is the behavioral event vocabulary, validated with the shared -// schemas — the trust boundary for anything crossing into this process. -if (import.meta.main) { - // Standalone (spawned process) — wire the stdio line lane. An in-process - // import (the composition's frontier embed) wires nothing: the host's - // stdin is never touched. - wireInbound((message) => handleInbound(message)) -} diff --git a/src/faculties/mcp/tests/faculty.spec.ts b/src/faculties/mcp/tests/faculty.spec.ts deleted file mode 100644 index 619577e8e..000000000 --- a/src/faculties/mcp/tests/faculty.spec.ts +++ /dev/null @@ -1,174 +0,0 @@ -import { describe, expect, test } from 'bun:test' -import type { JsonObject } from '../../../behavioral/behavioral.types.ts' -import { FACULTY_MESSAGE_KINDS } from '../../faculties.constants.ts' -import { spawnFaculty } from '../../tests/faculty-harness.ts' -import { startMcpServer } from './fixtures/mcp-server-fixture.ts' - -/** - * MCP client worker integration tests — exercised through the real worker - * boundary speaking the behavioral event wire: `mcp_request` events in - * (dispatched by `detail.op`), one `mcp_request_result` out, `mcp_cancel` - * for in-flight aborts. - * - * @remarks - * The worker is spawned by URL (never imported for logic) and talks to a - * REAL loopback MCP server: the fixture's in-process handler wrapped in - * `Bun.serve`, so every call crosses a genuine HTTP round-trip — the - * worker's global fetch is its own, so no fetch-swapping seam applies. - * - * Auth is fail-closed by construction: the worker's module-scope binding - * reads broker env-data (absent in tests) and falls back to the keychain - * floor (empty in tests) — unauthenticated calls against 401 servers must - * surface the typed `authorization_required` status, never a throw. - * - * @packageDocumentation - */ - -type WireResult = { - id: string - ok: boolean - result?: Record - error?: Record - space?: string -} - -/** Spawn the mcp faculty PROCESS and expose the same wire harness API. */ -const spawnMcpWorker = () => { - const worker = spawnFaculty({ - file: 'mcp/faculty.ts', - requestType: FACULTY_MESSAGE_KINDS.mcp_request, - resultType: FACULTY_MESSAGE_KINDS.mcp_request_result, - }) - const call = (id: string, op: string, input: unknown, space?: string): void => { - worker.call({ id, op, input } as JsonObject, space) - } - const cancel = (id: string): void => { - worker.post({ type: FACULTY_MESSAGE_KINDS.mcp_cancel, detail: { id } } as never) - } - const resultFor = async (id: string): Promise => { - const raw = await worker.resultFor(id) - return { ...raw.detail, id: raw.id, space: raw.space } as WireResult - } - return { call, cancel, resultFor, terminate: (): void => worker.terminate() } -} - -/** Wrap the fixture handler in a real loopback HTTP server. */ -const startLoopbackServer = async (): Promise<{ url: string; close: () => Promise }> => { - const { fetch, close } = await startMcpServer() - const server = Bun.serve({ port: 0, fetch: (req) => fetch(req.url, req) }) - return { - url: `http://127.0.0.1:${server.port}/mcp`, - close: async () => { - server.stop(true) - await close() - }, - } -} - -describe('mcp client worker — event wire', () => { - test('a call-tool request round-trips against a real MCP server', async () => { - const mcp = spawnMcpWorker() - const server = await startLoopbackServer() - try { - mcp.call('c1', 'call-tool', { url: server.url, tool: 'echo', args: { message: 'hi' } }) - const { id, ok, result } = await mcp.resultFor('c1') - expect(id).toBe('c1') - expect(ok).toBe(true) - const output = result?.output as { content: Array<{ text: string }> } - expect(output.content[0]?.text).toBe('echo:hi') - } finally { - await server.close() - mcp.terminate() - } - }) - - test('input that fails the op schema is error data, not silence', async () => { - const mcp = spawnMcpWorker() - try { - // call-tool requires `tool` + `args` — this input has neither - mcp.call('c2', 'call-tool', { url: 'http://127.0.0.1:1/mcp' }) - const { ok, error } = await mcp.resultFor('c2') - expect(ok).toBe(false) - expect(String(error?.message).includes('invalid input')).toBe(true) - expect(error && 'output' in (error as Record)).toBe(false) - } finally { - mcp.terminate() - } - }) - - test('a 401 server surfaces typed authorization_required and echoes the request', async () => { - const mcp = spawnMcpWorker() - const denied = Bun.serve({ - port: 0, - fetch: () => new Response('unauthorized', { status: 401 }), - }) - try { - mcp.call('c3', 'list-tools', { url: `http://127.0.0.1:${denied.port}/mcp` }) - const { ok, error } = await mcp.resultFor('c3') - expect(ok).toBe(false) - expect(error?.code).toBe('authorization_required') - // the request echo is the replay spine's capture payload - const request = error?.request as { op: string; input: Record } - expect(request.op).toBe('list-tools') - expect(String(request.input.url)).toBe(`http://127.0.0.1:${denied.port}/mcp`) - } finally { - denied.stop(true) - mcp.terminate() - } - }) - - test('a cancel stops an in-flight call and reports canceled', async () => { - const mcp = spawnMcpWorker() - const hanging = Bun.serve({ - port: 0, - fetch: async () => { - await Bun.sleep(5_000) - return new Response('{}') - }, - }) - try { - mcp.call('c4', 'list-tools', { url: `http://127.0.0.1:${hanging.port}/mcp` }) - Bun.sleep(150).then(() => mcp.cancel('c4')) - const { ok, error } = await mcp.resultFor('c4') - expect(ok).toBe(false) - expect(error?.code).toBe('canceled') - } finally { - hanging.stop(true) - mcp.terminate() - } - }) - - test('a deadline breach reports timeout — the second stop door', async () => { - const mcp = spawnMcpWorker() - const hanging = Bun.serve({ - port: 0, - fetch: async () => { - await Bun.sleep(5_000) - return new Response('{}') - }, - }) - try { - mcp.call('c5', 'list-tools', { url: `http://127.0.0.1:${hanging.port}/mcp`, timeoutMs: 150 }) - const { ok, error } = await mcp.resultFor('c5') - expect(ok).toBe(false) - expect(error?.code).toBe('timeout') - } finally { - hanging.stop(true) - mcp.terminate() - } - }) - - test('the request space is echoed on the result', async () => { - const mcp = spawnMcpWorker() - const server = await startLoopbackServer() - try { - mcp.call('c6', 'list-tools', { url: server.url }, 'demo') - const { space, ok } = await mcp.resultFor('c6') - expect(ok).toBe(true) - expect(space).toBe('demo') - } finally { - await server.close() - mcp.terminate() - } - }) -}) diff --git a/src/faculties/mcp/tests/fixtures/mcp-server-fixture.ts b/src/faculties/mcp/tests/fixtures/mcp-server-fixture.ts deleted file mode 100644 index 56a92f146..000000000 --- a/src/faculties/mcp/tests/fixtures/mcp-server-fixture.ts +++ /dev/null @@ -1,73 +0,0 @@ -import type { FetchLike } from '@modelcontextprotocol/client' -import { createMcpHandler, McpServer } from '@modelcontextprotocol/server' -import * as z from 'zod' - -/** - * Create the test MCP server — registers the echo tool, greet prompt, and - * note resource used by the mcp-client test suite. - */ -const createTestServer = (): McpServer => { - const server = new McpServer({ name: 'behavioral-test-server', version: '0.0.0' }) - - server.registerTool( - 'echo', - { - description: 'Echo back the message argument as text.', - inputSchema: z.object({ message: z.string().optional() }), - }, - async (args: { message?: string }) => ({ - content: [{ type: 'text', text: `echo:${args.message ?? ''}` }], - }), - ) - - server.registerPrompt( - 'greet', - { - description: 'A greeting prompt.', - argsSchema: z.object({ name: z.string().optional() }), - }, - async (args: { name?: string }) => ({ - messages: [{ role: 'user', content: { type: 'text', text: `hello ${args.name ?? 'world'}` } }], - }), - ) - - server.registerResource( - 'note', - 'test://note', - { description: 'A short note resource.', mimeType: 'text/plain' }, - async () => ({ - contents: [{ uri: 'test://note', mimeType: 'text/plain', text: 'a note' }], - }), - ) - - return server -} - -/** - * Spin an in-process MCP server (no port, no socket) and return a fetch - * function + close hook. Backs MCP client tool tests with a real SDK server - * + real handler.fetch transport rather than a loopback HTTP server. - * - * The returned `url` is a synthetic identifier for pool keying — the - * `fetch` function routes requests directly through the handler without - * touching the network. - */ -export const startMcpServer = async (): Promise<{ - url: string - fetch: FetchLike - close: () => Promise -}> => { - const handler = createMcpHandler(() => createTestServer()) - - return { - url: `in-process://mcp-${crypto.randomUUID()}`, - fetch: (input: string | URL, init?: RequestInit) => handler.fetch(new Request(input, init)), - close: async () => { - try { - await handler.close() - } catch { - /* best-effort */ - } - }, - } -} diff --git a/src/faculties/mcp/tests/threads.spec.ts b/src/faculties/mcp/tests/threads.spec.ts deleted file mode 100644 index 3f8d67a78..000000000 --- a/src/faculties/mcp/tests/threads.spec.ts +++ /dev/null @@ -1,122 +0,0 @@ -/** - * The mcp-client thread library against the real engine — the cross-turn - * replay spine over the mcp worker's wire: typed `authorization_required` - * results are captured in the store (the only cross-turn memory — cold - * per-turn kills in-flight state), surfaced to the host as - * `mcp_authorization_required`, and replayed after grant ingress. - * - * The worker owns auth state and request echoes; these threads keep ONLY what - * crosses turns. Successful calls are never captured (result-cleaner is dead - * by design — no per-call store churn). - */ -import { describe, expect, test } from 'bun:test' -import { TRACE_MESSAGE_KINDS } from '../../../behavioral/behavioral.constants.ts' -import { behavioral } from '../../../behavioral/behavioral.ts' -import type { BPEvent, JsonObject, SelectionTrace, Trace } from '../../../behavioral/behavioral.types.ts' -import { FACULTY_MESSAGE_KINDS } from '../../faculties.constants.ts' -import { MCP_CALLS_COLLECTION, MCP_EVENT_TYPES, mcpThreads } from '../threads.ts' - -type Selected = { type: string; detail: Record | undefined } - -const runProgram = (events: BPEvent[]): Selected[] => { - const program = behavioral() - const selected: Selected[] = [] - program.useTrace((trace: Trace) => { - if (trace.kind === TRACE_MESSAGE_KINDS.selection) - selected.push({ - type: (trace as SelectionTrace).selected.type, - detail: (trace as SelectionTrace).selected.detail as Record | undefined, - }) - }) - for (const thread of mcpThreads) program.addThread(thread) - for (const event of events) - program.addThread({ label: `producer/${event.type}`, once: true, rules: [{ request: event }] }) - // addThread is inert — trigger admits one ingress event and runs one - // super-step; the second pump cascades transform re-entries. - program.trigger({ type: 'mcp_gate_pump', detail: {} }) - program.trigger({ type: 'mcp_gate_pump', detail: {} }) - return selected -} - -const authRequiredResult = (id: string): BPEvent => ({ - type: FACULTY_MESSAGE_KINDS.mcp_request_result, - detail: { - id, - ok: false, - error: { - code: 'authorization_required', - durationMs: 12, - message: 'unauthorized', - request: { op: 'list-tools', input: { url: 'https://mcp.example.com/mcp' } }, - }, - }, -}) - -const completedResult = (id: string): BPEvent => ({ - type: FACULTY_MESSAGE_KINDS.mcp_request_result, - detail: { - id, - result: { id, status: 'completed', durationMs: 40, output: { tools: [] } }, - }, -}) - -describe('mcp threads — the replay spine', () => { - test('an authorization_required result is captured in the store with the echoed request', () => { - const selected = runProgram([authRequiredResult('c1')]) - const put = selected.find((s) => s.type === FACULTY_MESSAGE_KINDS.store_request && s.detail?.op === 'put') - expect(put).toBeDefined() - const input = put?.detail?.input as JsonObject - expect(input.collection).toBe(MCP_CALLS_COLLECTION) - expect(input.key).toBe('c1') - expect(input.value).toEqual({ op: 'list-tools', input: { url: 'https://mcp.example.com/mcp' } }) - }) - - test('a completed result is never captured — no per-call store churn', () => { - const selected = runProgram([completedResult('c2')]) - expect(selected.some((s) => s.type === FACULTY_MESSAGE_KINDS.store_request && s.detail?.op !== 'get')).toBe(false) - }) - - test('an authorization_required result surfaces mcp_authorization_required to the host', () => { - const selected = runProgram([authRequiredResult('c3')]) - const surfaced = selected.find((s) => s.type === MCP_EVENT_TYPES.authorizationRequired) - expect(surfaced?.detail?.id).toBe('c3') - expect(String(surfaced?.detail?.reason).length > 0).toBe(true) - }) - - test('grant ingress replays the captured request and deletes the capture', () => { - const selected = runProgram([ - authRequiredResult('c4'), - { type: MCP_EVENT_TYPES.authorizationGranted, detail: { id: 'c4' } }, - // the store worker's get result, as the router would re-enter it - { - type: FACULTY_MESSAGE_KINDS.store_request_result, - detail: { - id: 'c4', - result: { value: { op: 'list-tools', input: { url: 'https://mcp.example.com/mcp' } } }, - }, - }, - ]) - const get = selected.find((s) => s.type === FACULTY_MESSAGE_KINDS.store_request && s.detail?.op === 'get') - expect((get?.detail?.input as JsonObject)?.key).toBe('c4') - const retry = selected.find((s) => s.type === FACULTY_MESSAGE_KINDS.mcp_request && s.detail?.id === 'c4-retry') - expect(retry?.detail?.op).toBe('list-tools') - expect((retry?.detail?.input as JsonObject)?.url).toBe('https://mcp.example.com/mcp') - const del = selected.find( - (s) => s.type === FACULTY_MESSAGE_KINDS.store_request && s.detail?.op === 'delete' && s.detail?.id === 'c4', - ) - expect(del).toBeDefined() - }) - - test('a store value that is not a captured request does not replay', () => { - const selected = runProgram([ - { - type: FACULTY_MESSAGE_KINDS.store_request_result, - detail: { - id: 'c5', - result: { value: { skills: [{ name: 'alpha' }], warnings: [] } }, - }, - }, - ]) - expect(selected.some((s) => s.type === FACULTY_MESSAGE_KINDS.mcp_request)).toBe(false) - }) -}) diff --git a/src/faculties/mcp/threads.ts b/src/faculties/mcp/threads.ts deleted file mode 100644 index f4a2a0866..000000000 --- a/src/faculties/mcp/threads.ts +++ /dev/null @@ -1,130 +0,0 @@ -/** - * The mcp faculty's default threads — the cross-turn auth replay spine: - * capture-on-auth-required, host surfacing, grant-triggered store get, and - * the replayer. Ships with the faculty ("threads arrive with the worker they - * drive"); requires store + mcp — bProgram mounts it only when both - * are on. - * - * Moved from src/threads/mcp-client.ts when the threads became faculty-shipped. - * - * @packageDocumentation - */ - -import type { Thread } from '../../behavioral/behavioral.types.ts' -import { FACULTY_MESSAGE_KINDS } from '../faculties.constants.ts' - -// ── Vocabulary ─────────────────────────────────────────────────────────────── - -/** Thread-owned event types (hosts route mcp_authorization_required; granted is host ingress). */ -export const MCP_EVENT_TYPES = { - authorizationRequired: 'mcp_authorization_required', - authorizationGranted: 'mcp_authorization_granted', -} as const - -/** The store collection holding captured mcp requests keyed by call id. */ -export const MCP_CALLS_COLLECTION = 'mcp-calls' - -const MCP_RESULT_DETAIL = { - type: 'object', - properties: { - id: { type: 'string', minLength: 1 }, - ok: { type: 'boolean' }, - result: { type: 'object' }, - error: { type: 'object' }, - }, - required: ['id', 'ok'], -} as const - -// ── Threads ─────────────────────────────────────────────────────────────────── - -/** auth-capture — file ONLY auth-failed requests; the result's echo is the payload. */ -const authCapture: Thread = { - label: 'mcp/auth-capture', - rules: [ - { - transform: [ - { - type: FACULTY_MESSAGE_KINDS.mcp_request_result, - query: `. as $d | select($d.ok == false and $d.error.code? == "authorization_required") | {id: $d.id, op: "put", input: {collection: "${MCP_CALLS_COLLECTION}", key: $d.id, value: $d.error.request}}`, - target: FACULTY_MESSAGE_KINDS.store_request, - detailSchema: MCP_RESULT_DETAIL, - }, - ], - }, - ], -} - -/** auth-surfacer — the typed result surfaces to the host; the capture stays pending. */ -const authSurfacer: Thread = { - label: 'mcp/auth-surfacer', - rules: [ - { - transform: [ - { - type: FACULTY_MESSAGE_KINDS.mcp_request_result, - query: - '. as $d | select($d.ok == false and $d.error.code? == "authorization_required") | {id: $d.id, reason: "mcp call requires authorization"}', - target: MCP_EVENT_TYPES.authorizationRequired, - detailSchema: MCP_RESULT_DETAIL, - }, - ], - }, - ], -} - -/** auth-retry — granted ingress triggers the store get; the replayer takes it from there. */ -const authRetry: Thread = { - label: 'mcp/auth-retry', - rules: [ - { - transform: [ - { - type: MCP_EVENT_TYPES.authorizationGranted, - query: '. as $d | {id: $d.id, op: "get", input: {collection: "mcp-calls", key: $d.id}}', - target: FACULTY_MESSAGE_KINDS.store_request, - detailSchema: { - type: 'object', - properties: { id: { type: 'string', minLength: 1 } }, - required: ['id'], - }, - }, - ], - }, - ], -} - -/** replayer — a store get carrying a captured request replays the mcp_request and deletes the capture. */ -const replayer: Thread = { - label: 'mcp/replayer', - rules: [ - { - transform: [ - { - type: FACULTY_MESSAGE_KINDS.store_request_result, - query: - '. as $d | select($d.result.value.op != null) | {id: ($d.id + "-retry"), op: $d.result.value.op, input: $d.result.value.input}', - target: FACULTY_MESSAGE_KINDS.mcp_request, - detailSchema: { - type: 'object', - properties: { id: { type: 'string', minLength: 1 }, result: { type: 'object' } }, - required: ['id', 'result'], - }, - }, - { - type: FACULTY_MESSAGE_KINDS.store_request_result, - query: - '. as $d | select($d.result.value.op != null) | {id: $d.id, op: "delete", input: {collection: "mcp-calls", key: $d.id}}', - target: FACULTY_MESSAGE_KINDS.store_request, - detailSchema: { - type: 'object', - properties: { id: { type: 'string', minLength: 1 }, result: { type: 'object' } }, - required: ['id', 'result'], - }, - }, - ], - }, - ], -} - -/** The mcp-client thread library — add to the program alongside the satellites. */ -export const mcpThreads: Thread[] = [authCapture, authSurfacer, authRetry, replayer] diff --git a/src/faculties/mcp/types.ts b/src/faculties/mcp/types.ts deleted file mode 100644 index 3eff6f398..000000000 --- a/src/faculties/mcp/types.ts +++ /dev/null @@ -1,176 +0,0 @@ -/** - * Types shared by the mcp faculty process (`mcp/faculty.ts`) and its - * event-wire consumers. - * - * @remarks - * Types + op-input schemas only — no runtime behavior beyond compiled - * validators, so importing this module has no side effects on either side of - * the process boundary. The faculty runs as a spawned process (stdio lines, - * never imported by the host), so the host must never import the faculty for - * types; both sides import here instead. The wire itself is the behavioral - * event vocabulary (`mcp_request` / `mcp_cancel` in, one `mcp_request_result` - * out) defined in `src/faculties/faculties.types.ts` — only the - * `detail.input` and `detail.result` payload shapes live here. - * - * Per the 2026-09-21 worker-conversion ruling: per-call input - * credentials are RETIRED — `detail.input` carries the server URL and the - * op's own fields, never an `auth` config or credential headers. Auth binds - * at the worker's module scope from env-data (broker) with the keychain - * floor. Failure is typed `authorization_required` — errors-as-data, the - * replay spine's trigger. - * - * MINIMAL: op outputs are not schema-validated — remote MCP data passes - * through loose (payloads-loose); consumers gate with their own - * detailSchema. Upgrade path: per-op output schemas as exported schema-data - * when a consumer needs them. - * - * @packageDocumentation - */ - -import type { JSONSchemaType } from 'ajv' -import { ajv, type JsonObject } from '../../behavioral/behavioral.types.ts' -import type { McpOp } from '../faculties.types.ts' - -// --------------------------------------------------------------------------- -// Env-data — the vend's broker binding. One home: `security/types.ts` (the -// security faculty's); re-exported here until the mcp faculty's deprecation. -// --------------------------------------------------------------------------- - -export { MCP_BROKER_BOOT_SECRET_KEY, MCP_BROKER_URL_KEY } from '../security/types.ts' - -// --------------------------------------------------------------------------- -// Op inputs — one shape per op, no `mode` discriminator, no auth fields -// --------------------------------------------------------------------------- - -export type McpSharedInput = { - /** Remote MCP server URL. */ - url: string - /** Wall-clock deadline for the whole call (connect + op). */ - timeoutMs?: number -} - -export type McpCallToolOpInput = McpSharedInput & { tool: string; args: Record } -export type McpListToolsOpInput = McpSharedInput -export type McpListPromptsOpInput = McpSharedInput -export type McpGetPromptOpInput = McpSharedInput & { name: string; args?: Record } -export type McpListResourcesOpInput = McpSharedInput -export type McpReadResourceOpInput = McpSharedInput & { uri: string } -export type McpDiscoverOpInput = McpSharedInput - -const sharedProperties = { - url: { type: 'string', minLength: 1, description: 'remote MCP server URL' }, - timeoutMs: { - type: 'integer', - minimum: 1, - nullable: true, - description: 'wall-clock deadline for the whole call (connect + op) in milliseconds', - }, -} as const - -export const McpCallToolOpInputSchema: JSONSchemaType = { - type: 'object', - properties: { - ...sharedProperties, - tool: { type: 'string', minLength: 1, description: 'tool name to call' }, - args: { - type: 'object', - additionalProperties: true, - description: 'tool arguments — a JSON object, passed through to the remote server', - }, - }, - required: ['url', 'tool', 'args'], - additionalProperties: false, -} as unknown as JSONSchemaType - -export const McpListToolsOpInputSchema: JSONSchemaType = { - type: 'object', - properties: { ...sharedProperties }, - required: ['url'], - additionalProperties: false, -} as unknown as JSONSchemaType - -export const McpListPromptsOpInputSchema: JSONSchemaType = McpListToolsOpInputSchema - -export const McpListResourcesOpInputSchema: JSONSchemaType = McpListToolsOpInputSchema - -export const McpDiscoverOpInputSchema: JSONSchemaType = McpListToolsOpInputSchema - -export const McpGetPromptOpInputSchema: JSONSchemaType = { - type: 'object', - properties: { - ...sharedProperties, - name: { type: 'string', minLength: 1, description: 'prompt name' }, - args: { - type: 'object', - additionalProperties: { type: 'string' }, - nullable: true, - description: 'prompt arguments', - }, - }, - required: ['url', 'name'], - additionalProperties: false, -} as unknown as JSONSchemaType - -export const McpReadResourceOpInputSchema: JSONSchemaType = { - type: 'object', - properties: { - ...sharedProperties, - uri: { type: 'string', minLength: 1, description: 'resource URI to read' }, - }, - required: ['url', 'uri'], - additionalProperties: false, -} as unknown as JSONSchemaType - -// --------------------------------------------------------------------------- -// The result envelope — typed auth-state rides first-class -// --------------------------------------------------------------------------- - -/** Outcome discriminator for one mcp op execution. */ -export type McpCallStatus = 'completed' | 'authorization_required' | 'timeout' | 'canceled' | 'error' - -/** - * The single terminal payload the worker returns on `mcp_request_result`. - * - * @remarks - * Errors-as-data throughout: no throw crosses the wire. `authorization_required` - * carries the originating request verbatim — the replay spine's capture - * payload (the store never saw the request; the result is the only carrier). - */ -export type McpCallResult = { - /** Correlation id, echoed from the request. */ - id: string - /** Outcome discriminator. */ - status: McpCallStatus - /** Elapsed wall-clock time in milliseconds. */ - durationMs: number - /** Op output when status is completed. Loose remote MCP data (payloads-loose). */ - output?: JsonObject - /** Failure detail; the authorization reason when status is authorization_required. */ - message?: string - /** The originating request, echoed only on authorization_required — the replay payload. */ - request?: { op: McpOp; input: JsonObject } -} - -// --------------------------------------------------------------------------- -// Op input boundary — the trust boundary for anything crossing into this -// process; one compiled validator per op. -// --------------------------------------------------------------------------- - -export const validateMcpCallToolOpInput = ajv.compile(McpCallToolOpInputSchema) -export const validateMcpListToolsOpInput = ajv.compile(McpListToolsOpInputSchema) -export const validateMcpListPromptsOpInput = ajv.compile(McpListPromptsOpInputSchema) -export const validateMcpGetPromptOpInput = ajv.compile(McpGetPromptOpInputSchema) -export const validateMcpListResourcesOpInput = ajv.compile(McpListResourcesOpInputSchema) -export const validateMcpReadResourceOpInput = ajv.compile(McpReadResourceOpInputSchema) -export const validateMcpDiscoverOpInput = ajv.compile(McpDiscoverOpInputSchema) - -/** Op input validator registry — one compiled validator per McpOp. */ -export const MCP_OP_INPUT_VALIDATORS: Record boolean> = { - 'call-tool': validateMcpCallToolOpInput, - 'list-tools': validateMcpListToolsOpInput, - 'list-prompts': validateMcpListPromptsOpInput, - 'get-prompt': validateMcpGetPromptOpInput, - 'list-resources': validateMcpListResourcesOpInput, - 'read-resource': validateMcpReadResourceOpInput, - discover: validateMcpDiscoverOpInput, -} diff --git a/src/faculties/security/faculty.ts b/src/faculties/security/faculty.ts index d49a23d8f..0406d15e5 100644 --- a/src/faculties/security/faculty.ts +++ b/src/faculties/security/faculty.ts @@ -112,6 +112,7 @@ const postResult = ({ echo, error, space, + ctx, }: { id: string token?: string @@ -119,12 +120,27 @@ const postResult = ({ echo?: JsonObject error?: { code: string; message?: string } space?: string + ctx?: JsonObject }): void => { emit({ type: FACULTY_MESSAGE_KINDS.credential_result, + // The request's ctx echoes on BOTH branches — the caller's join lane + // round-trips through failures too (the absent-credential surfacing). detail: (error === undefined - ? { id, ok: true, result: { token, ...(echo === undefined ? {} : { echo }) } } - : { id, ok: false, error: error as JsonObject }) as JsonObject & { id: string }, + ? { + id, + ok: true, + result: { token, ...(echo === undefined ? {} : { echo }) }, + ...(ctx === undefined ? {} : { ctx }), + } + : { + id, + ok: false, + error: error as JsonObject, + ...(ctx === undefined ? {} : { ctx }), + }) as unknown as JsonObject & { + id: string + }, ...(space === undefined ? {} : { space }), }) } @@ -157,6 +173,7 @@ const handleInbound = async (message: unknown): Promise => { id, space: event.space, error: { code: 'error', message: `invalid input: ${ajv.errorsText(validate.errors)}` }, + ctx, }) return } @@ -170,6 +187,7 @@ const handleInbound = async (message: unknown): Promise => { id, space: event.space, error: { code: 'error', message: `invalid input: ${ajv.errorsText(validateContext.errors)}` }, + ctx, }) return } @@ -196,10 +214,11 @@ const handleInbound = async (message: unknown): Promise => { id, space: event.space, error: { code: 'error', message: `no credential available for ${serverUrl}` }, + ctx, }) return } - postResult({ id, space: event.space, token, echo }) + postResult({ id, space: event.space, token, echo, ctx }) } catch (err) { // The vend is fail-closed by construction; this is the last-resort guard // so no worker-side throw ever escapes as a crash. @@ -207,6 +226,7 @@ const handleInbound = async (message: unknown): Promise => { id, space: event.space, error: { code: 'error', message: err instanceof Error ? err.message : String(err) }, + ctx, }) } finally { active.delete(id) diff --git a/src/faculties/shell/faculty.ts b/src/faculties/shell/faculty.ts index f5c29e80c..aae214396 100644 --- a/src/faculties/shell/faculty.ts +++ b/src/faculties/shell/faculty.ts @@ -355,6 +355,7 @@ const runRpcOp = async ({ // The replayed token rides the input (thread-injected); the op itself // never knows OAuth. getAuthToken: async () => input.authToken, + ...(input.headers === undefined ? {} : { headers: input.headers }), signal: controller.signal, }) const durationMs = Math.round(performance.now() - started) @@ -363,6 +364,17 @@ const runRpcOp = async ({ if (execution.stopReason === 'canceled') return { code: 'canceled', durationMs } if (execution.stopReason === 'timeout') return { code: 'timeout', durationMs } if (outcome.ok) return { output: outcome.result, durationMs } + // The reactive auth path: a 401 challenge on an unauthenticated call maps + // to the typed vend-and-replay capture payload (the thread pack vends and + // replays). A 401 on a token'd call stays a remote error — bounded. + if (outcome.error.code === 401 && input.authToken === undefined) { + return { + code: 'credential_required', + durationMs, + message: `credential required for ${input.url}`, + request: { op: 'rpc', input }, + } + } return { code: 'error', durationMs, @@ -598,17 +610,26 @@ const postResult = ({ payload, error, space, + ctx, }: { id: string payload?: ShellSuccess | RpcOpSuccess error?: ShellError | RpcOpError space?: string + ctx?: JsonObject }): void => { emit({ type: FACULTY_MESSAGE_KINDS.shell_request_result, + // The request's ctx echoes at detail level — the out-of-band join lane + // (thread orchestration state round-trips beside ok, never model-facing). detail: (error === undefined - ? { id, ok: true, result: (payload ?? {}) as unknown as JsonObject } - : { id, ok: false, error: error as unknown as JsonObject }) as JsonObject & { id: string }, + ? { id, ok: true, result: (payload ?? {}) as unknown as JsonObject, ...(ctx === undefined ? {} : { ctx }) } + : { + id, + ok: false, + error: error as unknown as JsonObject, + ...(ctx === undefined ? {} : { ctx }), + }) as unknown as JsonObject & { id: string }, ...(space === undefined ? {} : { space }), }) } @@ -642,7 +663,7 @@ const handleInbound = async (message: unknown): Promise => { // defense in depth at the process boundary. if (!validateShellRequestEvent(message)) return const event = message as ShellRequestEvent - const { id, input } = event.detail + const { id, input, ctx } = event.detail // Input that fails the boundary is error data, not a throw: the id is // valid, so the caller learns why nothing ran. @@ -652,6 +673,7 @@ const handleInbound = async (message: unknown): Promise => { id, space: event.space, error: errorInterior({ message: `invalid input: ${ajv.errorsText(validate.errors)}` }), + ctx, }) return } @@ -665,15 +687,16 @@ const handleInbound = async (message: unknown): Promise => { // The clamp report rides whichever branch ran. const withClamp = clamped.length === 0 ? interior : { ...interior, clamped } if ('code' in interior) { - postResult({ id, space: event.space, error: withClamp as ShellError }) + postResult({ id, space: event.space, error: withClamp as ShellError, ctx }) } else { - postResult({ id, space: event.space, payload: withClamp as ShellSuccess }) + postResult({ id, space: event.space, payload: withClamp as ShellSuccess, ctx }) } } catch (err) { postResult({ id, space: event.space, error: errorInterior({ message: err instanceof Error ? err.message : String(err) }), + ctx, }) } } diff --git a/src/faculties/shell/remote-mcp.threads.ts b/src/faculties/shell/remote-mcp.threads.ts new file mode 100644 index 000000000..ce7c3d7e8 --- /dev/null +++ b/src/faculties/shell/remote-mcp.threads.ts @@ -0,0 +1,512 @@ +/** + * The remote-mcp thread pack — the MCP layering over the shell faculty's + * generic `rpc` op. This is where "MCP" lives: the op is transport-shaped, + * and this pack stamps the protocol envelope, drives discovery, executes + * tools, runs the multi-round-trip elicitation loop, and retries retryable + * remote failures. + * + * @remarks + * Per the Direction/A ruling (MCP 2026-07-28 is stateless): no handshake, no + * session id — every request carries the protocol stamp in-band, and every + * response is one stateless HTTP POST's answer. + * + * - **Request stamping** — every rpc op the pack issues carries the + * `_meta` envelope (`io.modelcontextprotocol/protocolVersion` + + * `clientInfo` + `clientCapabilities`, the reserved request-envelope keys + * for this revision) and the `MCP-Protocol-Version` header. The op carries + * the envelope; the pack stamps it. + * - **Discovery** — `remote_mcp_discover { url }` issues `server/discover`, + * chains `tools/list`, registers the tools in the store registry + * (alongside the skills/plugins tenants), and surfaces + * `remote_mcp_discovered`. + * - **Execution** — `remote_mcp_call { url, tool, args }` issues `tools/call` + * and surfaces `remote_mcp_call_result`. + * - **MRTR** — an `input_required` result (the reserved `inputRequests` / + * `requestState` members; at-least-one) surfaces + * `remote_mcp_elicitation` to the host; the host answers with + * `remote_mcp_elicitation_response` (the elicitation detail echoed + the + * bare `inputResponses`) and the pack retries `tools/call` with the + * answers + a byte-exact `requestState` echo, on a FRESH request id, up to + * the round cap. + * - **Retry** — retryable remote failures (deadline, network, generic 5xx) + * re-request the op with the attempt advanced, bounded. `retry-after` + * rides as data when a server sends it (the op surfaces the remote code). + * + * THE JOIN LANE: cross-event state (the original call behind a result, the + * attempt counter, the MRTR round) rides the shell wire's `ctx` echo — the + * out-of-band lane beside `input` (the you.com MCP `_meta` pattern: + * host-supplied, round-tripped verbatim, never model-facing). Auth rides the + * credential seam (`shell/rpc-auth.threads.ts`): a 401 challenge maps to the + * typed `credential_required`, and the seam vends + replays with the token. + * + * Trusted response shapes — the pack AJV-validates ONLY the four responses + * it acts on (`server/discover`, `tools/list`, `tools/call`, + * `InputRequiredResult`); a response failing its trusted shape silently + * no-matches the acting transform (the result stays visible as an unmatched + * event in the frontier traces) — fail-closed, not silently-wrong. + * + * Requires shell + security + store — bProgram mounts it only when all three + * are on. + * + * MINIMAL: the failed-vend surfacing rides the security faculty's error-branch + * ctx echo (the vend-failure thread) — the caller learns the typed + * absent-credential error instead of pending forever. + * + * @packageDocumentation + */ + +import type { Thread } from '../../behavioral/behavioral.types.ts' +import { FACULTY_MESSAGE_KINDS } from '../faculties.constants.ts' + +// ── Vocabulary ─────────────────────────────────────────────────────────────── + +/** Thread-owned event types: discover/call are ingress; discovered/callResult/elicitation surface; the response is host ingress. */ +export const REMOTE_MCP_EVENT_TYPES = { + discover: 'remote_mcp_discover', + call: 'remote_mcp_call', + discovered: 'remote_mcp_discovered', + callResult: 'remote_mcp_call_result', + elicitation: 'remote_mcp_elicitation', + elicitationResponse: 'remote_mcp_elicitation_response', +} as const + +/** The store registry tenant holding registered remote tools, keyed by server URL. */ +export const REMOTE_MCP_STORE_COLLECTION = 'remote-mcp' + +/** The protocol revision this pack speaks (the 2026-07-28 stateless era). */ +export const REMOTE_MCP_PROTOCOL_VERSION = '2026-07-28' + +/** The label every pack-issued rpc op carries (trace annotation, no routing weight). */ +export const REMOTE_MCP_LABEL = 'remote-mcp' + +/** Bounded retry: a retryable failure re-requests the op until this many attempts. */ +export const REMOTE_MCP_MAX_ATTEMPTS = 2 + +/** Bounded MRTR: the elicitation loop runs until this many rounds. */ +export const REMOTE_MCP_MAX_ROUNDS = 2 + +// The protocol stamp — the reserved request-envelope `_meta` keys for this +// revision, plus the header form. RESULTS never carry these (they are +// request-envelope keys only). +const STAMP_HEADERS = `{ "MCP-Protocol-Version": "${REMOTE_MCP_PROTOCOL_VERSION}" }` + +const STAMP_META = `{ "io.modelcontextprotocol/protocolVersion": "${REMOTE_MCP_PROTOCOL_VERSION}", "io.modelcontextprotocol/clientInfo": { name: "behavioral", version: "0.0.0" }, "io.modelcontextprotocol/clientCapabilities": { elicitation: {} } }` + +// ── The four trusted response shapes ───────────────────────────────────────── +// Minimal slices of the 2026-07-28 responses the pack acts on — the trust +// boundary for anything crossing in from a remote server. Loose on members +// the pack doesn't consume (the responses carry _meta, icons, …). + +/** `server/discover` result — the pack trusts the advertised versions. */ +export const REMOTE_MCP_DISCOVER_RESULT_SCHEMA = { + type: 'object', + properties: { + supportedVersions: { type: 'array', items: { type: 'string' }, minItems: 1 }, + capabilities: { type: 'object' }, + }, + required: ['supportedVersions'], + additionalProperties: true, +} as const + +/** `tools/list` result — the pack trusts the tool names (registration keys). */ +export const REMOTE_MCP_TOOLS_LIST_RESULT_SCHEMA = { + type: 'object', + properties: { + tools: { + type: 'array', + items: { + type: 'object', + properties: { + name: { type: 'string', minLength: 1 }, + description: { type: 'string' }, + inputSchema: { type: 'object' }, + }, + required: ['name'], + additionalProperties: true, + }, + }, + }, + required: ['tools'], + additionalProperties: true, +} as const + +/** `tools/call` result — the pack trusts the content array shape (loose members). */ +export const REMOTE_MCP_CALL_RESULT_SCHEMA = { + type: 'object', + properties: { + content: { type: 'array', items: { type: 'object' } }, + isError: { type: 'boolean' }, + }, + additionalProperties: true, +} as const + +/** + * `InputRequiredResult` — the reserved `inputRequests` / `requestState` + * members, at-least-one (the server seam's rule). The MRTR handshake's + * embedded requests ride `inputRequests`; the opaque `requestState` echoes + * byte-exact on the retry. + */ +export const REMOTE_MCP_INPUT_REQUIRED_RESULT_SCHEMA = { + type: 'object', + properties: { + inputRequests: { type: 'object' }, + requestState: { type: 'string', minLength: 1 }, + }, + anyOf: [{ required: ['inputRequests'] }, { required: ['requestState'] }], + additionalProperties: true, +} as const + +// ── Shared jq fragments ────────────────────────────────────────────────────── + +/** Retryable remote failure: deadline, network, or a generic 5xx (data, not policy). */ +const RETRYABLE_REMOTE = + '(($d.error.code == "timeout") or ($d.error.remoteCode == "network") or (($d.error.remoteCode | tonumber? // 0) >= 500))' + +/** The gate every acting transform shares: a shell result with the join lane present. */ +const RESULT_DETAIL_GATE = { + type: 'object', + properties: { + id: { type: 'string', minLength: 1 }, + ok: { type: 'boolean' }, + result: { type: 'object' }, + error: { type: 'object' }, + ctx: { type: 'object' }, + }, + required: ['id', 'ok'], +} as const + +// ── Threads ────────────────────────────────────────────────────────────────── + +/** discover-issue — `remote_mcp_discover` issues the stamped `server/discover` op. */ +const discoverIssue: Thread = { + label: 'remote-mcp/discover-issue', + rules: [ + { + transform: [ + { + type: REMOTE_MCP_EVENT_TYPES.discover, + query: `. as $d | select($d.input.url != null) | { id: ($d.id + "-discover"), label: "${REMOTE_MCP_LABEL}", ctx: { echo: { source: $d.id, url: $d.input.url, leg: "discover", attempt: 0 } }, input: { op: "rpc", url: $d.input.url, method: "server/discover", headers: ${STAMP_HEADERS}, params: { _meta: ${STAMP_META} } } }`, + target: FACULTY_MESSAGE_KINDS.shell_request, + detailSchema: { + type: 'object', + properties: { + id: { type: 'string', minLength: 1 }, + input: { type: 'object', properties: { url: { type: 'string', minLength: 1 } }, required: ['url'] }, + }, + required: ['id', 'input'], + }, + }, + ], + }, + ], +} + +/** tools-issue — the discover result (trusted shape) chains the stamped `tools/list` op. */ +const toolsIssue: Thread = { + label: 'remote-mcp/tools-issue', + rules: [ + { + transform: [ + { + type: FACULTY_MESSAGE_KINDS.shell_request_result, + query: `. as $d | select($d.ok == true and $d.ctx.echo.leg == "discover") | { id: ($d.ctx.echo.source + "-tools"), label: "${REMOTE_MCP_LABEL}", ctx: { echo: { source: $d.ctx.echo.source, url: $d.ctx.echo.url, leg: "tools", attempt: 0 } }, input: { op: "rpc", url: $d.ctx.echo.url, method: "tools/list", headers: ${STAMP_HEADERS}, params: { _meta: ${STAMP_META} } } }`, + target: FACULTY_MESSAGE_KINDS.shell_request, + detailSchema: { + type: 'object', + properties: { + id: { type: 'string', minLength: 1 }, + ok: { type: 'boolean' }, + result: { + type: 'object', + properties: { output: REMOTE_MCP_DISCOVER_RESULT_SCHEMA }, + required: ['output'], + }, + }, + required: ['id', 'ok', 'result'], + }, + }, + ], + }, + ], +} + +/** register — the tools result (trusted shape) registers the tenant and surfaces the outcome. */ +const register: Thread = { + label: 'remote-mcp/register', + rules: [ + { + transform: [ + { + type: FACULTY_MESSAGE_KINDS.shell_request_result, + query: `. as $d | select($d.ok == true and $d.ctx.echo.leg == "tools") | { id: ("rmcp-register-" + $d.ctx.echo.source), op: "put", input: { collection: "${REMOTE_MCP_STORE_COLLECTION}", key: $d.ctx.echo.url, value: { url: $d.ctx.echo.url, tools: $d.result.output.tools } } }`, + target: FACULTY_MESSAGE_KINDS.store_request, + detailSchema: { + type: 'object', + properties: { + id: { type: 'string', minLength: 1 }, + ok: { type: 'boolean' }, + result: { + type: 'object', + properties: { output: REMOTE_MCP_TOOLS_LIST_RESULT_SCHEMA }, + required: ['output'], + }, + }, + required: ['id', 'ok', 'result'], + }, + }, + { + type: FACULTY_MESSAGE_KINDS.shell_request_result, + query: `. as $d | select($d.ok == true and $d.ctx.echo.leg == "tools") | { id: $d.ctx.echo.source, ok: true, input: { url: $d.ctx.echo.url, tools: $d.result.output.tools } }`, + target: REMOTE_MCP_EVENT_TYPES.discovered, + detailSchema: { + type: 'object', + properties: { + id: { type: 'string', minLength: 1 }, + ok: { type: 'boolean' }, + result: { + type: 'object', + properties: { output: REMOTE_MCP_TOOLS_LIST_RESULT_SCHEMA }, + required: ['output'], + }, + }, + required: ['id', 'ok', 'result'], + }, + }, + ], + }, + ], +} + +/** call-issue — `remote_mcp_call` issues the stamped `tools/call` op. */ +const callIssue: Thread = { + label: 'remote-mcp/call-issue', + rules: [ + { + transform: [ + { + type: REMOTE_MCP_EVENT_TYPES.call, + query: `. as $d | select($d.input.url != null and $d.input.tool != null) | { id: ($d.id + "-call"), label: "${REMOTE_MCP_LABEL}", ctx: { echo: { source: $d.id, url: $d.input.url, tool: $d.input.tool, args: ($d.input.args // {}), leg: "call", round: 0, attempt: 0 } }, input: { op: "rpc", url: $d.input.url, method: "tools/call", headers: ${STAMP_HEADERS}, params: { name: $d.input.tool, arguments: ($d.input.args // {}), _meta: ${STAMP_META} } } }`, + target: FACULTY_MESSAGE_KINDS.shell_request, + detailSchema: { + type: 'object', + properties: { + id: { type: 'string', minLength: 1 }, + input: { + type: 'object', + properties: { url: { type: 'string', minLength: 1 }, tool: { type: 'string', minLength: 1 } }, + required: ['url', 'tool'], + }, + }, + required: ['id', 'input'], + }, + }, + ], + }, + ], +} + +/** elicitation — an `input_required` result (trusted shape) surfaces the embedded requests to the host. */ +const elicitation: Thread = { + label: 'remote-mcp/elicitation', + rules: [ + { + transform: [ + { + type: FACULTY_MESSAGE_KINDS.shell_request_result, + query: `. as $d | select($d.ok == true and $d.ctx.echo.leg == "call") | { id: $d.ctx.echo.source, input: { url: $d.ctx.echo.url, tool: $d.ctx.echo.tool, args: $d.ctx.echo.args, round: $d.ctx.echo.round, inputRequests: $d.result.output.inputRequests, requestState: $d.result.output.requestState } }`, + target: REMOTE_MCP_EVENT_TYPES.elicitation, + detailSchema: { + type: 'object', + properties: { + id: { type: 'string', minLength: 1 }, + ok: { type: 'boolean' }, + result: { + type: 'object', + properties: { output: REMOTE_MCP_INPUT_REQUIRED_RESULT_SCHEMA }, + required: ['output'], + }, + }, + required: ['id', 'ok', 'result'], + }, + }, + ], + }, + ], +} + +/** call-result — a complete `tools/call` result (trusted shape) surfaces to the caller. */ +const callResult: Thread = { + label: 'remote-mcp/call-result', + rules: [ + { + transform: [ + { + type: FACULTY_MESSAGE_KINDS.shell_request_result, + query: `. as $d | select($d.ok == true and $d.ctx.echo.leg == "call" and ($d.result.output.inputRequests == null) and ($d.result.output.requestState == null)) | { id: $d.ctx.echo.source, ok: true, result: $d.result.output }`, + target: REMOTE_MCP_EVENT_TYPES.callResult, + detailSchema: { + type: 'object', + properties: { + id: { type: 'string', minLength: 1 }, + ok: { type: 'boolean' }, + result: { type: 'object', properties: { output: REMOTE_MCP_CALL_RESULT_SCHEMA }, required: ['output'] }, + }, + required: ['id', 'ok', 'result'], + }, + }, + ], + }, + ], +} + +/** retry-issue — the host's answers retry `tools/call`: fresh request id, answers + the byte-exact `requestState` echo. */ +const retryIssue: Thread = { + label: 'remote-mcp/retry-issue', + rules: [ + { + transform: [ + { + type: REMOTE_MCP_EVENT_TYPES.elicitationResponse, + query: `. as $d | select(($d.input.round // 0) < ${REMOTE_MCP_MAX_ROUNDS}) | { id: ($d.id + "-call-r" + ((($d.input.round // 0) + 1) | tostring)), label: "${REMOTE_MCP_LABEL}", ctx: { echo: { source: $d.id, url: $d.input.url, tool: $d.input.tool, args: ($d.input.args // {}), leg: "call", round: (($d.input.round // 0) + 1), attempt: 0 } }, input: { op: "rpc", url: $d.input.url, method: "tools/call", headers: ${STAMP_HEADERS}, params: (({ name: $d.input.tool, arguments: ($d.input.args // {}), _meta: ${STAMP_META} }) + (if $d.input.inputResponses != null then { inputResponses: $d.input.inputResponses } else {} end) + (if $d.input.requestState != null then { requestState: $d.input.requestState } else {} end)) } }`, + target: FACULTY_MESSAGE_KINDS.shell_request, + detailSchema: { + type: 'object', + properties: { + id: { type: 'string', minLength: 1 }, + input: { + type: 'object', + properties: { url: { type: 'string' }, tool: { type: 'string' } }, + required: ['url', 'tool'], + }, + }, + required: ['id', 'input'], + }, + }, + ], + }, + ], +} + +/** round-cap — the MRTR loop exhausted: the caller gets the typed cap error. */ +const roundCap: Thread = { + label: 'remote-mcp/round-cap', + rules: [ + { + transform: [ + { + type: REMOTE_MCP_EVENT_TYPES.elicitationResponse, + query: `. as $d | select(($d.input.round // 0) >= ${REMOTE_MCP_MAX_ROUNDS}) | { id: $d.id, ok: false, error: { code: "round_cap", message: "multi-round-trip cap exhausted" } }`, + target: REMOTE_MCP_EVENT_TYPES.callResult, + detailSchema: { + type: 'object', + properties: { id: { type: 'string', minLength: 1 }, input: { type: 'object' } }, + required: ['id', 'input'], + }, + }, + ], + }, + ], +} + +/** + * retry — a retryable remote failure re-requests the op with the attempt + * advanced (the leg decides the rebuild). Bounded: at the cap the failure + * surfaces instead. The credential seam's `credential_required` failures are + * excluded — the vend-and-replay owns those. + */ +const retry: Thread = { + label: 'remote-mcp/retry', + rules: [ + { + transform: [ + { + type: FACULTY_MESSAGE_KINDS.shell_request_result, + query: `. as $d | select($d.ok == false and ($d.error.code != "credential_required") and ($d.ctx.echo != null) and (($d.ctx.echo.attempt // 0) < ${REMOTE_MCP_MAX_ATTEMPTS}) and ${RETRYABLE_REMOTE}) | { id: $d.id, label: "${REMOTE_MCP_LABEL}", ctx: { echo: ($d.ctx.echo + { attempt: (($d.ctx.echo.attempt // 0) + 1) }) }, input: (if $d.ctx.echo.leg == "call" then { op: "rpc", url: $d.ctx.echo.url, method: "tools/call", headers: ${STAMP_HEADERS}, params: { name: $d.ctx.echo.tool, arguments: ($d.ctx.echo.args // {}), _meta: ${STAMP_META} } } elif $d.ctx.echo.leg == "tools" then { op: "rpc", url: $d.ctx.echo.url, method: "tools/list", headers: ${STAMP_HEADERS}, params: ${STAMP_META} } else { op: "rpc", url: $d.ctx.echo.url, method: "server/discover", headers: ${STAMP_HEADERS}, params: ${STAMP_META} } end) }`, + target: FACULTY_MESSAGE_KINDS.shell_request, + detailSchema: RESULT_DETAIL_GATE, + }, + ], + }, + ], +} + +/** call-failure — a non-retryable (or exhausted) call failure surfaces to the caller. */ +const callFailure: Thread = { + label: 'remote-mcp/call-failure', + rules: [ + { + transform: [ + { + type: FACULTY_MESSAGE_KINDS.shell_request_result, + query: `. as $d | select($d.ok == false and $d.ctx.echo.leg == "call" and ($d.error.code != "credential_required") and ((${RETRYABLE_REMOTE} and (($d.ctx.echo.attempt // 0) < ${REMOTE_MCP_MAX_ATTEMPTS})) | not)) | { id: $d.ctx.echo.source, ok: false, error: { code: $d.error.code, message: $d.error.message, remoteCode: $d.error.remoteCode } }`, + target: REMOTE_MCP_EVENT_TYPES.callResult, + detailSchema: RESULT_DETAIL_GATE, + }, + ], + }, + ], +} + +/** discover-failure — a non-retryable (or exhausted) discovery failure surfaces to the caller. */ +const discoverFailure: Thread = { + label: 'remote-mcp/discover-failure', + rules: [ + { + transform: [ + { + type: FACULTY_MESSAGE_KINDS.shell_request_result, + query: `. as $d | select($d.ok == false and (($d.ctx.echo.leg == "discover") or ($d.ctx.echo.leg == "tools")) and ($d.error.code != "credential_required") and ((${RETRYABLE_REMOTE} and (($d.ctx.echo.attempt // 0) < ${REMOTE_MCP_MAX_ATTEMPTS})) | not)) | { id: $d.ctx.echo.source, ok: false, error: { code: $d.error.code, message: $d.error.message, remoteCode: $d.error.remoteCode } }`, + target: REMOTE_MCP_EVENT_TYPES.discovered, + detailSchema: RESULT_DETAIL_GATE, + }, + ], + }, + ], +} + +/** + * vend-failure — the security faculty echoes the request ctx on FAILED vends, + * so a pack call whose vend fails surfaces the typed absent-credential error + * to the caller (no pending-forever wait). Direct (non-pack) callers carry + * no pack ctx — their vend failures never surface a pack result. + */ +const vendFailure: Thread = { + label: 'remote-mcp/vend-failure', + rules: [ + { + transform: [ + { + type: FACULTY_MESSAGE_KINDS.credential_result, + query: `. as $d | select($d.ok == false and (($d.ctx.echo.ctx.echo.leg // "") == "call")) | { id: $d.ctx.echo.ctx.echo.source, ok: false, error: { code: "error", message: $d.error.message } }`, + target: REMOTE_MCP_EVENT_TYPES.callResult, + detailSchema: { + type: 'object', + properties: { + id: { type: 'string', minLength: 1 }, + ok: { type: 'boolean' }, + error: { type: 'object' }, + ctx: { type: 'object' }, + }, + required: ['id', 'ok', 'error'], + }, + }, + ], + }, + ], +} + +/** The remote-mcp thread library — add to the program alongside the shell faculty. */ +export const remoteMcpThreads: Thread[] = [ + discoverIssue, + toolsIssue, + register, + callIssue, + elicitation, + callResult, + retryIssue, + roundCap, + retry, + callFailure, + discoverFailure, + vendFailure, +] diff --git a/src/faculties/shell/rpc-auth.threads.ts b/src/faculties/shell/rpc-auth.threads.ts index 454259d1b..3a6e107d9 100644 --- a/src/faculties/shell/rpc-auth.threads.ts +++ b/src/faculties/shell/rpc-auth.threads.ts @@ -55,7 +55,10 @@ const RPC_AUTH_RESULT_DETAIL = { * requestor — a `credential_required` shell result (first attempt only: the * gate requires no `authToken`, bounding the loop on a post-vend 401) * requests a credential for the call's server URL, carrying the original - * request out-of-band in `ctx.echo` for the replay join. + * request — and its own out-of-band `ctx` join lane — in `ctx.echo` for the + * replay join. Serves BOTH paths with one gate: the declarative one (the op + * short-circuits `auth: true` before calling) and the reactive one (a 401 + * challenge on an unauthenticated call maps to the same typed result). */ const credRequestor: Thread = { label: 'rpc-auth/requestor', @@ -65,7 +68,7 @@ const credRequestor: Thread = { { type: FACULTY_MESSAGE_KINDS.shell_request_result, query: - '. as $d | select($d.ok == false and $d.error.code? == "credential_required" and $d.error.request.input.auth == true and ($d.error.request.input.authToken == null)) | {id: ($d.id + "-cred"), input: {serverUrl: $d.error.request.input.url}, ctx: {echo: {id: $d.id, input: $d.error.request.input}}}', + '. as $d | select($d.ok == false and $d.error.code? == "credential_required" and ($d.error.request.input.authToken == null)) | {id: ($d.id + "-cred"), input: {serverUrl: $d.error.request.input.url}, ctx: {echo: {id: $d.id, input: $d.error.request.input, ctx: $d.ctx}}}', target: FACULTY_MESSAGE_KINDS.credential_request, detailSchema: RPC_AUTH_RESULT_DETAIL, }, @@ -77,7 +80,8 @@ const credRequestor: Thread = { /** * replayer — a vended `credential_result` carrying the echoed request * replays the original `shell_request` with the bearer merged into the - * input. The op proceeds with the token; the op itself never knew OAuth. + * input and the request's `ctx` join lane restored. The op proceeds with + * the token; the op itself never knew OAuth. */ const credReplayer: Thread = { label: 'rpc-auth/replayer', @@ -87,7 +91,7 @@ const credReplayer: Thread = { { type: FACULTY_MESSAGE_KINDS.credential_result, query: - '. as $d | select($d.ok == true and ($d.result.echo != null)) | {id: $d.result.echo.id, input: ($d.result.echo.input + {authToken: $d.result.token})}', + '. as $d | select($d.ok == true and ($d.result.echo != null)) | {id: $d.result.echo.id, input: ($d.result.echo.input + {authToken: $d.result.token}), ctx: $d.result.echo.ctx}', target: FACULTY_MESSAGE_KINDS.shell_request, detailSchema: { type: 'object', diff --git a/src/faculties/shell/rpc.client.ts b/src/faculties/shell/rpc.client.ts index 5fd184ecb..d2867677e 100644 --- a/src/faculties/shell/rpc.client.ts +++ b/src/faculties/shell/rpc.client.ts @@ -43,6 +43,8 @@ export type SendInput = { id?: string | number /** Optional bearer-token vendor, consulted per call. */ getAuthToken?: GetAuthToken + /** Caller-supplied headers (transport stamps; the vended bearer always rides last). */ + headers?: Record /** Injectable transport. Defaults to the platform `fetch`. */ fetch?: typeof fetch /** Abort signal for the in-flight POST — cancellation rides the transport. */ @@ -95,17 +97,20 @@ export const send = async ({ params, id = crypto.randomUUID(), getAuthToken, + headers, fetch: fetchImpl = fetch, signal, }: SendInput): Promise => { const token = getAuthToken === undefined ? undefined : await getAuthToken().catch(() => undefined) - const headers: Record = { 'content-type': 'application/json', accept: 'application/json' } - if (token !== undefined) headers.authorization = `Bearer ${token}` + const base: Record = { 'content-type': 'application/json', accept: 'application/json' } + // Caller stamps ride the base; the vended bearer rides last (it always wins). + const requestHeaders: Record = { ...base, ...(headers ?? {}) } + if (token !== undefined) requestHeaders.authorization = `Bearer ${token}` let response: Response try { response = await fetchImpl(url, { method: 'POST', - headers, + headers: requestHeaders, body: JSON.stringify(envelope({ method, params, id })), ...(signal === undefined ? {} : { signal }), }) diff --git a/src/faculties/shell/tests/remote-mcp.threads.spec.ts b/src/faculties/shell/tests/remote-mcp.threads.spec.ts new file mode 100644 index 000000000..691e8c5b6 --- /dev/null +++ b/src/faculties/shell/tests/remote-mcp.threads.spec.ts @@ -0,0 +1,359 @@ +import { describe, expect, test } from 'bun:test' +import { TRACE_MESSAGE_KINDS } from '../../../behavioral/behavioral.constants.ts' +import { behavioral } from '../../../behavioral/behavioral.ts' +import type { BPEvent, JsonObject, SelectionTrace, Trace } from '../../../behavioral/behavioral.types.ts' +import { FACULTY_MESSAGE_KINDS } from '../../faculties.constants.ts' +import { + REMOTE_MCP_EVENT_TYPES, + REMOTE_MCP_PROTOCOL_VERSION, + REMOTE_MCP_STORE_COLLECTION, + remoteMcpThreads, +} from '../remote-mcp.threads.ts' + +/** + * The remote-mcp thread pack against the real engine — the MCP layering over + * the generic `rpc` op: request stamping (`_meta` envelope + the + * MCP-Protocol-Version header), discovery (server/discover + tools/list → + * the store registry), execution (tools/call), the multi-round-trip + * elicitation loop (input_required → host → retry), and the bounded retry + * on retryable remote failures. + */ + +type Selected = { type: string; detail: Record | undefined } + +const runProgram = (events: BPEvent[]): Selected[] => { + const program = behavioral() + const selected: Selected[] = [] + program.useTrace((trace: Trace) => { + if (trace.kind === TRACE_MESSAGE_KINDS.selection) + selected.push({ + type: (trace as SelectionTrace).selected.type, + detail: (trace as SelectionTrace).selected.detail as Record | undefined, + }) + }) + for (const thread of remoteMcpThreads) program.addThread(thread) + for (const event of events) { + program.addThread({ label: `producer/${event.type}`, once: true, rules: [{ request: event }] }) + // addThread is inert — trigger admits one ingress event and runs one + // super-step; the second pump cascades transform re-entries. + program.trigger({ type: 'rmcp_pump', detail: {} }) + program.trigger({ type: 'rmcp_pump', detail: {} }) + program.trigger({ type: 'rmcp_pump', detail: {} }) + } + return selected +} + +const URL = 'https://mcp.example.com/mcp' + +/** A shell result for one of the pack's stamped rpc legs — the ctx echo rides. */ +const rpcResult = (id: string, source: string, leg: string, extraEcho: JsonObject, output: JsonObject): BPEvent => ({ + type: FACULTY_MESSAGE_KINDS.shell_request_result, + detail: { + id, + ok: true, + result: { output, durationMs: 5 }, + ctx: { echo: { source, url: URL, leg, attempt: 0, ...extraEcho } }, + }, +}) + +describe('remote-mcp pack — discovery', () => { + test('a discover event issues a stamped server/discover rpc op', () => { + const selected = runProgram([{ type: REMOTE_MCP_EVENT_TYPES.discover, detail: { id: 'r1', input: { url: URL } } }]) + const request = selected.find( + (s) => + s.type === FACULTY_MESSAGE_KINDS.shell_request && + (s.detail?.input as { method?: string })?.method === 'server/discover', + ) + expect(request).toBeDefined() + const detail = request?.detail as { + id?: string + label?: string + ctx?: { echo?: { source?: string; url?: string; leg?: string } } + input?: { + op?: string + url?: string + headers?: Record + params?: { _meta?: Record } + } + } + expect(detail.id).toBe('r1-discover') + expect(detail.label).toBe('remote-mcp') + expect(detail.ctx?.echo?.leg).toBe('discover') + expect(detail.input?.op).toBe('rpc') + expect(detail.input?.url).toBe(URL) + expect(detail.input?.headers?.['MCP-Protocol-Version']).toBe(REMOTE_MCP_PROTOCOL_VERSION) + expect(detail.input?.params?._meta?.['io.modelcontextprotocol/protocolVersion']).toBe(REMOTE_MCP_PROTOCOL_VERSION) + }) + + test('the discover result chains tools/list; the tools register in the store and surface', () => { + const selected = runProgram([ + { type: REMOTE_MCP_EVENT_TYPES.discover, detail: { id: 'r1', input: { url: URL } } }, + rpcResult( + 'r1-discover', + 'r1', + 'discover', + {}, + { supportedVersions: ['2026-07-28'], capabilities: { tools: {} } }, + ), + rpcResult('r1-tools', 'r1', 'tools', {}, { tools: [{ name: 'echo', description: 'echoes' }] }), + ]) + const toolsRequest = selected.find( + (s) => + s.type === FACULTY_MESSAGE_KINDS.shell_request && + (s.detail?.input as { method?: string })?.method === 'tools/list', + ) + expect(toolsRequest).toBeDefined() + const put = selected.find((s) => s.type === FACULTY_MESSAGE_KINDS.store_request) + expect(put).toBeDefined() + const input = put?.detail as { + id?: string + op?: string + input?: { collection?: string; key?: string; value?: { url?: string; tools?: Array<{ name?: string }> } } + } + expect(input.op).toBe('put') + expect(input.input?.collection).toBe(REMOTE_MCP_STORE_COLLECTION) + expect(input.input?.key).toBe(URL) + expect(input.input?.value?.tools?.[0]?.name).toBe('echo') + const surfaced = selected.find((s) => s.type === REMOTE_MCP_EVENT_TYPES.discovered) + expect(surfaced).toBeDefined() + const surfacedDetail = surfaced?.detail as { id?: string; input?: { url?: string; tools?: unknown[] } } + expect(surfacedDetail.id).toBe('r1') + expect(surfacedDetail.input?.url).toBe(URL) + }) +}) + +describe('remote-mcp pack — execution', () => { + test('a call event issues a stamped tools/call rpc op; the result surfaces', () => { + const selected = runProgram([ + { + type: REMOTE_MCP_EVENT_TYPES.call, + detail: { id: 'c1', input: { url: URL, tool: 'echo', args: { message: 'hi' } } }, + }, + rpcResult( + 'c1-call', + 'c1', + 'call', + { tool: 'echo', args: { message: 'hi' }, round: 0 }, + { content: [{ type: 'text', text: 'hi' }] }, + ), + ]) + const request = selected.find( + (s) => + s.type === FACULTY_MESSAGE_KINDS.shell_request && + (s.detail?.input as { method?: string })?.method === 'tools/call', + ) + expect(request).toBeDefined() + const detail = request?.detail as { + id?: string + input?: { params?: { name?: string; arguments?: unknown; _meta?: Record } } + } + expect(detail.id).toBe('c1-call') + expect(detail.input?.params?.name).toBe('echo') + expect(detail.input?.params?.arguments).toEqual({ message: 'hi' }) + expect(detail.input?.params?._meta?.['io.modelcontextprotocol/protocolVersion']).toBe(REMOTE_MCP_PROTOCOL_VERSION) + const result = selected.find((s) => s.type === REMOTE_MCP_EVENT_TYPES.callResult) + expect(result).toBeDefined() + const d = result?.detail as { id?: string; ok?: boolean; error?: { code?: string } } + expect(d?.ok).toBe(true) + }) + + test('an input_required result surfaces the elicitation; the response retries with the answers', () => { + const selected = runProgram([ + { + type: REMOTE_MCP_EVENT_TYPES.call, + detail: { id: 'c2', input: { url: URL, tool: 'deploy', args: { env: 'prod' } } }, + }, + rpcResult( + 'c2-call', + 'c2', + 'call', + { tool: 'deploy', args: { env: 'prod' }, round: 0 }, + { inputRequests: { confirm: { message: 'Deploy to prod?' } }, requestState: 'opaque-state-1' }, + ), + { + type: REMOTE_MCP_EVENT_TYPES.elicitationResponse, + detail: { + id: 'c2', + input: { + url: URL, + tool: 'deploy', + args: { env: 'prod' }, + round: 0, + requestState: 'opaque-state-1', + inputResponses: { confirm: { action: 'accept' } }, + }, + }, + }, + rpcResult( + 'c2-call-r1', + 'c2', + 'call', + { tool: 'deploy', args: { env: 'prod' }, round: 1 }, + { content: [{ type: 'text', text: 'deployed' }] }, + ), + ]) + const elicitation = selected.find((s) => s.type === REMOTE_MCP_EVENT_TYPES.elicitation) + expect(elicitation).toBeDefined() + const elicited = elicitation?.detail as { + id?: string + input?: { url?: string; tool?: string; requestState?: string } + } + expect(elicited.id).toBe('c2') + expect(elicited.input?.url).toBe(URL) + expect(elicited.input?.tool).toBe('deploy') + expect(elicited.input?.requestState).toBe('opaque-state-1') + const retry = selected.find( + (s) => + s.type === FACULTY_MESSAGE_KINDS.shell_request && + (s.detail?.input as { params?: Record })?.params?.inputResponses !== undefined, + ) + expect(retry).toBeDefined() + const retryDetail = retry?.detail as { + id?: string + input?: { + url?: string + method?: string + params?: { + name?: string + arguments?: unknown + inputResponses?: unknown + requestState?: string + _meta?: Record + } + } + } + // A FRESH request id per the MRTR contract; the answers + the byte-exact + // requestState echo ride the retry params. + expect(retryDetail.id).toBe('c2-call-r1') + expect(retryDetail.input?.url).toBe(URL) + expect(retryDetail.input?.method).toBe('tools/call') + expect(retryDetail.input?.params?.name).toBe('deploy') + expect(retryDetail.input?.params?.inputResponses).toEqual({ confirm: { action: 'accept' } }) + expect(retryDetail.input?.params?.requestState).toBe('opaque-state-1') + const result = selected.find((s) => s.type === REMOTE_MCP_EVENT_TYPES.callResult) + expect(result).toBeDefined() + const d = result?.detail as { id?: string; ok?: boolean } + expect(d?.ok).toBe(true) + }) + + test('the MRTR round cap exhausts as a typed round_cap error', () => { + const selected = runProgram([ + { + type: REMOTE_MCP_EVENT_TYPES.elicitationResponse, + detail: { + id: 'c3', + input: { url: URL, tool: 'deploy', args: {}, round: 2, requestState: 's', inputResponses: {} }, + }, + }, + ]) + const result = selected.find((s) => s.type === REMOTE_MCP_EVENT_TYPES.callResult) + expect(result).toBeDefined() + const d = result?.detail as { id?: string; ok?: boolean; error?: { code?: string } } + expect(d?.ok).toBe(false) + expect(d?.error?.code).toBe('round_cap') + }) +}) + +describe('remote-mcp pack — retry', () => { + test('a retryable remote failure re-requests the op with the attempt advanced', () => { + const selected = runProgram([ + { + type: FACULTY_MESSAGE_KINDS.shell_request_result, + detail: { + id: 'c4-call', + ok: false, + ctx: { echo: { source: 'c4', url: URL, tool: 'x', args: {}, leg: 'call', round: 0, attempt: 0 } }, + error: { code: 'error', remoteCode: 503, message: 'HTTP 503', durationMs: 2 }, + }, + }, + ]) + const retry = selected.find((s) => s.type === FACULTY_MESSAGE_KINDS.shell_request) + expect(retry).toBeDefined() + const detail = retry?.detail as { + id?: string + ctx?: { echo?: { attempt?: number } } + input?: { method?: string; url?: string } + } + expect(detail.id).toBe('c4-call') + expect(detail.ctx?.echo?.attempt).toBe(1) + expect(detail.input?.method).toBe('tools/call') + }) + + test('the attempt bound exhausts; the failure surfaces to the caller', () => { + const selected = runProgram([ + { + type: FACULTY_MESSAGE_KINDS.shell_request_result, + detail: { + id: 'c5-call', + ok: false, + ctx: { echo: { source: 'c5', url: URL, tool: 'x', args: {}, leg: 'call', round: 0, attempt: 2 } }, + error: { code: 'error', remoteCode: 503, message: 'HTTP 503', durationMs: 2 }, + }, + }, + ]) + expect(selected.some((s) => s.type === FACULTY_MESSAGE_KINDS.shell_request)).toBe(false) + const result = selected.find((s) => s.type === REMOTE_MCP_EVENT_TYPES.callResult) + expect(result).toBeDefined() + const d = result?.detail as { ok?: boolean; error?: { remoteCode?: number } } + expect(d?.ok).toBe(false) + expect(d?.error?.remoteCode).toBe(503) + }) + + test('a non-retryable failure surfaces without a retry', () => { + const selected = runProgram([ + { + type: FACULTY_MESSAGE_KINDS.shell_request_result, + detail: { + id: 'c6-call', + ok: false, + ctx: { echo: { source: 'c6', url: URL, tool: 'x', args: {}, leg: 'call', round: 0, attempt: 0 } }, + error: { code: 'error', remoteCode: 400, message: 'bad request', durationMs: 2 }, + }, + }, + ]) + expect(selected.some((s) => s.type === FACULTY_MESSAGE_KINDS.shell_request)).toBe(false) + expect(selected.some((s) => s.type === REMOTE_MCP_EVENT_TYPES.callResult)).toBe(true) + }) + + test('a failed vend echoes the request ctx — the pack surfaces the absent credential', () => { + const selected = runProgram([ + { + type: FACULTY_MESSAGE_KINDS.credential_result, + detail: { + id: 'c7-call-cred', + ok: false, + ctx: { + echo: { + id: 'c7-call', + input: {}, + ctx: { echo: { source: 'c7', url: URL, leg: 'call', round: 0, attempt: 0 } }, + }, + }, + error: { code: 'error', message: 'no credential available' }, + }, + }, + ]) + const result = selected.find((s) => s.type === REMOTE_MCP_EVENT_TYPES.callResult) + expect(result).toBeDefined() + const d = result?.detail as { id?: string; ok?: boolean; error?: { message?: string } } + expect(d?.ok).toBe(false) + expect(d?.error?.message).toContain('no credential') + }) + + test('a failed vend for a non-pack caller never surfaces a pack result', () => { + // A direct (declarative) rpc caller's vend failure carries no pack ctx — + // the derived leg is not "call", so no pack-owned result fires. + const selected = runProgram([ + { + type: FACULTY_MESSAGE_KINDS.credential_result, + detail: { + id: 'direct-1-cred', + ok: false, + ctx: { echo: { id: 'direct-1', input: {}, ctx: null } }, + error: { code: 'error', message: 'no credential available' }, + }, + }, + ]) + expect(selected.some((s) => s.type === REMOTE_MCP_EVENT_TYPES.callResult)).toBe(false) + }) +}) diff --git a/src/faculties/shell/tests/rpc-auth.threads.spec.ts b/src/faculties/shell/tests/rpc-auth.threads.spec.ts index 1d639f7ba..54de7dcdb 100644 --- a/src/faculties/shell/tests/rpc-auth.threads.spec.ts +++ b/src/faculties/shell/tests/rpc-auth.threads.spec.ts @@ -50,7 +50,7 @@ const credentialRequired = (id: string, url: string, extraInput: JsonObject = {} }, }) -const vended = (credId: string, echo: { id: string; input: JsonObject }): BPEvent => ({ +const vended = (credId: string, echo: { id: string; input: JsonObject; ctx?: JsonObject }): BPEvent => ({ type: FACULTY_MESSAGE_KINDS.credential_result, detail: { id: credId, ok: true, result: { token: 'vended-1', echo } }, }) @@ -63,27 +63,68 @@ describe('rpc auth threads — the vend-and-replay spine', () => { const detail = request?.detail as { id?: string input?: { serverUrl?: string } - ctx?: { echo?: { id?: string; input?: Record } } + ctx?: { echo?: { id?: string; input?: Record; ctx?: unknown } } } expect(detail.id).toBe('c1-cred') expect(detail.input?.serverUrl).toBe('https://mcp.example.com/mcp') expect(detail.ctx?.echo).toEqual({ id: 'c1', input: { op: 'rpc', url: 'https://mcp.example.com/mcp', auth: true, method: 'tools/list' }, + ctx: null, }) }) - test('the vended credential replays the call with the bearer merged in', () => { + test('a remote 401 challenge (no auth flag) also requests a credential — the reactive path', () => { + // The pack's issued rpc ops carry ctx but no auth flag: the op maps a + // 401-on-unauthenticated-call to credential_required, so the seam serves + // both the declarative and the reactive path with one gate. const selected = runProgram([ - vended('c2-cred', { id: 'c2', input: { op: 'rpc', url: 'https://mcp.example.com/mcp', auth: true } }), + { + type: FACULTY_MESSAGE_KINDS.shell_request_result, + detail: { + id: 'c1r-call', + ok: false, + ctx: { echo: { source: 'c1r', url: 'https://mcp.example.com/mcp', leg: 'call', attempt: 0 } }, + error: { + code: 'credential_required', + durationMs: 3, + message: 'credential required for https://mcp.example.com/mcp', + request: { op: 'rpc', input: { op: 'rpc', url: 'https://mcp.example.com/mcp', method: 'tools/call' } }, + }, + }, + }, + ]) + const request = selected.find((s) => s.type === FACULTY_MESSAGE_KINDS.credential_request) + expect(request).toBeDefined() + const detail = request?.detail as { id?: string; ctx?: { echo?: { ctx?: unknown } } } + expect(detail.id).toBe('c1r-call-cred') + // The echoed ctx preserves the pack's join payload through the vend. + expect(detail.ctx?.echo?.ctx).toEqual({ + echo: { source: 'c1r', url: 'https://mcp.example.com/mcp', leg: 'call', attempt: 0 }, + }) + }) + + test('the vended credential replays the call with the bearer merged in and ctx restored', () => { + const selected = runProgram([ + vended('c2-cred', { + id: 'c2', + input: { op: 'rpc', url: 'https://mcp.example.com/mcp', auth: true }, + ctx: { echo: { source: 'c2', leg: 'call', round: 0, attempt: 0 } }, + }), ]) const replay = selected.find((s) => s.type === FACULTY_MESSAGE_KINDS.shell_request) expect(replay).toBeDefined() - const detail = replay?.detail as { id?: string; input?: { authToken?: string; auth?: boolean; url?: string } } + const detail = replay?.detail as { + id?: string + ctx?: unknown + input?: { authToken?: string; auth?: boolean; url?: string } + } expect(detail.id).toBe('c2') expect(detail.input?.authToken).toBe('vended-1') expect(detail.input?.auth).toBe(true) expect(detail.input?.url).toBe('https://mcp.example.com/mcp') + // The pack's join payload survives the vend round-trip. + expect(detail.ctx).toEqual({ echo: { source: 'c2', leg: 'call', round: 0, attempt: 0 } }) }) test('an absent credential never replays — the caller keeps the credential_required error', () => { diff --git a/src/faculties/shell/tests/rpc-op.spec.ts b/src/faculties/shell/tests/rpc-op.spec.ts index cd5634efc..29cdcbf6f 100644 --- a/src/faculties/shell/tests/rpc-op.spec.ts +++ b/src/faculties/shell/tests/rpc-op.spec.ts @@ -14,8 +14,9 @@ import { spawnFaculty } from '../../tests/faculty-harness.ts' type WireResult = { id: string ok: boolean + ctx?: unknown result?: { output?: JsonObject; durationMs?: number } - error?: { code?: string; message?: string; remoteCode?: number | string } + error?: { code?: string; message?: string; remoteCode?: number | string; request?: { input?: { url?: string } } } space?: string } @@ -190,4 +191,67 @@ describe('shell rpc op', () => { expect(result.ok).toBe(true) expect(server.requests[0]?.headers.authorization).toBe('Bearer vended-tok') }) + + test('a remote 401 on an unauthenticated call is reactive credential_required data', async () => { + const server = rpcServer(() => new Response('unauthorized', { status: 401 })) + servers.push(server) + const worker = spawnShellWorker() + workers.push(worker) + worker.call({ id: 'rpc10', input: { op: 'rpc', url: server.url, method: 'tools/list' } }) + const raw = await worker.resultFor('rpc10') + const result = wire(raw) + // The 401 challenge maps to the typed vend-and-replay capture payload — + // the seam vends and replays; a token'd 401 stays a remote error. + expect(result.ok).toBe(false) + expect(result.error?.code).toBe('credential_required') + const error = result.error as { request?: { input?: { url?: string } } } + expect(error.request?.input?.url).toBe(server.url) + }) + + test("a remote 401 on a token'd call stays a remote error — the loop is bounded", async () => { + const server = rpcServer(() => new Response('unauthorized', { status: 401 })) + servers.push(server) + const worker = spawnShellWorker() + workers.push(worker) + worker.call({ + id: 'rpc11', + input: { op: 'rpc', url: server.url, method: 'tools/list', authToken: 'stale-tok' }, + }) + const raw = await worker.resultFor('rpc11') + const result = wire(raw) + expect(result.ok).toBe(false) + expect(result.error?.code).toBe('error') + expect(result.error?.remoteCode).toBe(401) + }) + + test('request ctx echoes on the result — the thread join lane', async () => { + const server = rpcServer((body) => Response.json({ jsonrpc: '2.0', id: body.id, result: {} })) + servers.push(server) + const worker = spawnShellWorker() + workers.push(worker) + worker.call({ + id: 'rpc12', + input: { op: 'rpc', url: server.url, method: 'x' }, + ctx: { echo: { leg: 'call' } }, + }) + const raw = await worker.resultFor('rpc12') + const result = wire(raw) + expect(result.ok).toBe(true) + expect(result.ctx).toEqual({ echo: { leg: 'call' } }) + }) + + test('op-supplied headers ride the fetch', async () => { + const server = rpcServer((body) => Response.json({ jsonrpc: '2.0', id: body.id, result: {} })) + servers.push(server) + const worker = spawnShellWorker() + workers.push(worker) + worker.call({ + id: 'rpc13', + input: { op: 'rpc', url: server.url, method: 'x', headers: { 'MCP-Protocol-Version': '2026-07-28' } }, + }) + const raw = await worker.resultFor('rpc13') + const result = wire(raw) + expect(result.ok).toBe(true) + expect(server.requests[0]?.headers['mcp-protocol-version']).toBe('2026-07-28') + }) }) diff --git a/src/faculties/shell/types.ts b/src/faculties/shell/types.ts index 71cfd50af..79dd0737c 100644 --- a/src/faculties/shell/types.ts +++ b/src/faculties/shell/types.ts @@ -117,6 +117,8 @@ export type ShellRpcOpInput = { auth?: boolean /** The vended bearer token — set by the replaying thread, never model input. */ authToken?: string + /** Op-supplied headers (e.g. the remote-mcp pack's MCP-Protocol-Version stamp). */ + headers?: Record /** Wall-clock deadline for the call. @default 30_000 */ timeoutMs?: number } @@ -171,6 +173,7 @@ export const ShellRpcOpInputSchema: JSONSchemaType = { params: { type: 'object', required: [], additionalProperties: true, nullable: true }, auth: { type: 'boolean', nullable: true }, authToken: { type: 'string', nullable: true }, + headers: { type: 'object', required: [], additionalProperties: { type: 'string' }, nullable: true }, timeoutMs: { type: 'integer', minimum: 1, nullable: true }, }, required: ['op', 'url', 'method'], diff --git a/src/faculties/tests/faculties.types.spec.ts b/src/faculties/tests/faculties.types.spec.ts index 87502c4a6..96aaf76f8 100644 --- a/src/faculties/tests/faculties.types.spec.ts +++ b/src/faculties/tests/faculties.types.spec.ts @@ -4,9 +4,9 @@ import { validateBehaviorErrorEvent, validateFrontierRequestEvent, validateFrontierRequestResultEvent, - validateMcpCancelEvent, - validateMcpRequestEvent, - validateMcpRequestResultEvent, + validateSecurityCancelEvent, + validateSecurityRequestEvent, + validateSecurityRequestResultEvent, validateShellCancelEvent, validateShellRequestEvent, validateShellRequestResultEvent, @@ -257,85 +257,32 @@ describe('workers.types event vocabulary', () => { }) }) - describe('mcp_request', () => { - test('accepts a well-formed call-tool request', () => { - const valid = validateMcpRequestEvent({ - type: FACULTY_MESSAGE_KINDS.mcp_request, - detail: { id: 'm1', op: 'call-tool', input: { url: 'http://127.0.0.1:1/mcp', tool: 'echo', args: {} } }, + describe('credential_request / credential_result / credential_cancel', () => { + test('accepts a well-formed credential request with the ctx join lane', () => { + const valid = validateSecurityRequestEvent({ + type: FACULTY_MESSAGE_KINDS.credential_request, + detail: { + id: 'sec1', + input: { serverUrl: 'https://mcp.example.com/mcp' }, + ctx: { issuer: 'https://as.example.com' }, + }, }) expect(valid).toBe(true) }) - test('accepts optional space', () => { - const valid = validateMcpRequestEvent({ - type: FACULTY_MESSAGE_KINDS.mcp_request, - detail: { id: 'm1', op: 'list-tools', input: { url: 'http://127.0.0.1:1/mcp' } }, - space: 'demo', + test('accepts a well-formed credential result', () => { + const valid = validateSecurityRequestResultEvent({ + type: FACULTY_MESSAGE_KINDS.credential_result, + detail: { id: 'sec1', ok: true, result: { token: 't' } }, }) expect(valid).toBe(true) }) - test('rejects an op outside the enum — the 7 ops are the whole surface', () => { - const valid = validateMcpRequestEvent({ - type: FACULTY_MESSAGE_KINDS.mcp_request, - detail: { id: 'm1', op: 'purge', input: {} }, - }) - expect(valid).toBe(false) - }) - test('rejects a detail without op', () => { - const valid = validateMcpRequestEvent({ - type: FACULTY_MESSAGE_KINDS.mcp_request, - detail: { id: 'm1', input: {} }, - }) - expect(valid).toBe(false) - }) - test('rejects a detail without input', () => { - const valid = validateMcpRequestEvent({ - type: FACULTY_MESSAGE_KINDS.mcp_request, - detail: { id: 'm1', op: 'list-tools' }, - }) - expect(valid).toBe(false) - }) - test('rejects ingress — routed events are synthesized, never ingress', () => { - const valid = validateMcpRequestEvent({ - type: FACULTY_MESSAGE_KINDS.mcp_request, - detail: { id: 'm1', op: 'list-tools', input: { url: 'http://127.0.0.1:1/mcp' } }, - ingress: 'ui_event', - }) - expect(valid).toBe(false) - }) - }) - - describe('mcp_request_result', () => { - test('accepts a well-formed result', () => { - const valid = validateMcpRequestResultEvent({ - type: FACULTY_MESSAGE_KINDS.mcp_request_result, - detail: { id: 'm1', ok: true, result: { status: 'completed', durationMs: 12 } }, + test('accepts a well-formed credential cancel', () => { + const valid = validateSecurityCancelEvent({ + type: FACULTY_MESSAGE_KINDS.credential_cancel, + detail: { id: 'sec1' }, }) expect(valid).toBe(true) }) - test('rejects a non-object result payload', () => { - const valid = validateMcpRequestResultEvent({ - type: FACULTY_MESSAGE_KINDS.mcp_request_result, - detail: { id: 'm1', result: 'not-an-object' }, - }) - expect(valid).toBe(false) - }) - }) - - describe('mcp_cancel', () => { - test('accepts a well-formed cancel', () => { - const valid = validateMcpCancelEvent({ - type: FACULTY_MESSAGE_KINDS.mcp_cancel, - detail: { id: 'm1' }, - }) - expect(valid).toBe(true) - }) - test('rejects a cancel without id', () => { - const valid = validateMcpCancelEvent({ - type: FACULTY_MESSAGE_KINDS.mcp_cancel, - detail: {}, - }) - expect(valid).toBe(false) - }) }) describe('faculty_error', () => { diff --git a/src/faculties/use-faculty.ts b/src/faculties/use-faculty.ts index 14f74331a..023a770a1 100644 --- a/src/faculties/use-faculty.ts +++ b/src/faculties/use-faculty.ts @@ -27,7 +27,7 @@ export type FacultyEventSchemas = { * capability faculties run as Bun.spawn PROCESSES speaking the unchanged * behavioral wire over stdio lines (one JSON event per line), one process * instance per wiring (per space), replacing the Worker model for the - * shell/store/mcp and system-one/system-two faculties. + * shell/store/security and system-one/system-two faculties. * * @remarks * Why processes over Workers (the ruling's arithmetic): a shared Worker was From 76a50672b67f5f50ce930d97751b54a9a8dc898a Mon Sep 17 00:00:00 2001 From: Edward Irby Date: Fri, 25 Sep 2026 15:31:43 -0700 Subject: [PATCH 15/55] feat: bun randomUUIDv7 for the trace-identity ids MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The id axis deserves a CSPRNG id: CodeQL flags Math.random in ueid as high severity at the engine's instanceId mint. Swap the trace-identity mints — behavioral() and the frontier faculty's replay/explore input defaults — to `bp_${randomUUIDv7()}`: UUID v7 is CSPRNG-backed and monotonic (sortable), which suits the audit/trace-ordering axis. ueid itself is untouched and keeps its other call sites — it remains the pure correlation-id helper (protocol message ids, not security); only the identity axis moves to the Bun built-in. Spec: session-id.spec.ts adds a shape-only assertion that two minted instanceIds differ and both carry the bp_ prefix. --- src/behavioral/behavioral.ts | 6 ++++-- src/behavioral/tests/session-id.spec.ts | 8 ++++++++ src/faculties/frontier/faculty.ts | 16 ++++++++-------- 3 files changed, 20 insertions(+), 10 deletions(-) diff --git a/src/behavioral/behavioral.ts b/src/behavioral/behavioral.ts index 04dde12d7..6608de4d7 100644 --- a/src/behavioral/behavioral.ts +++ b/src/behavioral/behavioral.ts @@ -1,4 +1,4 @@ -import { ueid } from '../utils.ts' +import { randomUUIDv7 } from 'bun' import { FRONTIER_STATUS, TRACE_MESSAGE_KINDS } from './behavioral.constants.ts' import { type AddThread, @@ -104,7 +104,9 @@ const createSubject = (): SendTrace => { * nothing else. */ export const behavioral = (options?: { sessionId?: string }) => { - const instanceId = ueid('bp_') + // The trace-identity axis: a CSPRNG, monotonic (sortable) UUID v7 — not the + // correlation-id `ueid`, which leans on `Math.random`. + const instanceId = `bp_${randomUUIDv7()}` /** @internal Host session identity — accepted at factory time, never minted. */ const sessionId = options?.sessionId ?? instanceId /** diff --git a/src/behavioral/tests/session-id.spec.ts b/src/behavioral/tests/session-id.spec.ts index 97f5e39fb..f9b794d3b 100644 --- a/src/behavioral/tests/session-id.spec.ts +++ b/src/behavioral/tests/session-id.spec.ts @@ -34,4 +34,12 @@ describe('session id wiring', () => { expect(trace.sessionId).toBe(instanceId) } }) + + test('two minted instanceIds are distinct but share the bp_ prefix', () => { + const first = runProgram().instanceId + const second = runProgram().instanceId + expect(first.startsWith('bp_')).toBe(true) + expect(second.startsWith('bp_')).toBe(true) + expect(first).not.toBe(second) + }) }) diff --git a/src/faculties/frontier/faculty.ts b/src/faculties/frontier/faculty.ts index 173bb4329..7178a8f82 100644 --- a/src/faculties/frontier/faculty.ts +++ b/src/faculties/frontier/faculty.ts @@ -25,6 +25,7 @@ */ import type { JSONSchemaType } from 'ajv' +import { randomUUIDv7 } from 'bun' import { FRONTIER_STATUS, TRACE_MESSAGE_KINDS } from '../../behavioral/behavioral.constants.ts' import type { BPEvent, @@ -51,7 +52,6 @@ import { resumePendingThreadsForSelectedEvent, useThread, } from '../../behavioral/behavioral.utils.ts' -import { ueid } from '../../utils.ts' import { FACULTY_MESSAGE_KINDS } from '../faculties.constants.ts' import { type FrontierRequestEvent, validateFrontierRequestEvent } from '../faculties.types.ts' import { emit, wireInbound } from '../process-lane.ts' @@ -235,7 +235,7 @@ type DeadlockFinding = { * checked for enablement at the corresponding step. * @param args.space - Optional space stamp applied to all thread rules. * @param args.instanceId - Instance id stamped on synthetic interrupt/transform - * traces emitted during resumption. Defaults to a minted `ueid('bp_')`. + * traces emitted during resumption. Defaults to a minted `bp_${randomUUIDv7()}`. * @param args.sessionId - Host session id stamped on the same traces alongside * `instanceId`. Defaults to the `instanceId` — the faculty never mints one. * @returns The replay result containing the pending set and final frontier. @@ -248,7 +248,7 @@ const replayToFrontierRaw = ({ threads, messages = [], space, - instanceId = ueid('bp_'), + instanceId = `bp_${randomUUIDv7()}`, sessionId, }: { threads: Thread[] @@ -735,7 +735,7 @@ type ExploreFrontiersArgs = { maxDepth?: number /** Space stamp applied to all thread rules. */ space?: string - /** Instance id stamped on synthetic traces. Defaults to a minted `ueid('bp_')` — pass the analyzed kernel's id to make joins natural. */ + /** Instance id stamped on synthetic traces. Defaults to a minted `bp_${randomUUIDv7()}` — pass the analyzed kernel's id to make joins natural. */ instanceId?: string /** Host session id stamped on synthetic traces alongside `instanceId`. Defaults to the `instanceId` — the faculty never mints one. */ sessionId?: string @@ -783,7 +783,7 @@ const exploreFrontiersRaw = ({ selectionPolicy = 'all-enabled', maxDepth, space, - instanceId = ueid('bp_'), + instanceId = `bp_${randomUUIDv7()}`, sessionId, }: ExploreFrontiersArgs): ExploreFrontiersResult => { if (strategy !== 'bfs' && strategy !== 'dfs') { @@ -1017,7 +1017,7 @@ export const FrontierReplayInputSchema = { instanceId: { type: 'string', nullable: true, - description: 'instance id stamped on synthetic traces; defaults to a minted ueid("bp_")', + description: 'instance id stamped on synthetic traces; defaults to a minted bp_ UUID v7', }, sessionId: { type: 'string', @@ -1126,7 +1126,7 @@ export const FrontierExploreInputSchema = { instanceId: { type: 'string', nullable: true, - description: 'instance id stamped on synthetic traces; defaults to a minted ueid("bp_")', + description: 'instance id stamped on synthetic traces; defaults to a minted bp_ UUID v7', }, sessionId: { type: 'string', @@ -1216,7 +1216,7 @@ export const FrontierVerifyInputSchema = { instanceId: { type: 'string', nullable: true, - description: 'instance id stamped on synthetic traces; defaults to a minted ueid("bp_")', + description: 'instance id stamped on synthetic traces; defaults to a minted bp_ UUID v7', }, sessionId: { type: 'string', From 186a486bbd763ddc2a49b0f7aa509bad2cf56204 Mon Sep 17 00:00:00 2001 From: Edward Irby Date: Fri, 25 Sep 2026 15:36:44 -0700 Subject: [PATCH 16/55] =?UTF-8?q?feat:=20hello-with-id=20on=20attach=20?= =?UTF-8?q?=E2=80=94=20the=20host=20speaks=20first?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit attachTui learned the instance id from the first received trace, so on a fresh idle instance the "attached to running instance " notice lagged indefinitely. The host now says hello first. - socket-host: on every new client connection the host sends one connection-scoped `hello` notification carrying the engine identity (`{ method: 'hello', params: { instanceId, sessionId } }` — the carrier's method/params envelope is the existing outbound contract; type/detail map onto it) before any trace traffic. Not an engine event: nothing enters the engine, nothing triggers a super-step. `HelloDetailSchema` (AJV) is the one schema home at the boundary — the host validates before it sends (fail closed) and attach clients validate on receipt. - bProgram exposes the engine identity (`identity`) and HostRuntime carries it. behavioral()'s frozen return gains `instanceId` — the per-process id the engine self-mints and no caller holds; session ids stay accept-only per the Surface/B decision (never minted, never returned). - attach: onAttach resolves from the hello, the single home for the id handshake; the trace-derived fallback is gone. Specs: socket-host.spec proves the hello is per-connection, precedes trace traffic, enters nothing into the engine, and fails closed on a malformed identity; the two-process lifecycle spec proves an idle instance's notice is immediate with the correct id and that a subsequent attach prints exactly once. --- src/behavioral/behavioral.ts | 8 ++++ src/cli/attach.ts | 20 +++++--- src/cli/b-program.ts | 12 ++++- src/cli/serve.ts | 5 +- src/cli/socket-host.ts | 32 ++++++++++++- src/cli/tests/attach-or-start.spec.ts | 30 ++++++++++++ src/cli/tests/fixtures/attach-instance.ts | 13 +++++- src/cli/tests/serve.spec.ts | 1 + src/cli/tests/socket-host-gui.spec.ts | 1 + src/cli/tests/socket-host.spec.ts | 57 ++++++++++++++++++++++- 10 files changed, 167 insertions(+), 12 deletions(-) diff --git a/src/behavioral/behavioral.ts b/src/behavioral/behavioral.ts index 6608de4d7..9a18893b0 100644 --- a/src/behavioral/behavioral.ts +++ b/src/behavioral/behavioral.ts @@ -403,5 +403,13 @@ export const behavioral = (options?: { sessionId?: string }) => { /** Hook to subscribe to internal state traces for monitoring/debugging. */ useTrace, step: () => step(), + /** + * The per-process identity the engine self-mints and stamps on every + * trace. Exposed so a host can hand the identity to its clients (the + * attach hello) without sniffing the trace wire — which on an idle + * instance is silent. Session ids stay accept-only: never minted, + * never returned (the host layer owns session identity policy). + */ + instanceId, }) } diff --git a/src/cli/attach.ts b/src/cli/attach.ts index 3259999e9..2443a26e9 100644 --- a/src/cli/attach.ts +++ b/src/cli/attach.ts @@ -1,9 +1,10 @@ import type { Trace } from '../behavioral/behavioral.types.ts' +import { validateHelloDetail } from './socket-host.ts' import { createTui, TRACE_KIND_COLORS } from './tui.ts' /** How the attach loop ended. */ export type AttachResult = { - /** The engine's self-minted instance id, learned from the trace wire. */ + /** The engine's self-minted instance id, learned from the connection hello. */ instanceId?: string reason: 'stdin-ended' | 'socket-closed' } @@ -32,7 +33,7 @@ export const attachTui = async ({ socketPath: string input?: NodeJS.ReadableStream write?: (text: string) => void - /** Called once, with the instance id from the first received trace. */ + /** Called once, with the instance id from the connection hello. */ onAttach?: (instanceId: string) => void }): Promise => { const tui = createTui({ input, write }) @@ -68,12 +69,19 @@ export const attachTui = async ({ } catch { return } + // The hello is the ONE home for the id handshake: the host sends it on + // connect, before any trace traffic, so even a fresh idle instance + // identifies itself immediately. Validated at the trust boundary. + if (frame.method === 'hello') { + if (validateHelloDetail(frame.params)) { + const { instanceId: id } = frame.params as { instanceId: string } + instanceId = id + onAttach?.(instanceId) + } + return + } if (frame.method !== 'trace') return const trace = frame.params as Trace - if (instanceId === undefined && typeof trace?.instanceId === 'string') { - instanceId = trace.instanceId - onAttach?.(instanceId) - } // MINIMAL: trace lines render as compact JSON; richer rendering rides // the ui_* producers slice (upgrade path: a per-kind line formatter). tui.emit(JSON.stringify(trace), TRACE_KIND_COLORS[trace.kind]) diff --git a/src/cli/b-program.ts b/src/cli/b-program.ts index fd47eb562..7afad5f29 100644 --- a/src/cli/b-program.ts +++ b/src/cli/b-program.ts @@ -119,7 +119,16 @@ export const bProgram = ({ // ── The engine, in-process ──────────────────────────────────────────────── - const { addThread, step, trigger, useTrace } = behavioral() + const { addThread, step, trigger, useTrace, instanceId } = behavioral() + /** + * The identity handoff: the engine's self-minted per-process id, with the + * resolved session id. The composition supplies no host session id today, + * so the engine's `sessionId ?? instanceId` default makes the two equal — + * when a host session id reaches this composition it must flow into + * `behavioral({ sessionId })` AND into this pair (one home for the id + * handshake). + */ + const identity = { instanceId, sessionId: instanceId } /** The in-process re-entry law: addThread + the trailing step. */ const addThreads = (threads: Thread[]): void => { @@ -295,6 +304,7 @@ export const bProgram = ({ trigger, useTrace, start, + identity, terminate: (): void => { bindEmit(null) shell.terminate() diff --git a/src/cli/serve.ts b/src/cli/serve.ts index 07f260dd5..c8f1f8238 100644 --- a/src/cli/serve.ts +++ b/src/cli/serve.ts @@ -7,12 +7,15 @@ import { createJsonRpcServer, type JsonRpcMessage, type JsonRpcServer } from './ import { loadConfig } from './load-config.ts' import { collectSecretValues, createTraceConsumer, traceLogSink } from './trace-consumer.ts' +/** The engine identity a host hands to its clients — the hello's payload. */ +export type RuntimeIdentity = { instanceId: string; sessionId: string } + /** * The host's runtime surface — a narrow view of {@link bProgram}'s handle. * * @public */ -export type HostRuntime = Pick, 'trigger' | 'useTrace' | 'start' | 'terminate'> +export type HostRuntime = Pick, 'trigger' | 'useTrace' | 'start' | 'terminate' | 'identity'> /** * Map one inbound JSON-RPC message onto the engine — the ONE host-side diff --git a/src/cli/socket-host.ts b/src/cli/socket-host.ts index 74ff9db2b..7a138b684 100644 --- a/src/cli/socket-host.ts +++ b/src/cli/socket-host.ts @@ -1,9 +1,11 @@ import { join } from 'node:path' +import type { JSONSchemaType } from 'ajv' import type { ServerWebSocket } from 'bun' +import { ajv } from '../behavioral/behavioral.types.ts' import { bundleController, CONNECT_BEHAVIORAL_ROUTE } from '../controller/bundle-controller.ts' import { behavioralHome } from '../faculties/behavioral-home.ts' import type { JsonRpcMessage } from './json-rpc.ts' -import { dispatchToRuntime, type HostRuntime, wireRuntimeEgress } from './serve.ts' +import { dispatchToRuntime, type HostRuntime, type RuntimeIdentity, wireRuntimeEgress } from './serve.ts' /** * The instance socket — `/instance.sock`, the attach lane. @@ -31,6 +33,23 @@ export type SocketHost = { close: () => Promise } +/** + * The hello's wire shape — the engine identity a client receives on connect. + * The one schema home for the hello boundary: the host validates before it + * sends (fail closed), and attaching clients validate on receipt. + * + * @public + */ +export const HelloDetailSchema: JSONSchemaType = { + type: 'object', + properties: { instanceId: { type: 'string' }, sessionId: { type: 'string' } }, + required: ['instanceId', 'sessionId'], + additionalProperties: false, +} + +/** Compiled once — the host's egress gate for the hello; attach clients reuse it on receipt. */ +export const validateHelloDetail = ajv.compile(HelloDetailSchema) as (value: unknown) => boolean + /** * Start the attach lane: a unix-socket `Bun.serve` over the shared host * dispatcher, with redacted traces and `ui_*` selections fanning out to every @@ -77,6 +96,17 @@ export const createSocketHost = async ({ idleTimeout: 255, open: (ws) => { clients.add(ws) + // Hello-with-id: one connection-scoped notification carrying the + // engine identity, before any trace traffic — an attacher learns the + // instance id immediately, even on a fresh idle instance. Not an + // engine event: nothing enters the engine, nothing triggers a + // super-step. A malformed identity fails closed (stderr + no hello): + // the host never sends an unvalidated frame at the boundary. + if (validateHelloDetail(runtime.identity)) { + ws.send(frame('hello', runtime.identity)) + } else { + process.stderr.write(`instance socket: runtime identity failed its schema — no hello sent\n`) + } }, message: (ws, message) => { const line = typeof message === 'string' ? message : new TextDecoder().decode(message) diff --git a/src/cli/tests/attach-or-start.spec.ts b/src/cli/tests/attach-or-start.spec.ts index 4848f2ab8..483282fe1 100644 --- a/src/cli/tests/attach-or-start.spec.ts +++ b/src/cli/tests/attach-or-start.spec.ts @@ -109,6 +109,35 @@ describe('attachOrStart — the two-process lifecycle', () => { instance.kill('SIGTERM') await instance.exited }, 40_000) + + test('an idle instance helloes the attacher — the notice is immediate, carries the right id, and prints once per attach', async () => { + const home = tempHome() + const instance = spawnLifecycleProcess('start', home) + await eventually(() => existsSync(instanceSocketPath(home)), 'instance socket') + const instanceId = (await Bun.file(join(home, 'spec-instance-id')).text()).trim() + + // Attach WITHOUT sending any trigger: the echo runtime only emits traces + // on start (before the attacher exists) and on triggers, so a notice now + // can only come from the connection hello. + const attacher = spawnLifecycleProcess('attach', home) + const attacherOut = collect(attacher.stdout) + await eventually(() => attacherOut.text().includes('attached to running instance'), 'immediate attach notice') + expect(attacherOut.text()).toContain(`attached to running instance ${instanceId}`) + expect(attacherOut.text().match(/attached to running instance/g)).toHaveLength(1) + + // A subsequent attach prints exactly once as well. + const second = spawnLifecycleProcess('attach', home) + const secondOut = collect(second.stdout) + await eventually(() => secondOut.text().includes('attached to running instance'), 'second attach notice') + expect(secondOut.text().match(/attached to running instance/g)).toHaveLength(1) + + second.stdin.end() + await second.exited + attacher.stdin.end() + await attacher.exited + instance.kill('SIGTERM') + await instance.exited + }, 40_000) }) /** A still-open readable whose buffered contents readline consumes. */ @@ -127,6 +156,7 @@ const echoRuntime = (): HostRuntime => { } const base = { instanceId, sessionId: instanceId } return { + identity: base, trigger: (event) => emit({ kind: TRACE_MESSAGE_KINDS.selection, diff --git a/src/cli/tests/fixtures/attach-instance.ts b/src/cli/tests/fixtures/attach-instance.ts index b2b9eb23d..bbc6a89e4 100644 --- a/src/cli/tests/fixtures/attach-instance.ts +++ b/src/cli/tests/fixtures/attach-instance.ts @@ -1,3 +1,4 @@ +import { join } from 'node:path' import { TRACE_MESSAGE_KINDS } from '../../../behavioral/behavioral.constants.ts' import type { SelectionTrace, Trace } from '../../../behavioral/behavioral.types.ts' import { attachOrStart } from '../../attach-or-start.ts' @@ -14,15 +15,17 @@ import type { HostRuntime } from '../../serve.ts' * under test — lock, socket host, socket TUI client, signals, cleanup — is * fully real, including the cross-process wire. */ +/** The echo runtime's minted per-process identity — minted once at module scope so the spec can record it. */ +const instanceId = Bun.randomUUIDv7() + const echoRuntime = (): HostRuntime => { - // The engine self-mints the instance id (the host is the identity authority). - const instanceId = Bun.randomUUIDv7() const listeners = new Set<(trace: Trace) => void>() const emit = (trace: Trace): void => { for (const listener of listeners) listener(trace) } const base = { instanceId, sessionId: instanceId } return { + identity: base, trigger: (event) => { const trace: SelectionTrace = { kind: TRACE_MESSAGE_KINDS.selection, @@ -48,4 +51,10 @@ const echoRuntime = (): HostRuntime => { } const mode = process.argv[2] === 'attach' ? 'attach' : 'start' +// Spec visibility: record the echo runtime's minted instance id so the +// two-process spec can assert the attach notice carries exactly this id. +const specHome = process.env.BEHAVIORAL_HOME ?? '' +if (mode === 'start' && specHome !== '') { + await Bun.write(join(specHome, 'spec-instance-id'), instanceId) +} await attachOrStart(mode === 'start' ? { createRuntime: () => echoRuntime() } : {}) diff --git a/src/cli/tests/serve.spec.ts b/src/cli/tests/serve.spec.ts index ebe649d35..ec7a04675 100644 --- a/src/cli/tests/serve.spec.ts +++ b/src/cli/tests/serve.spec.ts @@ -12,6 +12,7 @@ const fakeRuntime = () => { const listeners: Array<(trace: Trace) => void> = [] const calls = { started: 0, terminated: 0 } const runtime = { + identity: { instanceId: 'bp_serve_test', sessionId: 'bp_serve_test' }, trigger: (event: BPEvent): void => { triggers.push(event) }, diff --git a/src/cli/tests/socket-host-gui.spec.ts b/src/cli/tests/socket-host-gui.spec.ts index 9e8f63d0e..9763bee84 100644 --- a/src/cli/tests/socket-host-gui.spec.ts +++ b/src/cli/tests/socket-host-gui.spec.ts @@ -21,6 +21,7 @@ afterAll(() => { const fakeRuntime = (): HostRuntime => { const listeners = new Set<(trace: Trace) => void>() return { + identity: { instanceId: 'bp_gui_test', sessionId: 'bp_gui_test' }, trigger: () => {}, useTrace: (l) => { listeners.add(l) diff --git a/src/cli/tests/socket-host.spec.ts b/src/cli/tests/socket-host.spec.ts index 1b8d881a3..874a81148 100644 --- a/src/cli/tests/socket-host.spec.ts +++ b/src/cli/tests/socket-host.spec.ts @@ -7,11 +7,15 @@ import type { BPEvent, JsonObject, SelectionTrace, Trace } from '../../behaviora import type { ClientMessage } from '../../controller/controller.types.ts' import { createSocketHost, instanceSocketPath } from '../socket-host.ts' +/** The identity the engine stamps on every trace — the hello's payload. */ +const identity = { instanceId: 'bp_instance_test', sessionId: 'sess_test' } + /** The host's runtime surface, faked: records triggers, traces, and lifecycle calls. */ -const fakeRuntime = () => { +const fakeRuntime = (withIdentity = identity) => { const triggers: BPEvent[] = [] const listeners: Array<(trace: Trace) => void> = [] const runtime = { + identity: withIdentity, trigger: (event: BPEvent): void => { triggers.push(event) }, @@ -107,6 +111,57 @@ afterAll(() => { }) describe('createSocketHost', () => { + test('a new client is helloed with the engine identity before anything else', async () => { + const home = tempHome() + const fake = fakeRuntime() + const host = await createSocketHost({ runtime: fake.runtime, home }) + const client = await attachClient(host.path) + const hello = await client.waitFor<{ method: string; params: unknown }>( + (frame) => (frame as { method?: string }).method === 'hello', + 'hello notification', + ) + expect(hello.params).toEqual(identity) + // Connection-scoped notification, not an engine event: nothing entered + // the engine, nothing triggered a super-step. + expect(fake.triggers).toEqual([]) + client.close() + await host.close() + }) + + test('the hello is per-connection and stays first on the wire', async () => { + const home = tempHome() + const fake = fakeRuntime() + const host = await createSocketHost({ runtime: fake.runtime, home }) + const first = await attachClient(host.path) + const second = await attachClient(host.path) + fake.emit(traceOf(TRACE_MESSAGE_KINDS.idle)) + await first.waitFor((frame) => (frame as { method?: string }).method === 'trace', 'trace on client one') + await second.waitFor((frame) => (frame as { method?: string }).method === 'trace', 'trace on client two') + // Each client saw exactly one hello, and it preceded every trace frame. + for (const client of [first, second]) { + const hellos = client.frames.filter((frame) => (frame as { method?: string }).method === 'hello') + expect(hellos).toHaveLength(1) + const helloIndex = client.frames.findIndex((frame) => (frame as { method?: string }).method === 'hello') + const traceIndex = client.frames.findIndex((frame) => (frame as { method?: string }).method === 'trace') + expect(helloIndex).toBeLessThan(traceIndex) + } + first.close() + second.close() + await host.close() + }) + + test('a runtime without a well-formed identity helloes nobody', async () => { + const home = tempHome() + const fake = fakeRuntime({ instanceId: 'bp_instance_test' } as typeof identity) + const host = await createSocketHost({ runtime: fake.runtime, home }) + const client = await attachClient(host.path) + await Bun.sleep(100) + const hellos = client.frames.filter((frame) => (frame as { method?: string }).method === 'hello') + expect(hellos).toEqual([]) + client.close() + await host.close() + }) + test('a trigger request lands as an engine event and answers accepted', async () => { const home = tempHome() const fake = fakeRuntime() From 006f50af9c988c25f93d4fd6621226d3cb790095 Mon Sep 17 00:00:00 2001 From: Edward Irby Date: Fri, 25 Sep 2026 16:19:04 -0700 Subject: [PATCH 17/55] =?UTF-8?q?docs:=20retire=20the=20behavioral-tools?= =?UTF-8?q?=20skill=20=E2=80=94=20the=20code=20self-documents?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The skill was pure delegation after the ICL conversion: its SKILL.md pointed at one reference and handed every domain elsewhere (skills/plugins to skill-conventions, git/shell to the shell faculty, HTML to the controller floors, TS LSP to a future faculty). That reference, remote-mcp.md, is fully duplicated by remote-mcp.threads.ts's self-describing surface — the @packageDocumentation header plus the vocabulary exported as data (REMOTE_MCP_EVENT_TYPES, the four REMOTE_MCP_*_SCHEMA gates, REMOTE_MCP_MAX_ROUNDS). The code is the authority and equally agent-readable; the skill added prose polish only. Also drops the three fleet-era dead links in skills/behavioral that targeted files already absent from behavioral-tools/references/ (mcp-client.md, html.md, frontier.md): SKILL.md and references/controller.md, references/frontier-analysis.md. Verified: nothing in code, config, or the docs references the skill; skill-conventions checked against the scan code and stands accurate. --- skills/behavioral-tools/SKILL.md | 33 ---- .../behavioral-tools/references/remote-mcp.md | 68 ------- skills/skill-conventions/SKILL.md | 175 ------------------ 3 files changed, 276 deletions(-) delete mode 100644 skills/behavioral-tools/SKILL.md delete mode 100644 skills/behavioral-tools/references/remote-mcp.md delete mode 100644 skills/skill-conventions/SKILL.md diff --git a/skills/behavioral-tools/SKILL.md b/skills/behavioral-tools/SKILL.md deleted file mode 100644 index 501f7bfd0..000000000 --- a/skills/behavioral-tools/SKILL.md +++ /dev/null @@ -1,33 +0,0 @@ ---- -name: behavioral-tools -description: Remote MCP operations for the behavioral agent via the remote-mcp thread pack — the MCP layering over the shell faculty's generic `rpc` op (the 2026-07-28 stateless era: stamped `_meta` envelope, discovery, tools/call, the MRTR elicitation loop, bounded retry), with auth via the credential seam. The CLI tool fleet is retired: skills and plugins run through the skill-conventions skill (threads + recipes + store), git and raw shell belong to the shell faculty, HTML validation belongs to the controller floors + the classifier story , TypeScript LSP is a future faculty (TS 7.1 stable API). -license: ISC -compatibility: Requires bun and the behavioral CLI -allowed-tools: Bash ---- - -# Behavioral Tools - -Reference for the behavioral agent's compiled operator surfaces. As of the -ICL conversion, **every compiled surface is a faculty or the shell** — -the CLI tool fleet is retired: - -- **Remote MCP** is the remote-mcp thread pack over the shell faculty's - generic `rpc` op (the 2026-07-28 stateless era: stamped `_meta` envelope, - discovery, tools/call, the MRTR elicitation loop, bounded retry; auth via - the credential seam). See - [references/remote-mcp.md](references/remote-mcp.md). -- **Skills and plugins** (discovery, reading, frontmatter validation, link - extraction/validation) run through threads + the shell faculty (`bun run -`) - + the store — taught by the **skill-conventions** skill. -- **Git and raw shell** belong to the shell faculty (`shell_request`, bun-direct — - `run` op TS scripts, `shell` op Bun Shell commands). -- **HTML validation** belongs to the controller floors and the classifier - story (`src/controller/css.schemas.ts` doubles as classifier context — the - gate design is a local working doc, `.prompts/html-classifier-gate.md`). - -## Module references - -- [remote-mcp](references/remote-mcp.md) — the remote-mcp thread pack: - the rpc op layering, trusted response shapes, the ctx join lane, - MRTR, composing. diff --git a/skills/behavioral-tools/references/remote-mcp.md b/skills/behavioral-tools/references/remote-mcp.md deleted file mode 100644 index 3dbc3c8ce..000000000 --- a/skills/behavioral-tools/references/remote-mcp.md +++ /dev/null @@ -1,68 +0,0 @@ -# remote-mcp — the remote MCP thread pack - -Remote MCP is no longer a faculty. It is a **thread pack** over the shell -faculty's generic `rpc` op (`src/faculties/shell/remote-mcp.threads.ts`): -the op is transport-shaped (one stateless HTTP JSON-RPC POST per call), and -this pack is where "MCP" lives — the protocol envelope, discovery, tool -execution, the multi-round-trip elicitation loop, and bounded retry. - -The pack speaks the MCP 2026-07-28 stateless era: no handshake, no session -id — every request carries the protocol stamp in-band (`_meta` envelope: -`io.modelcontextprotocol/protocolVersion` + `clientInfo` + -`clientCapabilities`, plus the `MCP-Protocol-Version` header), and -server→client interactions arrive **in-band** as `input_required` results -(no server→client JSON-RPC channel on this revision). - -## The events - -| Event | Detail | Direction | -|-------|--------|-----------| -| `remote_mcp_discover` | `{ id, input: { url } }` | program → pack (host/config ingress) | -| `remote_mcp_discovered` | `{ id, ok, input?: { url, tools } }` or `{ id, ok: false, error }` | pack → program | -| `remote_mcp_call` | `{ id, input: { url, tool, args } }` | program → pack | -| `remote_mcp_call_result` | `{ id, ok: true, result }` or `{ id, ok: false, error }` | pack → program | -| `remote_mcp_elicitation` | `{ id, input: { url, tool, args, round, inputRequests, requestState } }` | pack → program (host surfaces) | -| `remote_mcp_elicitation_response` | the elicitation detail echoed + `inputResponses` | host → pack (ingress) | - -All of the pack's remote work rides `shell_request` events (`op: 'rpc'`) -and comes back on `shell_request_result`; registration rides -`store_request` (the `remote-mcp` collection, keyed by server URL — the -tools sit alongside the skills/plugins tenants in the shell registry). - -## The join lane: `ctx` - -Cross-event state rides the shell wire's `detail.ctx` — the out-of-band -lane beside `input` (the you.com MCP `_meta` pattern: host-supplied, -round-tripped verbatim, never a model-facing field). The pack stamps -`ctx.echo { source, url, leg, round, attempt }` on every op it issues; the -shell faculty echoes `ctx` on the result; the pack's transforms join on it. -Auth rides the credential seam (`shell/rpc-auth.threads.ts`): a remote 401 -challenge maps to the typed `credential_required`, and the seam vends -(broker first, keychain floor second — issuer-bound via `ctx.issuer`) and -replays the call with the vended bearer. - -## Trusted response shapes - -The pack AJV-validates ONLY the four responses it acts on -(`server/discover`, `tools/list`, `tools/call`, `InputRequiredResult`) — -exported from `remote-mcp.threads.ts` as `REMOTE_MCP_*_SCHEMA`. A response -failing its trusted shape silently no-matches the acting transform (the -result stays visible as an unmatched event in the frontier traces) — -fail-closed, not silently-wrong. - -## Multi-round-trip (MRTR) - -A `tools/call` result carrying the reserved `inputRequests` / `requestState` -members (at-least-one) is an `input_required` answer: the pack surfaces -`remote_mcp_elicitation`, the host answers with -`remote_mcp_elicitation_response` (echo + bare `inputResponses`), and the -pack retries `tools/call` with the answers plus a byte-exact `requestState` -echo, on a fresh request id, up to `REMOTE_MCP_MAX_ROUNDS` — the cap -exhausts as a typed `round_cap` error on `remote_mcp_call_result`. - -## Composition - -`bProgram` mounts the pack when **shell + security + store** are all on -(executor + vending leg + registry). The retired `mcp` faculty's replay -spine is gone; its capture-on-auth-required pattern lives on in the -credential seam. diff --git a/skills/skill-conventions/SKILL.md b/skills/skill-conventions/SKILL.md deleted file mode 100644 index e14ef2f07..000000000 --- a/skills/skill-conventions/SKILL.md +++ /dev/null @@ -1,175 +0,0 @@ ---- -name: skill-conventions -description: > - The ICL conventions for the behavioral agent's skill and plugin domains: - how to discover, read, validate, and compose over local skills and plugins - through threads, the shell faculty (bun run -), and the store — not fleet - tools. Covers the scan recipes (frontmatter fence-slicing, lenient - validation), the store catalog/manifest/recipe tenants, links_request - dispatching, and how to author your own piped scripts for exploratory - operations. Use when working with .agents/skills or .agents/plugins, when - composing bun run - scripts over skill data, or when wiring skill/plugin - discovery threads. -license: ISC -compatibility: Requires bun and the behavioral runtime (threads + the shell and store faculties) -allowed-tools: Bash Read ---- - -# Skill Conventions (ICL) - -This skill teaches the **context layer** for the skill and plugin domains: -discovery, reading, validation, and composition all run through **threads + -the shell faculty (`bun run -`) + the store** — there are no fleet tools for -this domain. Everything below is a convention the runtime and the model -share; the model cannot fall back on training for these mechanics, so this -document is deliberately precise. - -## The architecture in one paragraph - -Boot threads run **scan recipes** through the shell faculty; the results are -schema-gated and land in the **store** as tenants (`skills/catalog`, -`plugins/manifests`, `skill-recipes`). The model (or a host) reads those -tenants to discover what exists, fires **`links_request`** for the -contract-pinned operations, and composes its own `bun run -` scripts for -exploratory work. Skills are discovered *progressively* — tier 1 metadata -first (name/description/location from the catalog), then a skill's body on -demand, then its bundled files. - -## The store tenants - -| Collection | Key | Value | Gated by | -|------------|-----|-------|----------| -| `skills` | `catalog` | `{ skills: [record…], warnings: [string…] }` | `SKILL_CATALOG_SCHEMA` | -| `plugins` | `manifests` | `{ plugins: [manifest…], warnings: [string…] }` | `PLUGIN_MANIFESTS_SCHEMA` | -| `skill-recipes` | `extract-links` / `validate-links` | the recipe script (verbatim string) | — (written by the seeder) | - -Records are open — frontmatter fields beyond `name`/`description`/`location` -ride along verbatim. Catalogs are space-scoped (the store's PK is -`(space, collection, key)`); two spaces never see each other's tenants. -The scan re-runs at every cold boot, so catalogs are never stale. - -## Frontmatter parsing (the contract) - -A SKILL.md is markdown with a YAML frontmatter block. The parse contract: - -1. The file **must start** with `---` (plus optional trailing spaces) on the - first line. -2. The closing `---` must be alone on its own line (whitespace-trailing ok). -3. Slice out ONLY the fence content. **Never `YAML.parse` the whole file** — - the body is markdown and will not parse. -4. `YAML.parse` (from `import { YAML } from 'bun'`) the slice. It throws - `SyntaxError` on invalid YAML — catch it; unparseable means *skip with a - warning*, never a crash. -5. Multi-document YAML (`---` separators inside) returns an array — treat a - non-plain-object result as unparseable. - -### Lenient per-skill validation (the recipe rules) - -| Condition | Action | -|-----------|--------| -| Unparseable/absent frontmatter | **Skip**; warning `"Skipped skill "" at : unparseable YAML frontmatter"` | -| Missing or empty `name` / `description` | **Skip**; warning naming the field | -| `name` ≠ parent directory name | **Load anyway**; warning | -| `name` > 64 chars | **Load anyway**; warning | -| Same `name` at user and project scope | **Project wins**; warning `"project-level overrides user-level"` | - -Warnings are catalog *data* (`warnings: []`), not stderr — a skipped skill is -visible to the model, which can act on it. - -## Scan roots - -- Project skills: `/.agents/skills/` -- User skills: `/.agents/skills/` -- Plugins (both scopes): `.agents/plugins/` — each subdirectory a package - with `plugin.json` (+ optional `mcp.json`, `skills/`, `threads/`) - -Scan order: user scope first, then project, so project overrides on name -collision. Directories without a `SKILL.md`/`plugin.json` are skipped -silently. - -## Discovering: read the tenants - -Tier-1 discovery is a **store read**, not a tool call: - -```ts -// inside a bun run - script (or via a catalog_request thread) -const catalog = /* store get skills/catalog */ -catalog.skills // → [{ name, description, location, …frontmatter }] -``` - -Then progressive disclosure, tier by tier: read `location`'s file for the -full instructions (slice the frontmatter off the body — same fence rules); -list bundled files with `Bun.Glob`/`readdirSync` inside the skill directory. -`SKILL.md` is the instruction file, not a bundled resource. - -## The contract pair: links_request - -Link extraction/validation semantics are **test-pinned contracts** — do not -recompose them by hand; request them: - -``` -{ type: 'links_request', detail: { - id: '', - recipe: 'extract-links' | 'validate-links', - input: { markdown: '', rootRelative?: boolean } // rootRelative: validate only -} } -``` - -The dispatcher threads turn this into the `shell_request` (`run` op) automatically — the -recipe text is static thread data and never enters model context. The result -re-enters as the correlated `shell_request_result` with `jsonData`: - -- extract → `{ links: [{ value, text }] }` — sorted, de-duplicated, local - links only (external http/mailto and fragment-only `#` dropped) -- validate → `{ present: [...], missing: [...] }` resolved against the - faculty's `cwd`; with `rootRelative: true`, leading-`/` links resolve - against cwd, otherwise against the filesystem root (legacy default) - -Extraction order (the pinned contract): inline markdown links first (display -text from the first occurrence), then inline `` (stripped of tags), -then `` — over BOTH the raw markdown and -`HTMLRewriter(Bun.markdown.html(body))`. Escapes are honored (`\[` is not a -link opener); destinations terminate at `)` unescaped or a newline. - -## Authoring your own recipe (the ICL flavor) - -For exploratory operations over skill data, compose a `bun run -` script -yourself. The shape every built-in recipe follows: - -```ts -// bun run - reads ITS SCRIPT from stdin — so per-call data rides env: -const markdown = process.env.LINKS_INPUT ?? '' -import { YAML } from 'bun' // YAML.parse for fences -import { readdirSync } from 'node:fs' -// … do the work, dependencies-free (node: + bun builtins only — the -// shell faculty does not guarantee node_modules resolution in cwd) … -console.log(JSON.stringify(result)) // stdout = JSON, exactly one object -``` - -Conventions: read config from `process.env`; write errors as caught data -into the result; never assume a package can be imported from the cwd. Store -a script worth keeping as a recipe tenant (`skill-recipes` collection) so it -replays verbatim — zero tokens, zero variance. - -## Plugin manifests (the reader contract) - -The plugin scan carries the Agent Plugins v1 §11.3 posture: - -- **Fatal** (plugin rejected, no components discovered): wrong/missing - `$schema` (must be `https://agent-plugins.org/schemas/1.0.0/plugin.schema.json`), - missing/invalid `name` (§5.5: 1–64 chars, lowercase alnum + `-`/`.`, - no `--`/`..`), wrong metadata field types, non-object `extensions`. -- **Report-and-ignore**: unknown top-level plugin.json fields (§5.2) — the - plugin loads, with a warning. -- **Skipped with isolation**: a bad `mcp.json` server entry (siblings still - load); `mcp.json` `$schema` version ≠ plugin.json's (MCP disabled, skills - still load). §7.2.1: stdio `command` is a single token (bare name or - `./`-prefixed); `cwd` is `./`-prefixed or `${PLUGIN_ROOT}`/`${PLUGIN_DATA}` - -rooted and stays inside the root; remote `url` is absolute http(s) with - no userinfo/fragment, https unless loopback; header names are RFC 7230 - tokens without case-insensitive duplicates. -- **Ignored**: `extensions` namespaces — unread client-owned annexes; never - validated, never interpreted. - -The manifest value is the governor's admission input; this scan is a -**reader** — no gating policy lives here. From ee46993206df9fe0058d36791887e7fbc7c2d113 Mon Sep 17 00:00:00 2001 From: Edward Irby Date: Fri, 25 Sep 2026 16:50:04 -0700 Subject: [PATCH 18/55] =?UTF-8?q?feat(frontier):=20the=20add=5Fthread=20op?= =?UTF-8?q?=20=E2=80=94=20frontier-mediated=20thread=20admission?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Dynamic thread addition, structural layer. The op joins FrontierOp (replay/explore/verify) as an analysis operation: a proposed thread tuple is schema-checked against the engine's ThreadSchema home (exported ThreadSchema — derived, not mirrored), then verifyFrontiersRaw analyzes the proposal joined to the current thread set over a selection-trace prefix (deadlock + progress-spec livelock). The verdict returns as data — ok + echoed thread + status/findings/livelocks/report — the frontier stays analysis-shaped and never writes to the engine. The composition admission rides the existing route: an add_thread request routed through the frontier lane registers its id against the validated proposal (the id map is the authorization); the correlated frontier_request_result carries the verdict, and the pump admits under the re-entry law (addThread + step) iff both verdict legs are ok. Rejections are data — the requester reads the why from the verdict. MINIMAL: candidates admit immediately after validation (structural only); the systemOne blocking judge is the next slice. --- src/behavioral/behavioral.types.ts | 9 +- src/cli/b-program.ts | 42 ++++++- src/cli/tests/b-program.spec.ts | 70 ++++++++++- src/faculties/faculties.types.ts | 4 +- src/faculties/frontier/faculty.ts | 126 ++++++++++++++++++- src/faculties/frontier/tests/faculty.spec.ts | 80 ++++++++++++ src/faculties/tests/faculties.types.spec.ts | 7 ++ 7 files changed, 332 insertions(+), 6 deletions(-) diff --git a/src/behavioral/behavioral.types.ts b/src/behavioral/behavioral.types.ts index 64b17f1a5..09c9089de 100644 --- a/src/behavioral/behavioral.types.ts +++ b/src/behavioral/behavioral.types.ts @@ -290,7 +290,14 @@ export type Thread = { rules: Idioms[] } -const ThreadSchema: JSONSchemaType = { +/** + * The runtime schema mirror of {@link Thread} — the admission gate's home. + * Exported so wire consumers (the frontier's `add_thread` op) derive their + * input schemas from it instead of hand-mirroring the tuple shape. + * + * @internal + */ +export const ThreadSchema: JSONSchemaType = { type: 'object', properties: { space: { type: 'string', nullable: true }, diff --git a/src/cli/b-program.ts b/src/cli/b-program.ts index 7afad5f29..07866bfba 100644 --- a/src/cli/b-program.ts +++ b/src/cli/b-program.ts @@ -1,6 +1,7 @@ import { TRACE_MESSAGE_KINDS } from '../behavioral/behavioral.constants.ts' import { behavioral } from '../behavioral/behavioral.ts' import type { BPEvent, JsonObject, SelectionTrace, Thread, Trace } from '../behavioral/behavioral.types.ts' +import { validateThread } from '../behavioral/behavioral.types.ts' import { FACULTY_MESSAGE_KINDS } from '../faculties/faculties.constants.ts' import { eventGuardEntries, facultiesThreads, guardThreads } from '../faculties/faculties.threads.ts' import { @@ -244,7 +245,31 @@ export const bProgram = ({ gate: (event: BPEvent): boolean => systemTwo.invalidEventGate(event), }) } - route([FACULTY_MESSAGE_KINDS.frontier_request], { send: frontier.send, gate: frontier.gate }) + // The admission path — pending add_thread ids. An id registers when its + // request routes through the frontier lane (the request leg below); the + // correlated frontier_request_result carries the verdict. The map is the + // authorization: only results correlated to requests this composition + // itself routed can ever admit. A null thread (the proposal failed the + // Thread-schema gate at registration) never admits. + const pendingAdmissions = new Map() + + route([FACULTY_MESSAGE_KINDS.frontier_request], { + send: (event: BPEvent): void => { + // The request leg: register the id against the proposed thread, then + // route through the frontier dispatch. The frontier stays + // analysis-shaped — it validates and returns; the composition owns the + // write (the verdict leg, in the pump below). + const detail = event.detail as { id?: string; op?: string; input?: { thread?: unknown } } | undefined + if (detail?.op === 'add_thread' && typeof detail.id === 'string') { + pendingAdmissions.set( + detail.id, + validateThread(detail.input?.thread) ? (detail.input as { thread: Thread }).thread : null, + ) + } + frontier.send(event) + }, + gate: frontier.gate, + }) if (has('store')) { route([FACULTY_MESSAGE_KINDS.store_request], { send: (event: BPEvent): void => store.send(event), @@ -267,9 +292,24 @@ export const bProgram = ({ // ── The engine pump: traces out, gated events to their faculty lanes ───── + // The verdict leg: a frontier_request_result correlated to a pending + // add_thread id. Both verdict legs must be ok for the thread to admit under + // the re-entry law (addThread + step): the outer envelope (the analysis ran) + // and the inner verdict (it verified). The rejection is data — the + // requester reads the why from the verdict trace. useTrace((trace: Trace) => { if (trace.kind !== TRACE_MESSAGE_KINDS.selection) return const candidate = (trace as SelectionTrace).selected + if (candidate.type === FACULTY_MESSAGE_KINDS.frontier_request_result) { + const detail = candidate.detail as { id?: string; ok?: boolean; result?: { ok?: boolean } } | undefined + const id = detail?.id + if (typeof id === 'string' && pendingAdmissions.has(id)) { + const thread = pendingAdmissions.get(id) + pendingAdmissions.delete(id) + if (detail?.ok === true && thread && detail.result?.ok === true) addThreads([thread]) + } + return + } const event = { type: candidate.type, detail: candidate.detail, space: candidate.space } as BPEvent const faculty = lanes[event.type] if (faculty === undefined) return diff --git a/src/cli/tests/b-program.spec.ts b/src/cli/tests/b-program.spec.ts index 44f693792..ef2411207 100644 --- a/src/cli/tests/b-program.spec.ts +++ b/src/cli/tests/b-program.spec.ts @@ -1,6 +1,6 @@ import { describe, expect, test } from 'bun:test' import { TRACE_MESSAGE_KINDS } from '../../behavioral/behavioral.constants.ts' -import type { SelectionTrace, Trace } from '../../behavioral/behavioral.types.ts' +import type { BPEvent, JsonObject, SelectionTrace, Trace } from '../../behavioral/behavioral.types.ts' import { FACULTY_MESSAGE_KINDS } from '../../faculties/faculties.constants.ts' import { SecurityCancelEventSchema, @@ -248,6 +248,74 @@ describe('bProgram — the runtime composition', () => { } }) + describe('add_thread — the admission path', () => { + const addThreadRequest = (id: string, thread: JsonObject, extra?: JsonObject): BPEvent => ({ + type: FACULTY_MESSAGE_KINDS.frontier_request, + detail: { id, op: 'add_thread', input: { thread, maxDepth: 8, ...extra } }, + }) + + const resultDetailFor = (traces: Trace[], id: string) => { + const sel = selectionsOf(traces).find( + (t) => + t.selected.type === FACULTY_MESSAGE_KINDS.frontier_request_result && + (t.selected.detail as { id?: string }).id === id, + ) + return sel?.selected.detail as { id?: string; ok?: boolean; result?: { ok?: boolean } } | undefined + } + + test('a valid proposal is admitted: the verdict returns and the thread_added provision fires', async () => { + const { runtime, traces } = startRuntime() + try { + runtime.trigger(addThreadRequest('at1', { label: 'greeter', rules: [{ request: { type: 'ping' } }] })) + await waitForTraces(traces, (s) => + s.some( + (t) => + (t.selected.detail as { id?: string } | undefined)?.id === 'at1' && + t.selected.type === FACULTY_MESSAGE_KINDS.frontier_request_result, + ), + ) + const detail = resultDetailFor(traces, 'at1') + // The verdict is data: the frontier validated, the composition owns the write. + expect(detail?.ok).toBe(true) + expect(detail?.result?.ok).toBe(true) + const added = traces.find( + (t) => + t.kind === TRACE_MESSAGE_KINDS.thread_added && + (t as { thread?: { label?: string } }).thread?.label === 'greeter', + ) + expect(added).toBeDefined() + } finally { + runtime.terminate() + } + }) + + test('an invalid proposal is rejected data — no admission, no thread_added', async () => { + const { runtime, traces } = startRuntime() + try { + // `rules` missing: the derived Thread-schema gate rejects the whole input. + runtime.trigger(addThreadRequest('at2', { label: 'broken' })) + await waitForTraces(traces, (s) => + s.some( + (t) => + (t.selected.detail as { id?: string } | undefined)?.id === 'at2' && + t.selected.type === FACULTY_MESSAGE_KINDS.frontier_request_result, + ), + ) + const detail = resultDetailFor(traces, 'at2') + expect(detail?.ok).toBe(false) + expect( + traces.some( + (t) => + t.kind === TRACE_MESSAGE_KINDS.thread_added && + (t as { thread?: { label?: string } }).thread?.label === 'broken', + ), + ).toBe(false) + } finally { + runtime.terminate() + } + }) + }) + test('terminate kills overridden faculties too — the composition owns every process it invokes', async () => { const factory = useFaculty({ command: ['bun', 'run', 'tests/fixtures/probe.proc.ts'], diff --git a/src/faculties/faculties.types.ts b/src/faculties/faculties.types.ts index 666bbaa90..67a7a4538 100644 --- a/src/faculties/faculties.types.ts +++ b/src/faculties/faculties.types.ts @@ -114,7 +114,7 @@ export type FacultyErrorEvent = { } /** Frontier operations — its own worker faculty, like the responses client. */ -export type FrontierOp = 'replay' | 'explore' | 'verify' +export type FrontierOp = 'replay' | 'explore' | 'verify' | 'add_thread' export type FrontierRequestEvent = { type: typeof FACULTY_MESSAGE_KINDS.frontier_request @@ -426,7 +426,7 @@ export const FrontierRequestEventSchema: JSONSchemaType = type: 'object', properties: { id: { type: 'string', minLength: 1 }, - op: { type: 'string', enum: ['replay', 'explore', 'verify'] }, + op: { type: 'string', enum: ['replay', 'explore', 'verify', 'add_thread'] }, input: jsonObjectSchema, }, required: ['id', 'op', 'input'], diff --git a/src/faculties/frontier/faculty.ts b/src/faculties/frontier/faculty.ts index 7178a8f82..aae04645a 100644 --- a/src/faculties/frontier/faculty.ts +++ b/src/faculties/frontier/faculty.ts @@ -43,7 +43,7 @@ import type { Thread, Trace, } from '../../behavioral/behavioral.types.ts' -import { ajv, BPEventSchema } from '../../behavioral/behavioral.types.ts' +import { ajv, BPEventSchema, ThreadSchema } from '../../behavioral/behavioral.types.ts' import { advanceRunningToPending, computeFrontier, @@ -1241,6 +1241,84 @@ export const FrontierVerifyInputSchema = { * channel). */ +export type FrontierAddThreadInput = { + /** The single proposed thread — validated against the engine's own ThreadSchema home. */ + thread: Thread + /** The currently mounted thread set the proposal is analyzed against. Defaults to none. */ + threads?: Thread[] + /** The current trace state — a selection-trace prefix the analysis replays. Defaults to none. */ + messages?: SelectionTrace[] + maxDepth: number + progress?: string[] + space?: string + instanceId?: string + sessionId?: string +} + +export type FrontierAddThreadOutput = { + /** The verdict: true iff the proposal verified. The op does NOT admit — the composition owns the write. */ + ok: boolean + /** The proposed thread, echoed verbatim. */ + thread: Thread + status: 'verified' | 'failed' | 'truncated' + findings: FrontierVerifyOutput['findings'] + livelocks: FrontierVerifyOutput['livelocks'] + report: FrontierReport + isError?: boolean + message?: string +} + +export const FrontierAddThreadInputSchema = { + type: 'object', + properties: { + thread: ThreadSchema, + threads: { + ...threadsJsonSchema, + nullable: true, + description: 'the currently mounted thread set to analyze against', + }, + messages: { ...messagesJsonSchema, nullable: true }, + maxDepth: { + type: 'integer', + minimum: 1, + description: + 'Required. Exploration bound. Finite-state programs close their state graph and terminate before this. Never treat truncated as a pass.', + }, + progress: { + type: 'array', + items: { type: 'string' }, + nullable: true, + description: 'Event types that count as progress; cycles never selecting one are livelocks (status "failed").', + }, + space: { type: 'string', nullable: true, description: 'space stamp applied to all thread rules' }, + instanceId: { + type: 'string', + nullable: true, + description: 'instance id stamped on synthetic traces; defaults to a minted bp_ UUID v7', + }, + sessionId: { + type: 'string', + nullable: true, + description: 'host session id stamped on synthetic traces; defaults to the instanceId', + }, + }, + required: ['thread', 'maxDepth'], + additionalProperties: false, + description: + 'Validate a proposed thread for admission: schema-check the thread tuple (the engine ThreadSchema home), then structurally verify the proposal against the current thread set + trace state. Returns the verdict + analysis findings; does NOT admit — the composition owns the write.', +} as unknown as JSONSchemaType + +/** + * Validate a proposed thread for admission — the structural layer of dynamic + * thread addition. + * + * The `thread` tuple is schema-checked against the engine's own ThreadSchema + * (derived, not mirrored); then `verifyFrontiersRaw` analyzes the proposal + * joined to the current thread set over the trace prefix. The result is the + * verdict + findings as data — the frontier stays analysis-shaped and never + * writes to the engine; the composition admits on `ok`. + */ + // --------------------------------------------------------------------------- // Event dispatch — the wire surface // --------------------------------------------------------------------------- @@ -1264,6 +1342,7 @@ const postResult = ({ id, result, space }: { id: string; result: unknown; space? const validateReplayInput = ajv.compile(FrontierReplayInputSchema) const validateExploreInput = ajv.compile(FrontierExploreInputSchema) const validateVerifyInput = ajv.compile(FrontierVerifyInputSchema) +const validateAddThreadInput = ajv.compile(FrontierAddThreadInputSchema) type ToolRunner = { validate: (input: unknown) => boolean @@ -1384,6 +1463,51 @@ const OP_RUNNERS: Record = { } }, }, + add_thread: { + validate: validateAddThreadInput, + errors: () => ajv.errorsText(validateAddThreadInput.errors), + run: ({ + thread, + threads, + messages, + maxDepth, + progress, + space, + instanceId, + sessionId, + }: FrontierAddThreadInput): FrontierAddThreadOutput => { + try { + const { status, findings, report, livelocks } = verifyFrontiersRaw({ + threads: [...(threads ?? []), thread], + messages, + maxDepth, + progress, + space, + instanceId, + sessionId, + }) + return { ok: status === 'verified', thread, status, findings, report, livelocks } + } catch (err) { + return { + ok: false, + thread, + status: 'failed' as const, + findings: [], + report: { + strategy: 'bfs' as const, + selectionPolicy: 'all-enabled' as const, + visitedCount: 0, + findingCount: 0, + truncated: false, + maxDepth, + }, + livelocks: [], + isError: true, + message: (err as Error).message, + } + } + }, + }, } // The wire is the behavioral event vocabulary, validated with the shared diff --git a/src/faculties/frontier/tests/faculty.spec.ts b/src/faculties/frontier/tests/faculty.spec.ts index 9133cff82..c009f62e2 100644 --- a/src/faculties/frontier/tests/faculty.spec.ts +++ b/src/faculties/frontier/tests/faculty.spec.ts @@ -825,3 +825,83 @@ describe('frontier-verify livelock integration (real programs)', () => { } }) }) + +describe('add_thread', () => { + type AddThreadResult = { + ok: boolean + thread: Thread + status: 'verified' | 'failed' | 'truncated' + findings: Array<{ code: string }> + livelocks: Array<{ code: string }> + report: { visitedCount: number } + isError?: boolean + message?: string + } + + test('a valid proposed thread verifies: ok true, the thread echoed, the analysis attached', async () => { + const frontier = spawnFrontierWorker() + try { + const thread: Thread = { label: 'greeter', rules: [{ request: { type: 'ping' } }] } + frontier.call('a1', 'add_thread', { thread, maxDepth: 8 }) + const result = (await frontier.resultFor('a1')).result as AddThreadResult + expect(result.ok).toBe(true) + expect(result.status).toBe('verified') + expect(result.thread).toEqual(thread) + expect(result.findings).toHaveLength(0) + } finally { + frontier.terminate() + } + }) + + test('a thread failing the Thread schema is boundary-rejected with error data', async () => { + const frontier = spawnFrontierWorker() + try { + // `rules` is required by the engine's Thread schema home — the derived + // input schema rejects the whole input at the boundary. + frontier.call('a1', 'add_thread', { thread: { label: 'broken' }, maxDepth: 8 }) + const { ok, error } = await frontier.resultFor('a1') + expect(ok).toBe(false) + expect(String(error?.message)).toContain('invalid input') + } finally { + frontier.terminate() + } + }) + + test('a deadlock-producing proposal is rejected with the analysis findings', async () => { + const frontier = spawnFrontierWorker() + try { + // The mounted set blocks `ping`; the proposed thread requests `ping` and + // waits forever — the joined set deadlocks. + const threads: Thread[] = [{ label: 'blocker', once: true, rules: [{ block: [{ type: 'ping' }] }] }] + const thread: Thread = { + label: 'greeter', + rules: [{ request: { type: 'ping' } }, { waitFor: [{ type: 'never' }] }], + } + frontier.call('a1', 'add_thread', { thread, threads, maxDepth: 8 }) + const result = (await frontier.resultFor('a1')).result as AddThreadResult + expect(result.ok).toBe(false) + expect(result.status).toBe('failed') + expect(result.findings.length).toBeGreaterThanOrEqual(1) + expect(result.findings[0]!.code).toBe('deadlock') + expect(result.thread).toEqual(thread) + } finally { + frontier.terminate() + } + }) + + test('a livelocking proposal is rejected via the progress spec', async () => { + const frontier = spawnFrontierWorker() + try { + // The proposed thread loops forever selecting only `tick` — with + // progress = ['done'] that cycle never makes progress. + const thread: Thread = { label: 'spinner', rules: [{ request: { type: 'tick' } }] } + frontier.call('a1', 'add_thread', { thread, progress: ['done'], maxDepth: 8 }) + const result = (await frontier.resultFor('a1')).result as AddThreadResult + expect(result.ok).toBe(false) + expect(result.status).toBe('failed') + expect(result.livelocks.length).toBeGreaterThanOrEqual(1) + } finally { + frontier.terminate() + } + }) +}) diff --git a/src/faculties/tests/faculties.types.spec.ts b/src/faculties/tests/faculties.types.spec.ts index 96aaf76f8..625228fa1 100644 --- a/src/faculties/tests/faculties.types.spec.ts +++ b/src/faculties/tests/faculties.types.spec.ts @@ -183,6 +183,13 @@ describe('workers.types event vocabulary', () => { }) expect(valid).toBe(true) }) + test('accepts the add_thread operation — the admission path', () => { + const valid = validateFrontierRequestEvent({ + type: FACULTY_MESSAGE_KINDS.frontier_request, + detail: { id: 'fr_1', op: 'add_thread', input: { thread: { label: 't', rules: [] }, maxDepth: 8 } }, + }) + expect(valid).toBe(true) + }) test('rejects an unknown operation — frontier is its own worker, not a tool', () => { const valid = validateFrontierRequestEvent({ type: FACULTY_MESSAGE_KINDS.frontier_request, From 6174d484aeeac9403fe8355fc0f6f4c038e9bda8 Mon Sep 17 00:00:00 2001 From: Edward Irby Date: Fri, 25 Sep 2026 16:52:28 -0700 Subject: [PATCH 19/55] =?UTF-8?q?feat(frontier):=20the=20admission=20path?= =?UTF-8?q?=20end-to-end=20=E2=80=94=20candidates=20go=20live?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The e2e proof of the candidate→live transition: an add_thread verdict that verifies admits under the re-entry law (addThread + step), so the admitted thread participates in the next super-step — its requests are candidates in the frontier and select like any other event, looping for a non-once thread. No new wiring: slice 1's admission write was already the re-entry law, so the transition was inherent; this test pins the observable behavior (the full path trigger → frontier_request → verdict → thread_added → live selections). Docs: plan.md Current State + Decision Log record the landed refinements — the verdict result IS the candidate record (no new engine trace kind; the composition cannot emit engine traces), the id map is the authorization against forged results, and the op input shape (thread strict via ThreadSchema, analysis context permissive). --- src/cli/tests/b-program.spec.ts | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/src/cli/tests/b-program.spec.ts b/src/cli/tests/b-program.spec.ts index ef2411207..8aa8c2137 100644 --- a/src/cli/tests/b-program.spec.ts +++ b/src/cli/tests/b-program.spec.ts @@ -314,6 +314,22 @@ describe('bProgram — the runtime composition', () => { runtime.terminate() } }) + + test('an admitted thread goes live: its request is a candidate in the next super-step', async () => { + const { runtime, traces } = startRuntime() + try { + runtime.trigger(addThreadRequest('at3', { label: 'greeter', rules: [{ request: { type: 'ping' } }] })) + // The candidate→live transition: admission re-enters (addThread + step), + // so the greeter's request selects — the thread participates in the + // program, not just the trace log. And it keeps participating: a + // looping (non-once) greeter re-requests ping each super-step. + await waitForTraces(traces, (s) => s.some((t) => t.selected.type === 'ping')) + const pingSelections = selectionsOf(traces).filter((t) => t.selected.type === 'ping') + expect(pingSelections.length).toBeGreaterThanOrEqual(2) + } finally { + runtime.terminate() + } + }) }) test('terminate kills overridden faculties too — the composition owns every process it invokes', async () => { From 42752af9d6eadca0560cbefdeb752ee88ed89b61 Mon Sep 17 00:00:00 2001 From: Edward Irby Date: Fri, 25 Sep 2026 17:01:07 -0700 Subject: [PATCH 20/55] =?UTF-8?q?docs(design):=20the=20shipped=20DESIGN.md?= =?UTF-8?q?=20=E2=80=94=20spec-conformant,=20light-dark=20theming?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The shipped design default now lives at skills/behavioral/assets/DESIGN.md (the Agent Skills bundled-files location; it seeds /DESIGN.md via the init story per the plan's Home default/G ruling). Restructured to the Google design.md format adopted this session: flat spec-named token groups (colors / typography / rounded / spacing — no geometry wrapper, no nested dark/light objects), every dual-mode color as a single light-dark() value, derived values (the notebook dot tint) via color-mix() over the primary token instead of hardcoded rgba, the iconography line removed, and components declared omitted via the spec-native omitted key. Usage annotations stay — guidance, not definitions. Rulings Default/J + Color conventions/K in plan.md; the maintenance contract lives in the skills-docs prompt (slice 5). --- skills/behavioral/assets/DESIGN.md | 317 +++++++++++++++++++++++++++++ 1 file changed, 317 insertions(+) create mode 100644 skills/behavioral/assets/DESIGN.md diff --git a/skills/behavioral/assets/DESIGN.md b/skills/behavioral/assets/DESIGN.md new file mode 100644 index 000000000..494192703 --- /dev/null +++ b/skills/behavioral/assets/DESIGN.md @@ -0,0 +1,317 @@ +--- +version: alpha +name: Behavioral (behavioral.sh) +description: > + Dual-theme design system and technical specification for behavioral.sh: + a sovereign agent harness and execution observer framework providing + multi-agent orchestration, telemetry inspection, memory visualization, and + autonomous node supervision. Basic theming — values only; component + styling emerges from generation, not component tokens. +omitted: + - section: components + reason: "basic theming — component styling emerges from generation (the interface pack derives structure from Structural IA; these tokens carry values only)" +brand: + name: behavioral.sh + product: behavioral.sh + domain: behavioral.sh + system_classification: "Sovereign Agent Harness & Execution Observer" + logo_mark: "B{ }" + logo_mark_description: "Capital letter B followed by curly brackets with explicit double spacing between brackets" + typography_font: "Ropa Sans" + font_specimen: "https://fonts.google.com/specimen/Ropa+Sans" + font_google_url: "https://fonts.googleapis.com/css2?family=Ropa+Sans:ital@0;1&display=swap" + weight_rule: "unweighted / regular rhythm (400 weight throughout, no aggressive heavy bolding); italics supported for annotations and agent prompts" + brand_anchor_gradient: "linear-gradient(135deg, #E2BAE0 0%, #FFC6CE 100%)" + brand_anchor_tokens: "Primary Lavender (#E2BAE0) → Tertiary Rose (#FFC6CE)" + aesthetic: "Technical, disciplined, notebook-inspired with muted violet and rose accents" +surfaces_texture: + pattern: "dotted-notebook" + dot_spacing: "24px" + dot_size: "1.5px" + implementation: | + background-color: light-dark(#FCF8FC, #110D11); + background-image: radial-gradient(color-mix(in srgb, light-dark(#755576, #E2BAE0) 12%, transparent) 1.5px, transparent 1.5px); + background-size: 24px 24px; + note: > + Dual-mode values use CSS light-dark(); derived values (the dot tint) use + color-mix() over the primary token rather than hardcoded rgba — theme + edits propagate to the texture. +typography: + display-lg: + fontFamily: "'Ropa Sans', sans-serif" + fontSize: "57px" + lineHeight: "64px" + letterSpacing: "-0.25px" + fontWeight: "400" + usage: "Runtime hero metrics, primary console display numbers" + display-md: + fontFamily: "'Ropa Sans', sans-serif" + fontSize: "45px" + lineHeight: "52px" + letterSpacing: "0px" + fontWeight: "400" + usage: "Telemetry counters, large status displays" + headline-lg: + fontFamily: "'Ropa Sans', sans-serif" + fontSize: "32px" + lineHeight: "40px" + letterSpacing: "0px" + fontWeight: "400" + usage: "Harness workspace titles, primary dashboard headings" + headline-md: + fontFamily: "'Ropa Sans', sans-serif" + fontSize: "28px" + lineHeight: "36px" + letterSpacing: "0px" + fontWeight: "400" + usage: "Inspector panels, section titles, modal headers" + headline-sm: + fontFamily: "'Ropa Sans', sans-serif" + fontSize: "24px" + lineHeight: "32px" + letterSpacing: "0px" + fontWeight: "400" + usage: "Agent cluster card titles, telemetry group headers" + title-lg: + fontFamily: "'Ropa Sans', sans-serif" + fontSize: "22px" + lineHeight: "28px" + letterSpacing: "0px" + fontWeight: "400" + usage: "Omnibar URI text, top navigation titles" + title-md: + fontFamily: "'Ropa Sans', sans-serif" + fontSize: "16px" + lineHeight: "24px" + letterSpacing: "0.15px" + fontWeight: "400" + usage: "Agent node status, session descriptors, table headers" + title-sm: + fontFamily: "'Ropa Sans', sans-serif" + fontSize: "14px" + lineHeight: "20px" + letterSpacing: "0.1px" + fontWeight: "400" + usage: "Subheadings, configuration keys, panel labels" + body-lg: + fontFamily: "'Ropa Sans', sans-serif" + fontSize: "16px" + lineHeight: "24px" + letterSpacing: "0.5px" + fontWeight: "400" + usage: "Agent trace logs, long-form execution telemetry" + body-md: + fontFamily: "'Ropa Sans', sans-serif" + fontSize: "14px" + lineHeight: "20px" + letterSpacing: "0.25px" + fontWeight: "400" + usage: "Standard console outputs, event logs, task descriptions" + body-sm: + fontFamily: "'Ropa Sans', sans-serif" + fontSize: "12px" + lineHeight: "16px" + letterSpacing: "0.4px" + fontWeight: "400" + usage: "Timestamps, metadata, memory addresses, node hashes" + label-lg: + fontFamily: "'Ropa Sans', sans-serif" + fontSize: "14px" + lineHeight: "20px" + letterSpacing: "0.1px" + fontWeight: "400" + usage: "Primary CTA buttons, interactive triggers" + label-md: + fontFamily: "'Ropa Sans', sans-serif" + fontSize: "12px" + lineHeight: "16px" + letterSpacing: "0.5px" + fontWeight: "400" + usage: "Filter chips, active state pills, mode toggles" + label-sm: + fontFamily: "'Ropa Sans', sans-serif" + fontSize: "11px" + lineHeight: "16px" + letterSpacing: "0.5px" + fontWeight: "400" + usage: "Telemetry badges, pulse indicators, status tags" +colors: + background: "light-dark(#FCF8FC, #110D11)" + surface: "light-dark(#FCF8FC, #110D11)" + surface-dim: "light-dark(#DED7DD, #110D11)" + surface-bright: "light-dark(#FFFFFF, #322A31)" + surface-container-lowest: "light-dark(#FFFFFF, #000000)" + surface-container-low: "light-dark(#F7F1F6, #171217)" + surface-container: "light-dark(#F2EDF0, #1E181E)" + surface-container-high: "light-dark(#EEE7EB, #241D24)" + surface-container-highest: "light-dark(#EAE1E6, #2B232B)" + on-surface: "light-dark(#1E151A, #F0E1EC)" + on-surface-variant: "light-dark(#564353, #B4A7B2)" + outline: "light-dark(#9D8B9B, #7D727C)" + outline-variant: "light-dark(#D5C1D2, #4E454E)" + primary: "light-dark(#755576, #E2BAE0)" + on-primary: "light-dark(#FFFFFF, #422644)" + primary-container: "light-dark(#F7CEF5, #674868)" + on-primary-container: "light-dark(#2B112E, #FFD7FD)" + secondary: "light-dark(#534152, #D7BFD5)" + on-secondary: "light-dark(#FFFFFF, #4C3B4C)" + secondary-container: "light-dark(#F4DBF1, #473647)" + on-secondary-container: "light-dark(#251726, #F4DBF1)" + tertiary: "light-dark(#86505A, #FFC6CE)" + on-tertiary: "light-dark(#FFFFFF, #4F222C)" + tertiary-container: "light-dark(#FFCFD5, #86505A)" + on-tertiary-container: "light-dark(#350E17, #FFCFD5)" + error: "light-dark(#BA1A1A, #FFB4AB)" + on-error: "light-dark(#FFFFFF, #690005)" + error-container: "light-dark(#FFDAD6, #93000A)" + on-error-container: "light-dark(#410002, #FFDAD6)" +rounded: + sm: "4px" + md: "8px" + default: "12px" + lg: "16px" + xl: "24px" + full: "9999px" +spacing: + xs: "4px" + sm: "8px" + md: "16px" + lg: "24px" + xl: "32px" + xxl: "48px" + gutter: "16px" + margin: "20px" +accessibility: + wcag_status: "WCAG 2.1 AA & AAA Compliant (audited pairs below; light-dark() preserves the audited values per mode)" + audited_contrast_pairs: + dark_primary_text: "15.2:1 (AAA Pass)" + dark_secondary_text: "8.1:1 (AAA Pass)" + dark_primary_action: "6.2:1 (AA / AAA Large Pass)" + dark_tertiary_accent: "7.4:1 (AAA Pass)" + light_primary_text: "16.1:1 (AAA Pass)" + light_secondary_text: "7.9:1 (AAA Pass)" + light_primary_action: "5.8:1 (AA Pass)" + light_tertiary_accent: "6.5:1 (AA Pass)" +--- + +# behavioral.sh — Design System Specification (DESIGN.md) + +> **Source of Truth:** This specification defines the visual identity, typography, dual-theme surface tokens, dotted-notebook background texture, and accessibility benchmarks for **behavioral.sh**. Basic theming: the frontmatter tokens are the normative values; component styling emerges from generation, not from this file. + +--- + +## 1. Brand Identity & Overview + +| Attribute | Specification | +|---|---| +| **Product & Domain** | `behavioral.sh` | +| **System Classification** | Sovereign Agent Harness & Execution Observer | +| **Primary Typography** | **Ropa Sans** (`font-family: 'Ropa Sans', sans-serif`) | +| **Font Specimen** | [Google Fonts: Ropa Sans](https://fonts.google.com/specimen/Ropa+Sans) | +| **Typographic Mark** | `B{ }` (Explicit double-space within brackets in Ropa Sans) | +| **Brand Anchor Gradient** | `#E2BAE0 → #FFC6CE` (Primary Lavender to Tertiary Rose) | +| **Surface Texture** | **Dotted Notebook Grid** (24px grid spacing, 1.5px dots) | +| **Default Corner Radius** | 12px (`rounded.default`) | + +--- + +## 2. Dotted Notebook Surfaces + +Both dark console and light workspace surfaces feature a disciplined 24px mathematical dot grid, evoking engineering notebooks and telemetry coordinate spaces. + +### 2.1 Dual-Mode Surface Texture + +```css +background-color: light-dark(#FCF8FC, #110D11); +background-image: radial-gradient(color-mix(in srgb, light-dark(#755576, #E2BAE0) 12%, transparent) 1.5px, transparent 1.5px); +background-size: 24px 24px; +``` + +The dot tint derives from the primary token via `color-mix()`, so a theme edit propagates to the texture with no separate maintenance. + +--- + +## 3. Typography: Ropa Sans + +**Ropa Sans** is the single source of typographic truth across all screens. To preserve technical discipline, typography is maintained in an **unweighted / 400 regular rhythm** without aggressive bolding; italics are supported for annotations and agent prompts. + +- **Google Font Import:** + ```html + + + + ``` + +- **Universal Application Rule:** + ```css + body, button, input, select, textarea, h1, h2, h3, h4, h5, h6, p, span, div, a, label, code, pre { + font-family: 'Ropa Sans', sans-serif !important; + } + ``` + +### Type Scale (Ropa Sans) + +| Token | Size | Weight | Line Height | Tracking | Usage | +|---|---|---|---|---|---| +| `display-lg` | 57px | 400 | 64px | -0.25px | Hero telemetry metrics, primary display figures | +| `display-md` | 45px | 400 | 52px | 0px | Secondary counters, harness cycle displays | +| `headline-lg` | 32px | 400 | 40px | 0px | Workspace & harness view headers | +| `headline-md` | 28px | 400 | 36px | 0px | Inspector panels, modal headers | +| `headline-sm` | 24px | 400 | 32px | 0px | Agent node cluster card headers | +| `title-lg` | 22px | 400 | 28px | 0px | Omnibar route text, main navigation tabs | +| `title-md` | 16px | 400 | 24px | 0.15px | Node status headers, agent labels | +| `title-sm` | 14px | 400 | 20px | 0.1px | Section subheadings, telemetry keys | +| `body-lg` | 16px | 400 | 24px | 0.5px | Agent memory logs, execution traces | +| `body-md` | 14px | 400 | 20px | 0.25px | Default console logs, task descriptions | +| `body-sm` | 12px | 400 | 16px | 0.4px | Timestamps, metadata, node hashes | +| `label-lg` | 14px | 400 | 20px | 0.1px | Primary interactive buttons, CTA elements | +| `label-md` | 12px | 400 | 16px | 0.5px | Filter chips, state pills, action triggers | +| `label-sm` | 11px | 400 | 16px | 0.5px | Telemetry badges, pulse indicators | + +--- + +## 4. Dual-Theme Surface Architecture & Contrast Compliance + +### 4.1 Dark Mode (Primary Console) + +| Token | Hex | Role & Mapping | Contrast Ratio | +|---|---|---|---| +| `surface` | `#110D11` | Primary console background | 15.2:1 against text | +| `surface-container-low` | `#171217` | Inset panels, statusbars | — | +| `surface-container` | `#1E181E` | Node cards, memory clusters | — | +| `surface-container-high` | `#241D24` | Elevated telemetry modules | — | +| `surface-container-highest` | `#2B232B` | Hover states, active layers | — | +| `primary` | `#E2BAE0` | Primary brand accent & active states | 6.2:1 on container | +| `on-primary` | `#422644` | High-contrast text on primary fill | 6.2:1 (AA / AAA Large) | +| `primary-container` | `#674868` | Secondary interactive fills | — | +| `tertiary` | `#FFC6CE` | Soft rose highlights, live AI spark | 7.4:1 on container | +| `on-surface` | `#F0E1EC` | Primary console readable text | 15.2:1 (AAA Pass) | +| `on-surface-variant` | `#B4A7B2` | Subdued telemetry labels, timestamps | 8.1:1 (AAA Pass) | +| `outline` | `#7D727C` | Architectural borders | 4.8:1 (AA UI Pass) | +| `outline-variant` | `#4E454E` | Inset dividers & grid rules | — | + +### 4.2 Light Mode (Mirrored Workspace) + +| Token | Hex | Role & Mapping | Contrast Ratio | +|---|---|---|---| +| `surface` | `#FCF8FC` | Primary workspace canvas | 16.1:1 against text | +| `surface-container-low` | `#F7F1F6` | Lowered canvas regions | — | +| `surface-container` | `#F2EDF0` | Node cards, container modules | — | +| `surface-container-high` | `#EEE7EB` | Elevated inspector panels, omnibar | — | +| `surface-container-highest` | `#EAE1E6` | High-contrast active borders | — | +| `primary` | `#755576` | Solid primary buttons & brand text | 5.8:1 (AA Pass) | +| `on-primary` | `#FFFFFF` | Text on solid primary actions | 5.8:1 (AA Pass) | +| `primary-container` | `#F7CEF5` | Muted chip and highlight backgrounds | — | +| `tertiary` | `#86505A` | Deep rose accent text and badges | 6.5:1 (AA Pass) | +| `on-surface` | `#1E151A` | Primary legible dark text | 16.1:1 (AAA Pass) | +| `on-surface-variant` | `#564353` | Secondary labels & property names | 7.9:1 (AAA Pass) | +| `outline` | `#9D8B9B` | Borders & card outlines | 4.6:1 (AA UI Pass) | +| `outline-variant` | `#D5C1D2` | Inset dividers & table borders | — | + +### 4.3 Dual-Mode & Derived Values + +Frontmatter color tokens carry dual-mode values via CSS [`light-dark()`](https://developer.mozilla.org/en-US/docs/Web/CSS/Reference/Values/color_value/light-dark) — one token, one custom property, the browser resolves the mode. Derived adjustments (hover tints, texture dots, translucent variants) use [`color-mix()`](https://developer.mozilla.org/en-US/docs/Web/CSS/Reference/Values/color_value/color-mix) over base tokens, never hardcoded rgba — a theme edit propagates to every derived value. + +--- + +*behavioral.sh · Ropa Sans & Dotted Notebook Specification v3.0.0* From 4a7a904017f5b2dea1008e2f59cfe0e01b0127b9 Mon Sep 17 00:00:00 2001 From: Edward Irby Date: Fri, 25 Sep 2026 17:28:04 -0700 Subject: [PATCH 21/55] =?UTF-8?q?feat(system-one):=20the=20admission=20jud?= =?UTF-8?q?gment=20pack=20=E2=80=94=20the=20BP-native=20blocking=20judge?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The system-one faculty's thread pack (threads.ts): when systemOne is wired, a validated add_thread candidate does not admit directly — its admission is blocked while a system-one Decision judges the proposed thread, and the Decision determines whether the block lifts. Three threads. admission-issue: a thread_candidate event (the composition's emission of a validated candidate) issues the correlated system_one_request (-judge) with the thread as the Decision input (state: the thread; question: an admit/reject choice). admission-gate: the blocking judge — waits for the candidate, blocks every thread_admission while the Decision is in flight, lifts on the approve-shaped result or on the rejection event (a rejection holds the line through its own processing without poisoning later candidates), then wraps to the next candidate. admission-verdict: the judge- correlated result maps to thread_admission { id, admit: true } on an explicit approve, thread_admission_rejected { id, admit: false } on anything else — not-ok, malformed, non-admit: fail-closed, error data, never a throw. The judge answer is normalized in jq (every non-object on the answer path coalesces to {}), so a hostile payload falls through to the reject listener instead of crashing the transform. Concurrency: candidates judge independently (id-correlated); the gate's block is the type-scoped judging window, advisory when judgments overlap. Specs run the pack against the real engine: the Decision request carries the thread; the probe admission stays blocked while judging and selects only after the lift; the reject holds the line — no admission for the candidate, the rejection visible, the gate released after. --- .../system-one/tests/threads.spec.ts | 190 ++++++++++++++++ src/faculties/system-one/threads.ts | 205 ++++++++++++++++++ 2 files changed, 395 insertions(+) create mode 100644 src/faculties/system-one/tests/threads.spec.ts create mode 100644 src/faculties/system-one/threads.ts diff --git a/src/faculties/system-one/tests/threads.spec.ts b/src/faculties/system-one/tests/threads.spec.ts new file mode 100644 index 000000000..27db53828 --- /dev/null +++ b/src/faculties/system-one/tests/threads.spec.ts @@ -0,0 +1,190 @@ +import { describe, expect, test } from 'bun:test' +import { TRACE_MESSAGE_KINDS } from '../../../behavioral/behavioral.constants.ts' +import { behavioral } from '../../../behavioral/behavioral.ts' +import type { BPEvent, SelectionTrace, Thread, Trace } from '../../../behavioral/behavioral.types.ts' +import { FACULTY_MESSAGE_KINDS } from '../../faculties.constants.ts' +import { ADMISSION_EVENT_TYPES, admissionJudgmentThreads } from '../threads.ts' + +/** + * The system-one admission judgment pack against the real engine — the + * BP-native blocking judge: a validated candidate's admission is BLOCKED + * while a system-one Decision judges the proposed thread; an approval + * lifts the block (the candidate admits), a rejection holds the line + * (the candidate never goes live). + */ + +type Selected = { type: string; detail: Record | undefined } + +/** One harness step: admit a producer event, mount threads, or both — then pump. */ +type Step = { + event?: BPEvent + threads?: Thread[] + check?: (selected: Selected[]) => void +} + +const PUMP = 'judgment_pump' +const PUMPS_PER_STEP = 6 + +const runJudgment = (steps: Step[]): Selected[] => { + const program = behavioral() + const selected: Selected[] = [] + program.useTrace((trace: Trace) => { + if (trace.kind === TRACE_MESSAGE_KINDS.selection) + selected.push({ + type: (trace as SelectionTrace).selected.type, + detail: (trace as SelectionTrace).selected.detail as Record | undefined, + }) + }) + for (const thread of admissionJudgmentThreads) program.addThread(thread) + for (const step of steps) { + if (step.event !== undefined) + program.addThread({ label: `producer/${step.event.type}`, once: true, rules: [{ request: step.event }] }) + for (const thread of step.threads ?? []) program.addThread(thread) + for (let i = 0; i < PUMPS_PER_STEP; i++) program.trigger({ type: PUMP, detail: {} }) + step.check?.(selected) + } + return selected +} + +/** The candidate event: a validated proposal awaiting judgment (the composition's emission). */ +const candidate = (id: string, label: string): BPEvent => ({ + type: ADMISSION_EVENT_TYPES.candidate, + detail: { id, thread: { label, rules: [{ request: { type: 'ping' } }] } }, +}) + +/** The correlated judge result — a choice answer on the `admission` question. */ +const judgeResult = (id: string, choice: string): BPEvent => ({ + type: FACULTY_MESSAGE_KINDS.system_one_request_result, + detail: { + id: `${id}-judge`, + ok: true, + result: { model: 'jev-1.13.0', answers: { admission: { type: 'choice', choice } } }, + }, +}) + +describe('system-one admission judgment pack', () => { + test('a candidate issues a system_one_request carrying the proposed thread as the Decision input', () => { + const selected = runJudgment([ + { + event: candidate('at1', 'greeter'), + check: (selected) => { + const request = selected.find( + (s) => + s.type === FACULTY_MESSAGE_KINDS.system_one_request && + (s.detail?.id as string | undefined) === 'at1-judge', + ) + expect(request).toBeDefined() + const input = request?.detail?.input as + | { state?: { thread?: { label?: string } }; questions?: Record } + | undefined + expect(input?.state?.thread?.label).toBe('greeter') + expect(input?.questions?.admission?.type).toBe('choice') + }, + }, + ]) + // The issued request exists in the settled program too. + expect( + selected.some( + (s) => s.type === FACULTY_MESSAGE_KINDS.system_one_request && (s.detail?.id as string) === 'at1-judge', + ), + ).toBe(true) + }) + + test('approve: the block lifts on the Decision and the admission fires with the candidate id', () => { + let candidateIndex = -1 + let resultIndex = -1 + const selected = runJudgment([ + { + event: candidate('at1', 'greeter'), + check: (selected) => { + candidateIndex = selected.findIndex((s) => s.type === ADMISSION_EVENT_TYPES.candidate) + }, + }, + { + // The probe: an admission request that MUST stay blocked while the + // Decision is in flight. + threads: [ + { + label: 'probe', + once: true, + rules: [{ request: { type: ADMISSION_EVENT_TYPES.admitted, detail: { id: 'probe-1', admit: true } } }], + }, + ], + check: (selected) => { + const admissions = selected.filter((s) => s.type === ADMISSION_EVENT_TYPES.admitted) + expect(admissions).toEqual([]) + }, + }, + { + event: judgeResult('at1', 'admit'), + check: (selected) => { + resultIndex = selected.findIndex( + (s) => + s.type === FACULTY_MESSAGE_KINDS.system_one_request_result && (s.detail?.id as string) === 'at1-judge', + ) + // The judgment's outcome: the admission fires with the CANDIDATE id… + const admitted = selected.find( + (s) => s.type === ADMISSION_EVENT_TYPES.admitted && (s.detail?.id as string) === 'at1', + ) + expect(admitted).toBeDefined() + expect((admitted?.detail as { admit?: boolean } | undefined)?.admit).toBe(true) + // …the probe selects only AFTER the Decision (the block was the only + // thing holding it — it lifts with the judgment)… + const probe = selected.find( + (s) => s.type === ADMISSION_EVENT_TYPES.admitted && (s.detail?.id as string) === 'probe-1', + ) + expect(probe).toBeDefined() + // …and nothing was admitted while the judgment was in flight. + const admissions = selected + .map((s, index) => ({ s, index })) + .filter(({ s }) => s.type === ADMISSION_EVENT_TYPES.admitted) + for (const { index } of admissions) expect(index).toBeGreaterThan(resultIndex) + expect(candidateIndex).toBeLessThan(resultIndex) + }, + }, + ]) + expect(selected.length).toBeGreaterThan(0) + }) + + test('reject: the block holds through the rejection — no admission for the candidate', () => { + const selected = runJudgment([ + { event: candidate('at2', 'suspicious') }, + { + event: judgeResult('at2', 'reject'), + check: (selected) => { + // The rejection is visible, stamped with the candidate id… + const rejected = selected.find( + (s) => s.type === ADMISSION_EVENT_TYPES.rejected && (s.detail?.id as string) === 'at2', + ) + expect(rejected).toBeDefined() + expect((rejected?.detail as { admit?: boolean } | undefined)?.admit).toBe(false) + // …and no admission ever fires for the rejected candidate. + expect( + selected.some((s) => s.type === ADMISSION_EVENT_TYPES.admitted && (s.detail?.id as string) === 'at2'), + ).toBe(false) + }, + }, + { + // After the rejection is processed the gate releases (the loop wraps to + // the next candidate) — but the rejected candidate stays dead. + threads: [ + { + label: 'probe', + once: true, + rules: [{ request: { type: ADMISSION_EVENT_TYPES.admitted, detail: { id: 'probe-2', admit: true } } }], + }, + ], + check: (selected) => { + const probe = selected.find( + (s) => s.type === ADMISSION_EVENT_TYPES.admitted && (s.detail?.id as string) === 'probe-2', + ) + expect(probe).toBeDefined() + expect( + selected.some((s) => s.type === ADMISSION_EVENT_TYPES.admitted && (s.detail?.id as string) === 'at2'), + ).toBe(false) + }, + }, + ]) + expect(selected.length).toBeGreaterThan(0) + }) +}) diff --git a/src/faculties/system-one/threads.ts b/src/faculties/system-one/threads.ts new file mode 100644 index 000000000..4598d5a55 --- /dev/null +++ b/src/faculties/system-one/threads.ts @@ -0,0 +1,205 @@ +import type { Thread } from '../../behavioral/behavioral.types.ts' +import { ThreadSchema } from '../../behavioral/behavioral.types.ts' +import { FACULTY_MESSAGE_KINDS } from '../faculties.constants.ts' + +/** + * The System One faculty's admission judgment thread pack — the BP-native + * blocking judge. When systemOne is wired, a validated candidate (the + * `add_thread` op's structural verdict) does not admit directly: its + * admission is BLOCKED while a system-one Decision judges the proposed + * thread, and the Decision determines whether the block lifts. + * + * @remarks + * Three threads, mounted by `bProgram` only when systemOne is on (the judge + * requires the Decisions lane; the frontier — its structural layer — is the + * in-process embed, always present): + * + * - **`admission-issue`** — a `thread_candidate` event (the composition's + * emission: a validated candidate awaiting judgment) issues the correlated + * `system_one_request` (`-judge`) carrying the proposed thread + * as the Decision input (state: the thread; question: a choice between + * `admit` and `reject`). + * - **`admission-gate`** — the blocking judge itself. It waits for a + * candidate, then BLOCKS every `thread_admission` while the Decision is in + * flight. The block lifts only when the judgment's outcome lands: an + * approve-shaped result, or the rejection event (so a rejection holds the + * line through its own processing — the candidate never goes live — without + * poisoning later candidates' admissions). Then the loop wraps to the next + * candidate (a block-only span would hold forever; the waitFor pair is + * what releases it, per the guard-thread pattern's inverse). + * - **`admission-verdict`** — the judge-correlated result maps to the + * outcome: `thread_admission { id, admit: true }` on an approve choice, + * `thread_admission_rejected { id, admit: false }` on anything else — + * a not-ok result, a malformed answer, a non-`admit` choice. Fail-closed: + * the only road to admission is an explicit approve. + * + * The composition owns the write: `thread_admission` selections admit the + * pending candidate (the id map stays the authorization); rejected ids drop. + * Concurrent candidates each judge independently (id-correlated); the gate's + * block is the type-scoped judging window, advisory when several judgments + * overlap — per-candidate correctness lives in the correlation, never in the + * window. + * + * MINIMAL: the Decision's policy (what makes a thread "appropriate") is the + * fixed instruction below; a policy input rides a config seam when a named + * need arrives. Hostile-thread forgery of `thread_admission` is the same + * class as the existing frontier_request self-request posture — a known + * frontier for a later hardening slice. + * + * @packageDocumentation + */ + +// ── Vocabulary ─────────────────────────────────────────────────────────────── + +/** Thread-owned event types: the candidate in, the judged outcome out. */ +export const ADMISSION_EVENT_TYPES = { + candidate: 'thread_candidate', + admitted: 'thread_admission', + rejected: 'thread_admission_rejected', +} as const + +/** The judge-request correlation suffix: `-judge` ↔ the result's echoed id. */ +export const ADMISSION_JUDGE_SUFFIX = '-judge' + +/** The Decision question the judgment asks (the `admission` choice). */ +export const ADMISSION_QUESTION = 'admission' + +// ── Trusted shapes ─────────────────────────────────────────────────────────── + +/** A candidate's detail: the composition-validated proposal, keyed by the pending admission id. */ +export const ADMISSION_CANDIDATE_SCHEMA = { + type: 'object', + properties: { + id: { type: 'string', minLength: 1 }, + thread: ThreadSchema, + }, + required: ['id', 'thread'], + additionalProperties: false, +} as const + +/** + * The judge-correlated result detail the verdict listeners trust — the id + * suffix scopes the lane (other Decisions on the same faculty never match). + */ +export const ADMISSION_JUDGE_RESULT_SCHEMA = { + type: 'object', + properties: { + id: { type: 'string', pattern: '-judge$' }, + ok: { type: 'boolean' }, + }, + required: ['id', 'ok'], + additionalProperties: true, +} as const + +/** + * The approve-shaped result — the gate's lift condition. The block releases + * on an explicit `admit` choice; every other outcome (reject, error, + * malformed) holds until the rejection event releases it. + */ +export const ADMISSION_JUDGE_APPROVAL_SCHEMA = { + type: 'object', + properties: { + id: { type: 'string', pattern: '-judge$' }, + ok: { type: 'boolean', const: true }, + result: { + type: 'object', + properties: { + answers: { + type: 'object', + properties: { + [ADMISSION_QUESTION]: { + type: 'object', + properties: { choice: { type: 'string', const: 'admit' } }, + required: ['choice'], + }, + }, + required: [ADMISSION_QUESTION], + }, + }, + required: ['answers'], + }, + }, + required: ['id', 'ok', 'result'], + additionalProperties: true, +} as const + +// ── Shared jq fragments ────────────────────────────────────────────────────── + +/** + * The judge answer, normalized: every non-object on the answer path coalesces + * to `{}` so a hostile/malformed payload can never crash the queries — it + * falls through to the reject listener (fail-closed), never a jq error. + */ +const JUDGE_ANSWER = + '(if ($d.result | type) == "object" then $d.result else {} end) as $r | ' + + '(if ($r.answers | type) == "object" then $r.answers else {} end) as $a | ' + + '(if ($a.admission | type) == "object" then $a.admission else {} end) as $q' + +// ── Threads ────────────────────────────────────────────────────────────────── + +/** admission-issue — a candidate issues the correlated `system_one_request` with the thread as Decision input. */ +const admissionIssue: Thread = { + label: 'system-one/admission-issue', + rules: [ + { + transform: [ + { + type: ADMISSION_EVENT_TYPES.candidate, + query: `. as $d | select($d.thread.label != null and $d.thread.rules != null) +| { id: ($d.id + "${ADMISSION_JUDGE_SUFFIX}"), input: { + state: { lane: "admission-judgment", thread: $d.thread }, + questions: { ${ADMISSION_QUESTION}: { type: "choice", + instructions: "Admission judgment: decide whether this proposed behavioral thread should join the live program. It is already structurally valid; judge appropriateness — what it requests, waits for, and blocks, and whether that is safe and in scope.", + criteria: { admit: "The thread is appropriate to admit.", reject: "The thread is not appropriate — keep it out." } } } } }`, + target: FACULTY_MESSAGE_KINDS.system_one_request, + detailSchema: ADMISSION_CANDIDATE_SCHEMA, + }, + ], + }, + ], +} + +/** admission-gate — the blocking judge: admission is blocked while the Decision is in flight. */ +const admissionGate: Thread = { + label: 'system-one/admission-gate', + rules: [ + { waitFor: [{ type: ADMISSION_EVENT_TYPES.candidate, detailSchema: ADMISSION_CANDIDATE_SCHEMA }] }, + { + block: [{ type: ADMISSION_EVENT_TYPES.admitted }], + waitFor: [ + { type: FACULTY_MESSAGE_KINDS.system_one_request_result, detailSchema: ADMISSION_JUDGE_APPROVAL_SCHEMA }, + { type: ADMISSION_EVENT_TYPES.rejected }, + ], + }, + ], +} + +/** admission-verdict — the judge-correlated result maps to the outcome; everything but an explicit approve rejects. */ +const admissionVerdict: Thread = { + label: 'system-one/admission-verdict', + rules: [ + { + transform: [ + { + type: FACULTY_MESSAGE_KINDS.system_one_request_result, + query: `. as $d | ${JUDGE_ANSWER} +| select(($d.id | endswith("${ADMISSION_JUDGE_SUFFIX}")) and (($d.ok // false) == true) and ($q.type == "choice") and ($q.choice == "admit")) +| { id: ($d.id | sub("${ADMISSION_JUDGE_SUFFIX}$"; "")), admit: true }`, + target: ADMISSION_EVENT_TYPES.admitted, + detailSchema: ADMISSION_JUDGE_RESULT_SCHEMA, + }, + { + type: FACULTY_MESSAGE_KINDS.system_one_request_result, + query: `. as $d | ${JUDGE_ANSWER} +| select(($d.id | endswith("${ADMISSION_JUDGE_SUFFIX}")) and ((($d.ok // false) != true) or ($q.type != "choice") or ($q.choice != "admit"))) +| { id: ($d.id | sub("${ADMISSION_JUDGE_SUFFIX}$"; "")), admit: false }`, + target: ADMISSION_EVENT_TYPES.rejected, + detailSchema: ADMISSION_JUDGE_RESULT_SCHEMA, + }, + ], + }, + ], +} + +/** The admission judgment pack — mounts with systemOne (the composition wires it). */ +export const admissionJudgmentThreads: Thread[] = [admissionIssue, admissionGate, admissionVerdict] From d40f035d7306160ca236bc221e6320dfe798a50e Mon Sep 17 00:00:00 2001 From: Edward Irby Date: Fri, 25 Sep 2026 17:30:12 -0700 Subject: [PATCH 22/55] =?UTF-8?q?feat(system-one):=20the=20admission=20jud?= =?UTF-8?q?gment=20Decision=20shapes=20=E2=80=94=20one=20home=20each?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The Decision input and the judged outcome get their AJV schema homes in the pack (JSONSchemaType, the shared ajv instance), derived from the existing homes rather than mirrored: - ADMISSION_INPUT_SCHEMA — the issued system_one_request input: state { lane, thread } with the thread validating against the engine's ThreadSchema, questions.admission against the question union's choiceQuestionSchema (now exported from system-one/schemas.ts — the one question home). validateAdmissionInput keeps the issue thread's jq honest: what it emits must be exactly this shape. - ADMISSION_VERDICT_SCHEMA — the judged outcome { id, admit, reason? }. validateAdmissionVerdict is the composition's admission-gate boundary (slice 3 consumes only conforming verdicts). reason stays optional and absent today: the choice answer carries no prose, so the mapping emits none — a reason rides a future answer type, never a guess. New specs: the issued input and both outcomes validate against their homes; a hostile Decision answer (admission as a bare string) and a faculty error result are error data — fail-closed rejection, the candidate never admits, the program keeps judging. --- src/faculties/system-one/schemas.ts | 3 +- .../system-one/tests/threads.spec.ts | 90 ++++++++++++++++++- src/faculties/system-one/threads.ts | 70 ++++++++++++++- 3 files changed, 159 insertions(+), 4 deletions(-) diff --git a/src/faculties/system-one/schemas.ts b/src/faculties/system-one/schemas.ts index 9e6d16d0a..2f7135dd0 100644 --- a/src/faculties/system-one/schemas.ts +++ b/src/faculties/system-one/schemas.ts @@ -98,7 +98,8 @@ const noulQuestionSchema = { additionalProperties: false, } as const -const choiceQuestionSchema = { +/** The choice question's schema — the admit/reject judgment derives its question shape from this one home. */ +export const choiceQuestionSchema = { type: 'object', properties: { type: { type: 'string', const: 'choice' }, diff --git a/src/faculties/system-one/tests/threads.spec.ts b/src/faculties/system-one/tests/threads.spec.ts index 27db53828..a03db6990 100644 --- a/src/faculties/system-one/tests/threads.spec.ts +++ b/src/faculties/system-one/tests/threads.spec.ts @@ -3,7 +3,12 @@ import { TRACE_MESSAGE_KINDS } from '../../../behavioral/behavioral.constants.ts import { behavioral } from '../../../behavioral/behavioral.ts' import type { BPEvent, SelectionTrace, Thread, Trace } from '../../../behavioral/behavioral.types.ts' import { FACULTY_MESSAGE_KINDS } from '../../faculties.constants.ts' -import { ADMISSION_EVENT_TYPES, admissionJudgmentThreads } from '../threads.ts' +import { + ADMISSION_EVENT_TYPES, + admissionJudgmentThreads, + validateAdmissionInput, + validateAdmissionVerdict, +} from '../threads.ts' /** * The system-one admission judgment pack against the real engine — the @@ -187,4 +192,87 @@ describe('system-one admission judgment pack', () => { ]) expect(selected.length).toBeGreaterThan(0) }) + + test('a malformed Decision result is error data — fail-closed rejection, no throw', () => { + // The answer is a hostile payload: `admission` is a bare string. The + // queries normalize it — no jq error, no admission; the reject listener + // holds the line. + const malformed: BPEvent = { + type: FACULTY_MESSAGE_KINDS.system_one_request_result, + detail: { id: 'at3-judge', ok: true, result: { model: 'm', answers: { admission: 'junk' } } }, + } + const selected = runJudgment([ + { event: candidate('at3', 'sneaky') }, + { + event: malformed, + check: (selected) => { + const rejected = selected.find( + (s) => s.type === ADMISSION_EVENT_TYPES.rejected && (s.detail?.id as string) === 'at3', + ) + expect(rejected).toBeDefined() + expect( + selected.some((s) => s.type === ADMISSION_EVENT_TYPES.admitted && (s.detail?.id as string) === 'at3'), + ).toBe(false) + }, + }, + ]) + // The program survived the hostile result: the loop wrapped and the next + // candidate still judges. + expect(selected.length).toBeGreaterThan(0) + }) + + test('a faculty error result is error data — the candidate rejects, never admits', () => { + const selected = runJudgment([ + { event: candidate('at4', 'unlucky') }, + { + event: { + type: FACULTY_MESSAGE_KINDS.system_one_request_result, + detail: { id: 'at4-judge', ok: false, error: { code: 'error', message: 'endpoint down' } }, + }, + check: (selected) => { + const rejected = selected.find( + (s) => s.type === ADMISSION_EVENT_TYPES.rejected && (s.detail?.id as string) === 'at4', + ) + expect(rejected).toBeDefined() + expect( + selected.some((s) => s.type === ADMISSION_EVENT_TYPES.admitted && (s.detail?.id as string) === 'at4'), + ).toBe(false) + }, + }, + ]) + expect(selected.length).toBeGreaterThan(0) + }) +}) + +describe('admission judgment — the Decision shapes', () => { + test('the issued Decision input validates against the input schema home', () => { + const selected = runJudgment([{ event: candidate('at1', 'greeter') }]) + const request = selected.find( + (s) => + s.type === FACULTY_MESSAGE_KINDS.system_one_request && (s.detail?.id as string | undefined) === 'at1-judge', + ) + expect(request).toBeDefined() + expect(validateAdmissionInput(request?.detail?.input as unknown)).toBe(true) + }) + + test('the judged outcomes validate against the verdict schema home', () => { + const selected = runJudgment([ + { event: candidate('at1', 'greeter') }, + { event: judgeResult('at1', 'admit') }, + { event: candidate('at2', 'suspicious') }, + { event: judgeResult('at2', 'reject') }, + ]) + const outcomes = selected.filter( + (s) => s.type === ADMISSION_EVENT_TYPES.admitted || s.type === ADMISSION_EVENT_TYPES.rejected, + ) + // Both outcomes fired and both conform to the one verdict home. + expect(outcomes.length).toBeGreaterThanOrEqual(2) + for (const outcome of outcomes) expect(validateAdmissionVerdict(outcome.detail)).toBe(true) + expect(outcomes.some((s) => s.type === ADMISSION_EVENT_TYPES.admitted && (s.detail?.id as string) === 'at1')).toBe( + true, + ) + expect(outcomes.some((s) => s.type === ADMISSION_EVENT_TYPES.rejected && (s.detail?.id as string) === 'at2')).toBe( + true, + ) + }) }) diff --git a/src/faculties/system-one/threads.ts b/src/faculties/system-one/threads.ts index 4598d5a55..c11d7d694 100644 --- a/src/faculties/system-one/threads.ts +++ b/src/faculties/system-one/threads.ts @@ -1,6 +1,7 @@ -import type { Thread } from '../../behavioral/behavioral.types.ts' -import { ThreadSchema } from '../../behavioral/behavioral.types.ts' +import type { JSONSchemaType } from 'ajv' +import { ajv, type Thread, ThreadSchema } from '../../behavioral/behavioral.types.ts' import { FACULTY_MESSAGE_KINDS } from '../faculties.constants.ts' +import { type ChoiceQuestion, choiceQuestionSchema } from './schemas.ts' /** * The System One faculty's admission judgment thread pack — the BP-native @@ -135,6 +136,71 @@ const JUDGE_ANSWER = '(if ($r.answers | type) == "object" then $r.answers else {} end) as $a | ' + '(if ($a.admission | type) == "object" then $a.admission else {} end) as $q' +/** The judged outcome — the Decision's answer mapped to a structured admission call. */ +export type AdmissionVerdict = { + /** The candidate's pending-admission id (the composition's key). */ + id: string + /** The Decision's call: true admits the candidate, false keeps it out. */ + admit: boolean + /** + * Optional free-text why. The choice answer carries no prose today, so the + * mapping emits none — a reason rides a future answer type, never a guess. + */ + reason?: string +} + +/** The judged outcome's schema — the composition's admission gate validates against this home. */ +export const ADMISSION_VERDICT_SCHEMA = { + type: 'object', + properties: { + id: { type: 'string', minLength: 1 }, + admit: { type: 'boolean' }, + reason: { type: 'string' }, + }, + required: ['id', 'admit'], + additionalProperties: false, +} as unknown as JSONSchemaType + +/** + * The Decision input the judgment issues: the proposed thread rides as state + * (the schema derives from the engine's ThreadSchema — one home, no + * mirroring), the question is the admit/reject choice over the shared + * question schema. + */ +export type AdmissionDecisionInput = { + state: { lane: 'admission-judgment'; thread: Thread } + questions: { [ADMISSION_QUESTION]: ChoiceQuestion } +} + +export const ADMISSION_INPUT_SCHEMA = { + type: 'object', + properties: { + state: { + type: 'object', + properties: { + lane: { type: 'string', const: 'admission-judgment' }, + thread: ThreadSchema, + }, + required: ['lane', 'thread'], + additionalProperties: false, + }, + questions: { + type: 'object', + properties: { [ADMISSION_QUESTION]: choiceQuestionSchema }, + required: [ADMISSION_QUESTION], + additionalProperties: false, + }, + }, + required: ['state', 'questions'], + additionalProperties: false, +} as unknown as JSONSchemaType + +/** The issued Decision input's boundary — the pack's jq must produce exactly this. */ +export const validateAdmissionInput = ajv.compile(ADMISSION_INPUT_SCHEMA) + +/** The judged outcome's boundary — the composition's admission gate consumes only conforming verdicts. */ +export const validateAdmissionVerdict = ajv.compile(ADMISSION_VERDICT_SCHEMA) + // ── Threads ────────────────────────────────────────────────────────────────── /** admission-issue — a candidate issues the correlated `system_one_request` with the thread as Decision input. */ From 76c7e9372feed1ff5e2e4cbc6a4782cbf8ae7f01 Mon Sep 17 00:00:00 2001 From: Edward Irby Date: Fri, 25 Sep 2026 17:44:56 -0700 Subject: [PATCH 23/55] =?UTF-8?q?feat(composition):=20the=20judged=20admis?= =?UTF-8?q?sion=20path=20=E2=80=94=20candidates=20wait=20on=20the=20Decisi?= =?UTF-8?q?on?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Wiring slice: with systemOne wired, the composition mounts the admission judgment pack alongside the faculty guard and routes a both-legs-ok add_thread verdict to a thread_candidate event instead of admitting — the judged path. The gate's block holds while the Decision runs; the pump's write legs are the judged outcome events: thread_admission (a conforming ADMISSION_VERDICT with admit === true) admits the pending thread under the re-entry law; thread_admission_rejected (or anything malformed — validateAdmissionVerdict is the boundary) drops the id, the rejection visible in the traces. Without systemOne the structural direct admit stands, unchanged (the existing admission-path specs pin it). E2E specs through the composition with the real system-one faculty and the Decisions fixture (now with a pickChoice override so the canned server can answer reject): approve — the Decision saw the proposed thread, the verdict precedes the admission, thread_added fires, the admitted thread goes live; reject — the line holds, no admission, no thread_added, nothing live. Docs: plan.md Current State + a 2026-09-25 Decision Log entry (the block is the judging window, type-scoped — per-candidate correctness is id-correlation; a literal permanent hold would deadlock all future admissions since pure-data threads cannot parameterize blocks; the fail-closed verdict mapping; the known engine frontier — the super-step cascade recurses unboundedly on self-sustaining loops). AGENTS.md's faculty map notes the pack. --- AGENTS.md | 4 +- src/cli/b-program.ts | 65 ++++++++++++- src/cli/tests/b-program.spec.ts | 95 +++++++++++++++++++ .../tests/fixtures/decisions-server.ts | 14 ++- 4 files changed, 170 insertions(+), 8 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 364b15205..0423c2eec 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -93,7 +93,9 @@ faculties), plus its `tests/`: `configSystemTwo(respond)` wires it, `useSystemTwo({ endpoints })` seeds the endpoint map - `system-one/` — TypeSafe/OpenRouter Decisions; `configSystemOne` + - `useSystemOne({ endpoint })`, with 429/529 retry + `useSystemOne({ endpoint })`, with 429/529 retry; `threads.ts` — the + admission judgment pack (the BP-native blocking judge over the Decisions + lane; the composition mounts it when systemOne is wired) - `shell/` — bun-direct script execution — `run` op TS scripts via `bun run -`, `shell` op Bun Shell commands through the wrapper; `rpc` op generic remote JSON-RPC (the remote-mcp layering is the thread pack, not the op); diff --git a/src/cli/b-program.ts b/src/cli/b-program.ts index 07866bfba..c3c8ad124 100644 --- a/src/cli/b-program.ts +++ b/src/cli/b-program.ts @@ -20,6 +20,11 @@ import { bindEmit } from '../faculties/process-lane.ts' import { remoteMcpThreads } from '../faculties/shell/remote-mcp.threads.ts' import { rpcAuthThreads } from '../faculties/shell/rpc-auth.threads.ts' import { shellThreads } from '../faculties/shell/threads.ts' +import { + ADMISSION_EVENT_TYPES, + admissionJudgmentThreads, + validateAdmissionVerdict, +} from '../faculties/system-one/threads.ts' import { useFaculty } from '../faculties/use-faculty.ts' import type { Faculty } from '../faculties.ts' @@ -230,6 +235,12 @@ export const bProgram = ({ // useFaculty compiled — a malformed system_one event is blocked (visible // in the frontier traces), not silently dropped. facultyAddThreads(guardThreads(`guard:${systemOne.name}-schema`, eventGuardEntries(systemOne.schemas))) + // The admission judgment pack: the BP-native blocking judge — it requires + // the Decisions lane (systemOne) and the structural layer (the in-process + // frontier embed, always present). With judgment wired, a validated + // candidate's admission is blocked while its Decision runs; the verdict + // events below are the judge's road back to the pump. + facultyAddThreads(admissionJudgmentThreads) route([FACULTY_MESSAGE_KINDS.system_one_request, FACULTY_MESSAGE_KINDS.system_one_cancel], { send: (event: BPEvent): void => systemOne.send(event), gate: (event: BPEvent): boolean => systemOne.invalidEventGate(event), @@ -250,7 +261,9 @@ export const bProgram = ({ // correlated frontier_request_result carries the verdict. The map is the // authorization: only results correlated to requests this composition // itself routed can ever admit. A null thread (the proposal failed the - // Thread-schema gate at registration) never admits. + // Thread-schema gate at registration) never admits. With systemOne wired + // the entry survives until the judgment resolves: the verdict is the + // CANDIDATE record — the judged outcome events below are the write legs. const pendingAdmissions = new Map() route([FACULTY_MESSAGE_KINDS.frontier_request], { @@ -300,13 +313,59 @@ export const bProgram = ({ useTrace((trace: Trace) => { if (trace.kind !== TRACE_MESSAGE_KINDS.selection) return const candidate = (trace as SelectionTrace).selected + // The judgment's outcome legs — the admission judgment pack's road back + // to the pump. Only a conforming verdict with admit === true admits; a + // rejection (or anything malformed — fail-closed) drops the pending id, + // the rejection visible in the traces. + if (candidate.type === ADMISSION_EVENT_TYPES.admitted || candidate.type === ADMISSION_EVENT_TYPES.rejected) { + const detail = candidate.detail as { id?: string } | undefined + const id = detail?.id + if (typeof id === 'string' && pendingAdmissions.has(id)) { + const thread = pendingAdmissions.get(id) + pendingAdmissions.delete(id) + if (thread && candidate.type === ADMISSION_EVENT_TYPES.admitted && validateAdmissionVerdict(candidate.detail)) + addThreads([thread]) + } + return + } if (candidate.type === FACULTY_MESSAGE_KINDS.frontier_request_result) { const detail = candidate.detail as { id?: string; ok?: boolean; result?: { ok?: boolean } } | undefined const id = detail?.id if (typeof id === 'string' && pendingAdmissions.has(id)) { const thread = pendingAdmissions.get(id) - pendingAdmissions.delete(id) - if (detail?.ok === true && thread && detail.result?.ok === true) addThreads([thread]) + if (detail?.ok === true && thread && detail.result?.ok === true) { + if (systemOne === undefined) { + // The structural-only path: no judge, no block — the candidate + // admits directly under the re-entry law. + pendingAdmissions.delete(id) + addThreads([thread]) + } else { + // The judged path: the verdict is the candidate record — emit it + // to the admission judgment pack (which blocks the admission + // while the Decision runs). The entry survives until the judged + // outcome leg above. (A validated thread is pure data — it + // serializes as JSON — but its listener schemas aren't statically + // JsonValue, hence the cast.) + addThreads([ + { + label: `thread-candidate:${id}`, + once: true, + rules: [ + { + request: { + type: ADMISSION_EVENT_TYPES.candidate, + detail: { id, thread: thread as unknown as JsonObject }, + }, + }, + ], + }, + ]) + } + } else { + // The rejection is data — the requester reads the why from the + // verdict trace. + pendingAdmissions.delete(id) + } } return } diff --git a/src/cli/tests/b-program.spec.ts b/src/cli/tests/b-program.spec.ts index 8aa8c2137..a3872ac3f 100644 --- a/src/cli/tests/b-program.spec.ts +++ b/src/cli/tests/b-program.spec.ts @@ -17,6 +17,7 @@ import { } from '../../faculties/shell/remote-mcp.threads.ts' import { useSystemOne } from '../../faculties/system-one/config.ts' import { startDecisionsServer } from '../../faculties/system-one/tests/fixtures/decisions-server.ts' +import { ADMISSION_EVENT_TYPES } from '../../faculties/system-one/threads.ts' import { useSystemTwo } from '../../faculties/system-two/config.ts' import { ASSISTANT_TEXT, startOpenResponsesServer } from '../../faculties/system-two/tests/fixtures/model-server.ts' import { useFaculty } from '../../faculties/use-faculty.ts' @@ -330,6 +331,100 @@ describe('bProgram — the runtime composition', () => { runtime.terminate() } }) + describe('add_thread — the admission judgment (systemOne wired)', () => { + test('the judged path: the Decision approves, the block lifts, the candidate admits and goes live', async () => { + const server = await startDecisionsServer() + const { runtime, traces } = startRuntime({ + systemOne: useSystemOne({ endpoint: { url: server.url, model: 'jev-latest' } }), + }) + try { + // The admitted thread is `once` — its ping selects and the thread completes. + // (A looping thread here would recurse the engine's super-step cascade + // unboundedly — a known engine frontier this test does not exercise.) + runtime.trigger( + addThreadRequest('aj1', { label: 'greeter', once: true, rules: [{ request: { type: 'ping' } }] }), + ) + // The judgment's outcome: the admission fires with the candidate id… + await waitForTraces(traces, (s) => + s.some( + (t) => + t.selected.type === ADMISSION_EVENT_TYPES.admitted && + (t.selected.detail as { id?: string }).id === 'aj1', + ), + ) + // …the Decision saw the proposed thread (the faculty's recorded request — + // the semantic layer judged the actual thread, not a schema echo). + const judged = server.requests.find( + (r) => (r.body.state as { thread?: { label?: string } } | undefined)?.thread?.label === 'greeter', + ) + expect(judged).toBeDefined() + expect(Object.keys(judged?.body.questions ?? {})).toContain('admission') + // The admission rides the judged outcome — the verdict precedes it. + const selections = selectionsOf(traces) + const judgeResultIndex = selections.findIndex( + (t) => + t.selected.type === FACULTY_MESSAGE_KINDS.system_one_request_result && + (t.selected.detail as { id?: string }).id === 'aj1-judge', + ) + const admittedIndex = selections.findIndex( + (t) => + t.selected.type === ADMISSION_EVENT_TYPES.admitted && (t.selected.detail as { id?: string }).id === 'aj1', + ) + expect(admittedIndex).toBeGreaterThan(judgeResultIndex) + // The thread_added provision fires — the composition owns the write. + expect( + traces.some( + (t) => + t.kind === TRACE_MESSAGE_KINDS.thread_added && + (t as { thread?: { label?: string } }).thread?.label === 'greeter', + ), + ).toBe(true) + // …and the admitted thread goes live — its request selects like any other thread's. + await waitForTraces(traces, (s) => s.some((t) => t.selected.type === 'ping')) + } finally { + runtime.terminate() + await server.close() + } + }) + + test('the judged path: a rejection holds the line — the candidate never admits', async () => { + const server = await startDecisionsServer({ pickChoice: 'reject' }) + const { runtime, traces } = startRuntime({ + systemOne: useSystemOne({ endpoint: { url: server.url, model: 'jev-latest' } }), + }) + try { + runtime.trigger(addThreadRequest('aj2', { label: 'suspicious', rules: [{ request: { type: 'evil' } }] })) + // The rejection is visible, stamped with the candidate id… + await waitForTraces(traces, (s) => + s.some( + (t) => + t.selected.type === ADMISSION_EVENT_TYPES.rejected && + (t.selected.detail as { id?: string }).id === 'aj2', + ), + ) + // …and the line held: no admission for the rejected candidate, no + // thread_added provision, nothing live. + expect( + selectionsOf(traces).some( + (t) => + t.selected.type === ADMISSION_EVENT_TYPES.admitted && + (t.selected.detail as { id?: string }).id === 'aj2', + ), + ).toBe(false) + expect( + traces.some( + (t) => + t.kind === TRACE_MESSAGE_KINDS.thread_added && + (t as { thread?: { label?: string } }).thread?.label === 'suspicious', + ), + ).toBe(false) + expect(selectionsOf(traces).some((t) => t.selected.type === 'evil')).toBe(false) + } finally { + runtime.terminate() + await server.close() + } + }) + }) }) test('terminate kills overridden faculties too — the composition owns every process it invokes', async () => { diff --git a/src/faculties/system-one/tests/fixtures/decisions-server.ts b/src/faculties/system-one/tests/fixtures/decisions-server.ts index d093ccd68..729069952 100644 --- a/src/faculties/system-one/tests/fixtures/decisions-server.ts +++ b/src/faculties/system-one/tests/fixtures/decisions-server.ts @@ -13,16 +13,19 @@ export const DECISIONS_MODEL = 'jev-1.13.0' type Question = { type: string; instructions: unknown; criteria?: unknown } -const answerFor = (question: Question, index: number): unknown => { +const answerFor = (question: Question, index: number, pickChoice?: string): unknown => { if (question.type === 'noul') return { type: 'noul', noul: 0.9 } if (question.type === 'choice') { const options = Object.keys((question.criteria ?? {}) as Record) + // `pickChoice` overrides the canned first-option answer (the rejection + // path needs a server that answers reject). + const choice = pickChoice !== undefined && options.includes(pickChoice) ? pickChoice : (options[0] ?? 'unknown') const probabilities = Object.fromEntries( - options.map((o, i) => [o, i === 0 ? 0.7 : 0.3 / Math.max(options.length - 1, 1)]), + options.map((o) => [o, o === choice ? 0.7 : 0.3 / Math.max(options.length - 1, 1)]), ) return { type: 'choice', - choice: options[0] ?? 'unknown', + choice, probabilities, confidence: 0.81, } @@ -51,10 +54,13 @@ export const startDecisionsServer = async ({ apiKey, rateLimitFirst = 0, delayMs = 0, + pickChoice, }: { apiKey?: string rateLimitFirst?: number delayMs?: number + /** Override the canned choice answer (e.g. `'reject'` for the judgment's rejection path). */ + pickChoice?: string } = {}): Promise => { const requests: RecordedDecisionRequest[] = [] let rateLimited = 0 @@ -88,7 +94,7 @@ export const startDecisionsServer = async ({ const questions = body.questions ?? {} const answers = Object.fromEntries( - Object.entries(questions).map(([id, question], index) => [id, answerFor(question, index)]), + Object.entries(questions).map(([id, question], index) => [id, answerFor(question, index, pickChoice)]), ) return Response.json({ model: DECISIONS_MODEL, From 3daa8e55495920126f1c996a618243b7865f26b1 Mon Sep 17 00:00:00 2001 From: Edward Irby Date: Fri, 25 Sep 2026 17:50:31 -0700 Subject: [PATCH 24/55] =?UTF-8?q?docs(skill):=20behavioral.md=20=E2=80=94?= =?UTF-8?q?=20the=20engine=20as=20the=20code=20has=20it?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The five verified deltas against src/behavioral/behavioral.ts and friends: - Package exports: `.` (defineConfig via src/main.ts), `./faculties`, `./controller`, `./utils` — `./tools` is gone, the engine is in none of them; the source-path import example drops the dead `UseAddThread` type. - Hook surface: five members `{ addThread, trigger, useTrace, step, instanceId }` — the curried `useAddThread(space?)` is gone; `space` rides on the Thread. `step()` documented as the re-entry pump. - Action channel: the canonical host is the composition (src/cli/b-program.ts) under the in-process re-entry law — addThread alone is inert, every re-entry pumps a super-step; the contentless kick and the dead src/kernel/kernel.ts reference are removed. - Transform: the engine now applies the contract in-engine (the jq worker + evaluateTransform bridge); errors-as-data via transform_error; the remote-mcp thread pack is the production consumer. The stale "no production host" caveat is gone. - Trace union table: all 12 TRACE_MESSAGE_KINDS, adding idle, thread_added, transform_error, step with their payloads from behavioral.types.ts. Validated with the shell faculty's validate-links recipe (missing: []). --- skills/behavioral/references/behavioral.md | 173 +++++++++++---------- 1 file changed, 89 insertions(+), 84 deletions(-) diff --git a/skills/behavioral/references/behavioral.md b/skills/behavioral/references/behavioral.md index 6a8d030e9..0288dc572 100644 --- a/skills/behavioral/references/behavioral.md +++ b/skills/behavioral/references/behavioral.md @@ -8,33 +8,35 @@ those matching any `block` are filtered out, the highest-priority remaining candidate is selected, threads waiting/requesting/interrupted by it are resumed, and the next step runs. If no unblocked candidate exists the program halts until an event arrives via `trigger` (the external admission -surface and the contentless internal kick). +surface and one super-step). ## Public surface The engine lives in-repo at `src/behavioral/behavioral.ts` (with types in `behavioral.types.ts`, constants in `behavioral.constants.ts`, utils in -`behavioral.utils.ts`). It is **not** a public package export — there is no -root `@behavioral/sh` export (the package exports only `./tools`, -`./controller`, `./utils`). Import it from its source path: +`behavioral.utils.ts`). It is **not** one of the package's public exports — +`@behavioral/sh` exports `.` (the `defineConfig` config helper, via +`src/main.ts`), `./faculties` (the faculty wire and config surface), +`./controller`, and `./utils`; none re-export the engine. Import it from its +source path: ```ts import { behavioral } from '../../behavioral/behavioral.ts' import type { + AddThread, BPEvent, Disconnect, Thread, Trace, - UseAddThread, UseTrace, } from '../../behavioral/behavioral.types.ts' ``` -`behavioral()` returns a frozen API object with **three hooks** — no +`behavioral()` returns a frozen API object with **five members** — no `useAddHandler`, no `sendTrace`, no generic type parameter: ```ts -const { useAddThread, trigger, useTrace } = behavioral({ sessionId?: string }) +const { addThread, trigger, useTrace, step, instanceId } = behavioral({ sessionId?: string }) ``` The optional `sessionId` is host-supplied session identity stamped on every @@ -48,20 +50,21 @@ thread on an event), and/or `transform` (match, hand off to external reshaping, re-enter via a `target` event). `detailSchema` on listeners is JSON Schema (draft 2020-12), compiled at registration. -## The three hooks +## The API surface -`const { useAddThread, trigger, useTrace } = behavioral()` +`const { addThread, trigger, useTrace, step, instanceId } = behavioral()` -| Hook | Signature | Use when | -|------|-----------|----------| -| `useAddThread(space?)` | `(args: Thread) => void` | Register a b-thread (`{ label, rules, once? }`). Optional `space` stamps all the thread's idioms. Inert — does not start a super-step. | +| Member | Signature | Use when | +|--------|-----------|----------| +| `addThread(args)` | `(args: Thread) => void` | Register a b-thread (`{ label, rules, once?, space? }`). The thread's optional `space` field stamps all its idioms (applied at registration). Inert — does not start a super-step. | | `trigger(event)` | `(event: BPEvent) => void` | Inject an external event (the event carries `space`; absent = root). Triggered candidates carry `ingress: true`, have highest priority (0), and can be blocked. Initiates a super-step. | -| `useTrace(listener)` | `Disconnect` | Observe internal state traces emitted after each event selection. Does not affect execution. | +| `useTrace(listener)` | `(listener) => Disconnect` | Observe internal state traces emitted after each event selection. Does not affect execution. | +| `step()` | `() => void` | Pump one super-step. The internal re-entry primitive — `addThread` alone is inert, so every re-entry pairs the thread addition with a `step()` pump. | +| `instanceId` | `string` | The per-process identity the engine self-mints and stamps on every trace. Exposed so a host can hand the identity to its clients without sniffing the trace wire (silent on an idle instance). | -### `useAddThread` — registering threads +### `addThread` — registering threads ```ts -const addThread = useAddThread() addThread({ label: 'producer', rules: [{ request: { type: 'task' } }], @@ -75,9 +78,10 @@ addThread({ ``` A thread is an object with `label`, `rules` (an array of `Idioms` sync points), -and optional `once`. Without `once`, the thread loops its `rules` indefinitely; -with `once: true`, it runs through the rules once and completes. The `label` -identifies the thread in traces. Invalid thread arguments (failing +and optional `once` and `space`. Without `once`, the thread loops its `rules` +indefinitely; with `once: true`, it runs through the rules once and completes. +The `label` identifies the thread in traces; the `space` scopes it (absent = +root). Invalid thread arguments (failing `ThreadSchema`, or an un-compilable `detailSchema`) are surfaced as an `add_thread_error` trace, not a throw — the thread simply isn't added. @@ -94,15 +98,16 @@ priority 0, stamped `ingress: true`. They are subject to `block` like any request. An event that fails `BPEvent` validation is rejected at the ingress boundary and surfaced as a `trigger_error` trace (not a throw), echoing the attempted `space` when present. `trigger` is how external systems (UI, -network, timers) drive the program — and also how internal re-entry starts a -super-step (the contentless kick; see the action channel). +network, timers) drive the program — it is external admission plus one +super-step, nothing else. Internal re-entry never uses `trigger`: it adds a +request thread and pumps a super-step with `step()` (see the action channel). #### The channel invariant A selected event carries `ingress: true` **iff** it was admitted externally -via `trigger`; everything internal arrives as a thread request added through -`useAddThread` (the dispatch bridge, the transform daemon, -`threads.registered`). Listeners opt into a channel with the optional +via `trigger`; everything internal (satellite results, transform +targets, admitted threads) arrives as a thread request added through +`addThread`. Listeners opt into a channel with the optional `ingressMatch` field: | listener field | value | matches | @@ -130,9 +135,8 @@ backpressure on external events is expressed. ```ts const disconnect = useTrace((msg: Trace) => { - // msg is the engine's closed Trace union — narrow by `kind`: - // 'pending_bids' | 'frontier' | 'selection' | 'deadlock' - // 'trigger_error' | 'add_thread_error' | 'interrupt' | 'transform' + // msg is the engine's closed Trace union — narrow by `kind` + // (12 kinds; the full table below). }) ``` @@ -140,33 +144,27 @@ const disconnect = useTrace((msg: Trace) => { may be sync or async (`void | Promise`); **the engine never awaits it.** Each listener return value is absorbed by `Promise.resolve(...)` with a rejection handler attached, so a rejecting promise never breaks the -super-step. A listener that throws synchronously is caught and logged via -`console.error('[behavioral] trace listener ...')` — listener failures are -**log-only**, never published as traces. +super-step. A listener failure (a sync throw or a rejecting promise) is +caught and logged via `console.error('[behavioral] trace listener ...')` — +listener failures are **log-only**, never published as traces, and the catch +is per-consumer (one failing listener cannot suppress the others). #### The action-channel pattern (replaces `useAddHandler`) There is no `useAddHandler` hook. Side effects — tool dispatch, I/O, model calls — are performed by `useTrace` listeners that observe `selection` traces and act outside the super-step, then **re-enter the engine as a -thread**. The kernel's dispatch bridge is the canonical implementation -(`src/kernel/kernel.ts`): +thread**. The canonical host is the composition (`src/cli/b-program.ts` — +the runtime composition `bProgram`): its pump subscribes `useTrace`, routes +selected events to their faculty lanes, and re-enters results through one +seam: ```ts -// The action channel: fire on selection, do async I/O, re-enter via a -// once-thread + kick. -const disconnect = useTrace((msg) => { - if (msg.kind !== 'selection') return - void bridge(msg.selected.type) // async I/O outside the super-step -}) - -// Re-entry is deferred past the current super-step so the bridge never -// re-enters the engine synchronously from inside a listener. -const reenter = (event: BPEvent): void => { - queueMicrotask(() => { - addThread(createReentryThread(event)) // label: `reentry:`; once: true - trigger({ type: KICK_EVENT_TYPE, space }) // contentless kick - }) +// The in-process re-entry law: addThread alone is inert — every re-entry +// pumps one super-step. +const addThreads = (threads: Thread[]): void => { + for (const thread of threads) addThread(thread) + step() } ``` @@ -174,20 +172,18 @@ The contract: - The listener filters on `msg.kind === 'selection'` and reads `msg.selected.type` to decide what to do. -- Async work happens **after** the listener returns — the engine continues the - super-step without waiting. +- Async work happens **after** the listener returns — the engine never awaits + a listener, so the super-step continues without waiting. - Results re-enter by **adding a `once` thread that `request`s the event**, - then firing the contentless `KICK_EVENT_TYPE` kick. `useAddThread` is inert, - so the kick is what starts the super-step. The kick is priority 0, carries - no detail, and nothing may listen for/wait on/block/transform it — an - external actor triggering it is harmless by construction. The re-entry - event therefore arrives as a **request-origin** candidate (`ingress` - absent), which is what lets `ingressMatch: false` listeners match internal - results while `ingressMatch: true` listeners stay external-only. -- Both calls are deferred with `queueMicrotask` so the action channel never - re-enters the engine synchronously from inside a `sendTrace` listener call. -- Tool/I/O failures return as **data** (`isError: true` on the output) and - drive a `turn.end` or recovery trigger — they never throw into the space. + then pumping one super-step (the re-entry law above). The composition's + `useFaculty` pump funnels every satellite's result threads through the same + seam, and engine-internal code (the transform executor) calls the internal + `step()` directly. The re-entry event therefore arrives as a + **request-origin** candidate (`ingress` absent), which is what lets + `ingressMatch: false` listeners match internal results while + `ingressMatch: true` listeners stay external-only. +- Faculty/I/O failures return as **data** on the result event and re-enter + through the same seam — they never throw into the space. - A listener throw is `console.error`'d and swallowed — it cannot corrupt the program. @@ -212,7 +208,7 @@ addThread({ transform: [{ type: 'order', // match this selected event detailSchema: { ... }, // optional JSON Schema guard - query: '.order', // applied to selected.detail (e.g. a jq expression) + query: '.order', // applied to selected.detail (a jq expression) target: 'ship', // re-enter the engine with this event type }], }], @@ -222,51 +218,60 @@ addThread({ Shape (`TransformListener` in `behavioral.types.ts`): a `BPListener` plus `query` (string) and `target` (string). When a matching event is selected, the engine emits a `transform` trace carrying `transformers: { query, target, -thread }[]` **immediately before** the `selection` trace, then resumes the -thread (a transform match wakes the thread like a `waitFor` match). The -engine does **no I/O** — it only publishes the contract. External code reads -the `transform` trace, evaluates each `query` over `selected.detail`, and -re-enters by adding a `once` thread that `request`s `{ type: target, detail }` -(one per target when fanning out) and firing the contentless kick — never via -`trigger`, so the target stays request-origin. - -This is a two-phase loop: **prime** (the `transform` trace carries the -contracts) then **execute** (the immediately following `selection` trace -carries the payload). The reference implementation is -`src/behavioral/tests/transform.spec.ts`. Honest caveat: today only that test -loop consumes the trace — the kernel-side consumer is not yet wired, so there -is no production host applying `query` → `target` yet. The trace contract is -stable; the host is what's missing. +thread, space? }[]` **immediately before** the `selection` trace, then resumes +the thread (a transform match wakes the thread like a `waitFor` match) and +applies the contract itself: each `query` is evaluated over `selected.detail` +by the engine's internal jq subprocess (`src/behavioral/jq.worker.ts`, driven +by the `evaluateTransform` bridge in `behavioral.utils.ts`), and the result +re-enters as a `once` thread requesting `{ type: target, detail: result.value }` +stamped with the contract's `space` — the target stays request-origin. The +engine does no arbitrary I/O; its only external dependency is the jq binary. + +Failures are errors-as-data: a contract that fails (jq error, no detail, +empty or non-object output) never fires its target — the failure surfaces as +a `transform_error` trace carrying the `transformer` and a machine-readable +`reason`. + +The contract test is `src/behavioral/tests/transform.spec.ts`, and the +production consumer is real: the remote-MCP thread pack +(`src/faculties/shell/remote-mcp.threads.ts`) drives its entire +discover/tools/call pipeline with transforms over the shell faculty's `rpc` +op. ## The trace union -`Trace` is a closed discriminated union (narrow by `kind`). The kinds: +`Trace` is a closed discriminated union (narrow by `kind`) — the 12 kinds of +`TRACE_MESSAGE_KINDS` (`src/behavioral/behavioral.constants.ts`): | `kind` | Carries | When | |--------|---------|------| +| `step` | `step`, `ingress?` | A super-step began; `ingress: true` marks an externally initiated step | | `pending_bids` | `step`, `threads` (serialized pending set) | Before event selection each step | | `frontier` | `step`, `status`, `candidates`, `enabled` | After computing the frontier | | `selection` | `step`, `selected` (the chosen candidate) | When an event is selected | +| `idle` | `step` | No candidates at all — the program is quiescent (not deadlocked); the settle signal | | `deadlock` | `step` | Candidates exist but all are blocked | +| `thread_added` | `thread` (the full validated `Thread`) | `addThread` registered a thread — the provision record; replay = `thread_added` payloads + ingress events in order | | `interrupt` | `selected`, `threadLabel`, `step` | A thread was terminated by an interrupt | -| `transform` | `step`, `transformers` | A transform listener matched; external code applies `query` → `target` | -| `add_thread_error` | `error` (AJV errors), `space?` | `useAddThread` rejected invalid args / un-compilable `detailSchema` | +| `transform` | `step`, `transformers` | A transform listener matched; the engine applies `query` → `target` in-engine | +| `transform_error` | `step`, `transformer`, `reason`, `stderr?`, `exitCode?` | A transform contract failed (jq error, no detail, empty or non-object output); the target never fires | +| `add_thread_error` | `error` (AJV errors), `space?` | `addThread` rejected invalid args / un-compilable `detailSchema` | | `trigger_error` | `error` (AJV errors), `space?` | `trigger` rejected an invalid `BPEvent` | -The two error kinds are the engine's only failure surfaces, and both are +The three error kinds are the engine's failure surfaces, and all are **traces, not throws** — invalid input is reported as data and the program keeps running. There is no `feedback_error` trace. ## A common wiring mistake to avoid -Forgetting to start the super-step after adding threads. `useAddThread` +Forgetting to start the super-step after adding threads. `addThread` registers a thread but does **not** start a super-step on its own; the program pauses until an event enters via `trigger`. A common symptom: threads are added, nothing happens. For an external event, trigger it; for internal re-entry (a result or a transform target), add the `once` request thread and -fire the contentless kick. This inertness is deliberate: pure-requesting -programs (tic-tac-toe, water) do not self-start at registration, so quiescence -is preserved until someone admits an event. +pump a super-step with `step()`. This inertness is deliberate: +pure-requesting programs (tic-tac-toe, water) do not self-start at +registration, so quiescence is preserved until someone admits an event. The second common mistake: expecting side effects to fire on `trigger`. The action channel fires on **selected** events — a triggered event that is @@ -278,5 +283,5 @@ event wasn't filtered out. - [Frontier analysis](./frontier-analysis.md) — deadlock/livelock verification over the closed state graph of a behavioral program. -- [Controller](./controller.md) — the browser-side message applier and the - stateless SSR html tools (one UI-layer reference). +- [Controller](./controller.md) — the browser-side message applier over the + `ui_*` wire (the UI-layer reference). From c0a15155d5fdabdac0a099c47a8614a8087da983 Mon Sep 17 00:00:00 2001 From: Edward Irby Date: Fri, 25 Sep 2026 17:51:06 -0700 Subject: [PATCH 25/55] =?UTF-8?q?docs(skill):=20controller.md=20=E2=80=94?= =?UTF-8?q?=20the=20ui=5F*=20wire=20and=20one=20live=20surface?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The controller wire vocabulary is uniformly `ui_*` (controller.constants.ts): ui_render/ui_attrs/ui_dispatch_custom_event/ ui_navigate/ui_scale_check in, ui_event/ui_error/ui_form_submit/ ui_success/ui_snapshot/ui_scale_check_result out. Every table, example, and prose mention takes the rename. Restructured around the Controller as the one live surface: the retired SSR html tools and the no-Renderer claim are now an explicit statement, not a pair of live surfaces. Added the schema home the doc omitted — CONTROLLER_DETAIL_SCHEMAS (controller.schemas.ts), imported by the host threads, never the browser bundle — alongside the deterministic floors and the classifier ceiling. Dropped the dead behavioral-tools html.md link and the retired html-scale-check tool name; verified the WebSocket retry codes (1006/1012/1013, max 3) and queue-flush behavior still match controller.utils.ts. Validated with the shell faculty's validate-links recipe (missing: []). --- skills/behavioral/references/controller.md | 119 +++++++++++---------- 1 file changed, 62 insertions(+), 57 deletions(-) diff --git a/skills/behavioral/references/controller.md b/skills/behavioral/references/controller.md index 5b28ebece..96224bb52 100644 --- a/skills/behavioral/references/controller.md +++ b/skills/behavioral/references/controller.md @@ -1,19 +1,15 @@ -# UI Layer — Controller and html tools +# UI Layer — the Controller Reference for an agent assisting an engineer in wiring up the UI layer of a -behavioral app. There are two surfaces, both driven by a -[behavioral program](./behavioral.md)'s `selection` listeners (the action -channel): +behavioral app. The live surface is the browser **`Controller`** — it applies +`ui_*` wire messages to a **live DOM** over a WebSocket (or an injected +`Transport`). It is driven by a [behavioral program](./behavioral.md)'s +`selection` listeners (the action channel). -- **Browser `Controller`** — applies `render`/`attrs` (plus - `dispatch_custom_event`/`navigate`/`scale_check`) to a **live DOM** over a - WebSocket. -- **Stateless html tools** — apply `render`/`attrs` to an **HTML string** in - memory, in a Bun process (SSR). - -There is **no `Renderer` class** — SSR is stateless html-in / html-out tools. -The two surfaces share the same `render`/`attrs` vocabulary; the substrate -(live DOM vs string) is the variable. +The compiled SSR html tools are **retired** (the ICL conversion removed the +fleet). There is **no `Renderer` class** and no server-side html surface — the +`ui_*` vocabulary's server side is emitted by the agent's behavioral program, +and its browser side is the Controller below. ## Browser Controller @@ -34,6 +30,7 @@ new Controller({ onPageSwap, // page swap callback onPageHide, // pagehide callback onPageShow, // pageshow callback + transport, // optional injected Transport (default: built-in WebSocket carrier) }) ``` @@ -41,34 +38,46 @@ new Controller({ This is the load-bearing concept: a behavioral page is **push-based**, not pull-based. The controller does not fetch state and render client-side; it -opens a WebSocket to its serving agent and applies server-pushed messages: +opens a carrier to its serving agent and applies server-pushed `ui_*` +messages: -| Server → browser (`CONTROLLER_INCOMING_MESSAGE_TYPES`) | What the Controller does | -|----------------------------------------------------------|---------------------------| -| `render` | Apply HTML to `[b-target]` elements per the `swap` mode | -| `attrs` | Set/remove attributes on `[b-target]` elements | -| `dispatch_custom_event` | Fire a `CustomEvent` on the target | -| `navigate` | Navigate the page (URL change) | -| `scale_check` | Resolve the effective `b-scale` for a target and reply with `scale_check_result` | +| Agent → browser (`CONTROLLER_INCOMING_MESSAGE_TYPES`) | What the Controller does | +|-------------------------------------------------------|--------------------------| +| `ui_render` | Apply HTML to `[b-target]` elements per the `swap` mode | +| `ui_attrs` | Set/remove attributes on `[b-target]` elements | +| `ui_dispatch_custom_event` | Fire a `CustomEvent` on the target | +| `ui_navigate` | Navigate the page (URL change) | +| `ui_scale_check` | Resolve the effective `b-scale` for a target and reply with `ui_scale_check_result` | -User interactions and page lifecycle emit messages back to the agent: +User interactions and page lifecycle emit `ui_*` messages back to the agent: | Browser → agent (`CONTROLLER_OUTGOING_MESSAGE_TYPES`) | When | -|---------------------------------------------------------|------| +|-------------------------------------------------------|------| | `ui_event` | A `b-trigger` declaration fired (DOM event → BP event with `getAttributes` detail) | -| `snapshot` | A page lifecycle event (`pagereveal`/`pageswap`/`pagehide`/`pageshow`) — serialized HTML via `getHTML({ serializableShadowRoots: true })` | -| `success` | A server message was applied successfully (carries the request `id`) | -| `error` | A message handler threw (carries `name`, `error`, `stack`, `id`) | -| `scale_check_result` | Reply to a `scale_check` message (carries `effectiveScale`) | -| `form_submit` | A `b-form` form POST completed | +| `ui_snapshot` | A page lifecycle event (`pagereveal`/`pageswap`/`pagehide`/`pageshow`) — serialized HTML via `getHTML({ serializableShadowRoots: true })` | +| `ui_success` | A server message was applied successfully (carries the request `id`) | +| `ui_error` | A message handler threw (carries `name`, `error`, `stack`, `id`) | +| `ui_scale_check_result` | Reply to a `ui_scale_check` message (carries `effectiveScale`) | +| `ui_form_submit` | A `b-form` form POST completed | The agent — running a behavioral program — is the source of truth for what the page shows; the Controller is the DOM applier. -## Server-side html (the floors + classifier story) +The kind names are `keyMirror` constants in +`src/controller/controller.constants.ts` +(`CONTROLLER_INCOMING_MESSAGE_TYPES` / `CONTROLLER_OUTGOING_MESSAGE_TYPES`). + +### The schema home + +`CONTROLLER_DETAIL_SCHEMAS` (`src/controller/controller.schemas.ts`) maps +every `ui_*` kind to its AJV detail schema — the guard/reflection home. The +host threads import it (`validateControllerDetail`) to gate controller +messages at the composition boundary. The browser bundle **never** carries +the compiled validators — it ships only the deterministic floors below. + +### The floors + classifier story -The compiled SSR html tools are **retired** (HTMLRewriter was Bun-only, dead -in both target hosts; the ICL conversion removed the fleet). What survives: +What keeps the wire safe across every host: - **Deterministic floors** — `isInvalidTrigger` / `detectXssVectors` (`src/controller/controller.utils.ts`): hardcoded invariants (empty @@ -77,15 +86,15 @@ in both target hosts; the ICL conversion removed the fleet). What survives: - **Schemas as data** — `src/controller/html.schemas.ts` + `css.schemas.ts`: pure JSON-schema data (the classifier's context, not compiled validators). - **The classifier ceiling** — the System One/Jev gate story: - - — probabilistic admission over the schema context, with the floors as the + probabilistic admission over the schema context, with the floors as the deterministic backstop. Probabilistic gates never own security invariants. -## When to use which +## Wiring guidance - **Wiring a multi-page app**: one `Controller` per page, constructed in the - page's `` async module. The WebSocket URL is derived from the page's - origin (`location.href.replace(/^http/, 'ws')`). + page's `` async module. The default carrier derives the WebSocket URL + from the page's origin (`location.href.replace(/^http/, 'ws')`); pass + `transport` to inject a different one. - **Binding interactive elements**: declare `b-trigger` and `b-form` attributes in the DOM; the Controller wires them to emit `ui_event` messages on user interaction. No manual `addEventListener` in your code. @@ -94,34 +103,30 @@ in both target hosts; the ICL conversion removed the fleet). What survives: sockets, timers) on unload and bfcache freeze; the Controller does **not** force-close the socket on `pagehide` so a queued snapshot can flush during teardown. -- **SSR / pre-render**: a behavioral-program `selection` listener calls the - html tools directly to produce an HTML string for an initial page load or - snapshot — see [html](../../behavioral-tools/references/html.md) for the tool - surface. -- **Scale pre-flight**: the agent sends `scale_check` (browser) or calls - `html-scale-check` (SSR) to learn the effective `b-scale` a render target - lives in before generating content. +- **Scale pre-flight**: the agent sends `ui_scale_check` before generating + content to learn the effective `b-scale` a render target lives in; the + Controller replies with `ui_scale_check_result` carrying `effectiveScale`. ## A common wiring mistake to avoid Calling `Controller` methods directly to mutate the DOM. The Controller is a -**message applier**, not a DOM API — `render`/`attrs`/`dispatch_custom_event`/ -`navigate` arrive as server-pushed messages and are dispatched internally, -not called by your code. If you find yourself reaching for a Controller method -to change the page, the correct path is to emit a `ui_event` (via a -`b-trigger`/`b-form` declaration) and let the agent's behavioral program -respond with a server-pushed `render`. The DOM is downstream of the agent, -not the other way around. +**message applier**, not a DOM API — `ui_render`/`ui_attrs`/ +`ui_dispatch_custom_event`/`ui_navigate` arrive as server-pushed messages and +are dispatched internally, not called by your code. If you find yourself +reaching for a Controller method to change the page, the correct path is to +emit a `ui_event` (via a `b-trigger`/`b-form` declaration) and let the +agent's behavioral program respond with a server-pushed `ui_render`. The DOM +is downstream of the agent, not the other way around. The second common mistake: expecting the WebSocket to be manually managed. -The Controller handles connect, retry (with bounded backoff on codes 1006/ -1012/1013, max 3 retries), and message queuing during disconnect internally. -Do not wrap it in your own reconnection logic — that duplicates the built-in -faculty and races with the Controller's own retry. +The Controller handles connect, retry (bounded backoff on close codes +1006/1012/1013 — max 3 attempts, jittered exponential delay capped at +`UI_CORE_MAX_RETRIES` in `controller.constants.ts`), and message queuing +during disconnect (the queue flushes on reconnect) internally. Do not wrap it +in your own reconnection logic — that races with the Controller's built-in +retry. ## See also - [behavioral](./behavioral.md) — the runtime whose `selection` listeners - drive both surfaces (the action channel). -- [html](../../behavioral-tools/references/html.md) — the SSR tool surface: - I/O contracts, dispatch examples, gotchas. + drive the agent side of the `ui_*` wire (the action channel). \ No newline at end of file From 42dc509a80c54b6399ae3b99890893228c39e11d Mon Sep 17 00:00:00 2001 From: Edward Irby Date: Fri, 25 Sep 2026 17:51:47 -0700 Subject: [PATCH 26/55] docs(skill): retire the design-spec surface, refresh SKILL routing MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Per the 2026-09-25 design.md ruling: references/design-spec.md is deleted (it self-declared a non-normative wayfinding consensus surface, not a reference) and its SKILL.md mentions go with it — the route-table row, the description's design-system-spec clause, and the "When to use" bullet. SKILL.md takes the current-surface rewrite: the description covers the runtime, faculties wire, controller `ui_*` protocol, frontier analysis, and eval — no SSR html tools (retired), no design-spec. The controller route row takes the ui_* rename. The companion-skills paragraph replaces the mcp faculty + dead behavioral-tools link with the remote-mcp thread pack over the shell faculty's rpc op. frontier-analysis.md light touch: the dead behavioral-tools frontier.md link is dropped; "three ops" becomes the four-op surface (replay, explore, verify, plus the landed add_thread admission op); the schema-home sentence reconciles the two homes — the wire event shape in faculties/faculties.types.ts, the per-op input schemas in faculties/frontier/faculty.ts; fleet-era frontier-verify/explore tool names become op phrasing. Validated: SKILL.md frontmatter parses under the scan's fence rules; validate-links missing: [] on both edited files. --- skills/behavioral/SKILL.md | 36 +- skills/behavioral/references/design-spec.md | 569 ------------------ .../references/frontier-analysis.md | 23 +- 3 files changed, 28 insertions(+), 600 deletions(-) delete mode 100644 skills/behavioral/references/design-spec.md diff --git a/skills/behavioral/SKILL.md b/skills/behavioral/SKILL.md index a296b0f94..e45600157 100644 --- a/skills/behavioral/SKILL.md +++ b/skills/behavioral/SKILL.md @@ -2,11 +2,10 @@ name: behavioral description: > Behavioral-programming runtime and UI layer — b-threads, triggers, - listeners, the controller/custom-element protocol, SSR via stateless html - tools, frontier analysis, behavioral eval capture, and the design-system - spec. Use when creating, reading, updating, or deleting code where - @behavioral/sh is a declared dependency or where the work is in the - behavioral repo itself. + listeners, the super-step model, the faculties event wire, the controller + `ui_*` protocol, frontier analysis, and behavioral eval capture. Use when + creating, reading, updating, or deleting code where @behavioral/sh is a + declared dependency or where the work is in the behavioral repo itself. license: ISC compatibility: Requires bun and the behavioral CLI allowed-tools: Bash Read @@ -25,15 +24,12 @@ Use this skill when the task involves the behavioral **runtime** or **UI layer** and you're working in a project where `@behavioral/sh` is a declared dependency or in the behavioral repo itself. Specifically: -- Wiring **behavioral programs** — b-threads, triggers, `useTrace` - listeners, the super-step model, deadlock/livelock analysis. -- Building **custom elements** via the controller protocol, or - **server-side rendering** via the stateless html tools. +- Wiring **behavioral programs** — b-threads, triggers, + `useTrace` listeners, the super-step model, deadlock/livelock analysis. +- Building **custom elements** wired by the controller `ui_*` protocol + (browser Controller). - Capturing or grading **agent runs** (eval) — trace primitives and divergence analysis. -- Designing the **design-system spec** — a DESIGN.md derivative re-grounded - on Structural IA, CSS custom properties, and `@scope`/`:host()`/`::part()` - modes. The `behavioral` CLI is the entry (`bin/behavioral.ts`). The tools-fleet dispatcher is retired with the ICL conversion; run `behavioral --help` for @@ -46,17 +42,17 @@ only when the task calls for it. | When the task involves… | Read | |-------------------------|------| -| Behavioral programs — b-threads, `useAddThread`/`trigger`/`useTrace`, the super-step model, the action-channel pattern | [`references/behavioral.md`](./references/behavioral.md) | +| Behavioral programs — b-threads, `addThread`/`trigger`/`useTrace`/`step`, the super-step model, the action-channel pattern | [`references/behavioral.md`](./references/behavioral.md) | | Deadlock/livelock verification — frontier analysis over the closed state graph | [`references/frontier-analysis.md`](./references/frontier-analysis.md) | -| UI layer — the browser Controller protocol (`render`/`attrs`/`scale_check`, `ui_event`/`snapshot`/`error`/`success`/`scale_check_result`/`form_submit`) and the stateless SSR html tools | [`references/controller.md`](./references/controller.md) | +| UI layer — the browser Controller over the `ui_*` wire (`ui_render`/`ui_attrs`/`ui_scale_check`, `ui_event`/`ui_snapshot`/`ui_error`/`ui_success`/`ui_scale_check_result`/`ui_form_submit`) | [`references/controller.md`](./references/controller.md) | | Capturing/grading an agent run — eval trace primitives, divergence analysis | [`references/eval.md`](./references/eval.md) | -| Design-system spec — DESIGN.md derivative, Structural IA, custom properties, `@scope`/`:host()`/`::part()`, scale + affordances/feedback (in-progress consensus surface) | [`references/design-spec.md`](./references/design-spec.md) | -**Companion skills:** remote MCP operations are the **mcp faculty** -(`mcp_request` wire — see `behavioral-tools/references/mcp-client.md`); the -skill/plugin domain conventions (store tenants, scan recipes, `links_request`, -ICL composition) are the **skill-conventions** skill. This skill owns the -concepts; those skills own the operator contracts. +**Companion skills:** remote MCP operations are the **remote-mcp thread +pack** over the shell faculty's generic `rpc` op +(`src/faculties/shell/remote-mcp.threads.ts`); the +skill/plugin domain conventions (store tenants, scan recipes, +`links_request`, ICL composition) are the **skill-conventions** skill. This +skill owns the concepts; those skills own the operator contracts. ## Repo conventions diff --git a/skills/behavioral/references/design-spec.md b/skills/behavioral/references/design-spec.md deleted file mode 100644 index e610dcf30..000000000 --- a/skills/behavioral/references/design-spec.md +++ /dev/null @@ -1,569 +0,0 @@ -# Design spec (consensus surface) - -Reference for an agent assisting in the design of behavioral's design-system -specification — a derivative of [DESIGN.md] re-grounded on Rachel Jaffe's -[Structural IA] to move beyond atomic visual styling into a full structural -+ expressive system an agent can use to build interfaces. This document is -the **editable consensus surface** for an in-progress wayfinding effort, not -the finished spec. It captures decisions locked so far, the substrate facts -gathered from the codebase and MDN, and the open frontier still to grill. -Edit it freely as exploration advances; when the way is clear, the hand-off -spec is written from it. - -> **Status: in progress.** Locks are marked **Locked**; unresolved -> questions live under [Open frontier](#open-frontier) and must not be -> treated as decided. Nothing here is normative yet — this is the shared -> map, not the territory. - -## Destination - -**Locked** — the effort's destination is a **hand-off spec**: a new -DESIGN.md-derivative format spec for the behavioral framework, handed off for -implementation and iteration. The map ends when every decision needed to -*write* that spec is made; the prose itself gets written *after* the map, by -whoever does the work. The destination fixes scope, so it was settled -first. - -## Two-phase usage model - -**Locked** — the spec is a **format template**, not a design system itself. -It is used in two phases by two different audiences: - -| Phase | Who | Reads | Produces | Needs | -|-------|-----|-------|----------|------| -| **A — authoring** | Agent + user | The **format spec** | A **project DESIGN.md** | Reasoning guidance: what concepts to elicit, how to choose patterns/affordances/feedback, what sections to fill | -| **B — building** | Agent | The **project DESIGN.md** | **Actual UI** (HTML + behavioral threads) | Declarative inventory: exact `--*` tokens, declared `affordances:` / `feedback:` / `patterns:` maps, prose explaining intent | - -The format spec therefore contains: **mechanism** (how `b-scale`, `@scope`, -DSD, custom properties work — stable, shared) + **starter vocabulary** -(default `affordances:` / `feedback:` / `patterns:` maps — shared defaults a -project overrides/extends) + **section templates** (what each body section -must contain — the project fills in) + **reasoning guidance** (woven into -section descriptions, teaching the Phase A agent how to elicit and choose). - -A project DESIGN.md is the **instance**: specific tokens, selected/customized -affordances, feedback states, and patterns, filled-in prose. The format -spec's frontmatter defaults are overridable; the project's frontmatter is the -conformance contract a Phase B agent generates against. - -## What lives where (the layer separation) - -**Locked** — the spec separates three layers, each with a different -relationship to HTML and to the framework: - -| Layer | What it declares | Appears in HTML? | Example | -|-------|-----------------|------------------|---------| -| **Structural** (`b-scale`, `patterns:`) | What a node *is* and what may nest inside it; the structural shape of a region | **Yes** — `b-scale` governs DOM nesting, so it appears in markup | `b-scale="s3"` (a block); a `Stream` pattern | -| **Functional** (`affordances:`, `feedback:`) | Named *interaction intents* and *loop response states* — the vocabulary reusable thread objects compose from | **No** — affordances and feedback are properties of *behavioral logic* (thread objects), not of HTML elements; the HTML is downstream of the thread, styled by the token bundles the vocabulary declares | `affordances: { danger, primary, secondary }`; `feedback: { error, confirmation, pending, success }` | -| **Expressive** (`--*` tokens, carrier model) | The *visual values* — CSS custom properties, resolved per scope via inheritance + `@scope` / DSD | Yes — as CSS custom properties, but the *names* come from the vocabulary, not from a mode attribute | `'--color-primary': "#1A1C1E"`; a `danger` affordance's token-bundle override | - -The key move: **affordances and feedback states are vocabulary for reusable -thread objects (behavioral structure), not HTML attributes.** A "danger -affordance" is not ` - -``` - -### Worked example — self-contained template (DSD) - -```html - - -``` - -`:host([b-scale="…"])` makes the host's light-DOM structural attribute drive -styling inside the shadow; inherited `--density-base` from a light ancestor -pierces in unless `:host()` redefines it. - -## Structural patterns - -**Locked (mechanism); placement open (vocabulary)** — the spec declares a -`patterns:` frontmatter map. Each pattern has the four Structural-IA -attributes: - -| Attribute | Meaning | In frontmatter | -|-----------|---------|----------------| -| **Content** | What activities/interactions take place; the *goal* for the user | Prose string | -| **Structure** | How information is organized; innate mechanics | Prose string | -| **Boundary** | What information shares in/out; permissions (prose contract, not a CSS mechanism) | Prose string | -| **Scale** | Which `b-scale` value this pattern occupies | Enum: `s1`–`s6` | - -### Starter vocabulary (default, overridable) - -The format spec ships a default `patterns:` map drawn from Structural IA: - -- **Blocks (S3):** Pools, Streams, Feeds, Collections, Walls, Threads -- **Platform structures (S7):** Strict Hierarchy, Nested Pools, Nested - Channels, Hypertext, Daisy, Multi-Dimensional Hierarchy - -A project DESIGN.md **overrides/extends** this: it selects which patterns it -uses, fills in project-specific Content/Boundary prose, and may declare its -own domain patterns. The project's `patterns:` frontmatter is what a Phase -B agent composes from. - -## Relationship to the original DESIGN.md - -The original DESIGN.md is purely the **expression layer** (Alexander: visual -atoms). This derivative adds a **structure layer** (Wurman/Jaffe: functional -units) and a **functional vocabulary layer** (affordances/feedback for -behavioral threads), connected by density-from-scale. The agent designs -top-down: function → structure → expression. - -| Original DESIGN.md | Effect | Status | -|---|---|---| -| **Design Tokens (frontmatter)** — grouped dot-notation, `{ref}` syntax, `components:` map | Transformed: flat `--*`→value map + `affordances:` / `feedback:` / `patterns:` maps; `var()` replaces `{ref}`; **no `components:` token block** | Transformed (kept) | -| **Overview** | Reframed for Behavioral/HTML-first + two-phase usage + functional flow | Keep | -| **Colors** | Prose; tokens live as `--color-*` in frontmatter | Keep | -| **Typography** | Keep; `font-size`/`line-height` couple to scale-implied density via `calc`/`em` | Keep | -| **Layout** | Reframe: density (from scale) × relationship-multipliers + regions, not grid + T-shirt scale | Keep (reframed) | -| **Elevation & Depth** | Prose; optional `--elevation-*` | Keep | -| **Shapes** | `--radius-*` custom properties | Keep | -| **Components** — token map with variant keys (`button-primary`, `button-primary-hover`) | **Eliminated.** No variant keys, no `components:` block. Components are `.html` templates styled with `--*` + `@scope`/`:host()`/`::part()`; their file format is an agent concern. | Eliminate | -| **Modes** | **Dropped.** Replaced by `affordances:` + `feedback:` functional vocabulary (for thread objects, not HTML attributes). | Drop / replace | -| **Structural Scale & Patterns** | *(NEW)* — `b-scale` (S1–S6 + `rel`), nesting constraint, density-from-scale, `patterns:` frontmatter map (overridable defaults) | Add | -| **Functional Vocabulary** | *(NEW)* — `affordances:` + `feedback:` maps, reusable thread objects, functional flow reasoning (substrate-neutral) | Add | -| **Do's and Don'ts** | Keep; scale + affordance/feedback-specific guidance | Keep | - -Net: the spec now has three layers — **structural** (`b-scale` + patterns, -appears in HTML), **functional** (affordances + feedback, vocabulary for -thread objects, not in HTML), and **expressive** (`--*` tokens + carrier -model). `b-scale` is the only spec attribute in HTML. `p-mode` and -`p-density` are gone. The original's `components:` token map is eliminated -(the anti-pattern of variant-per-intent token proliferation that structural -patterns + functional vocabulary dissolve). - -## Substrate facts (gathered from the codebase) - -These were looked up rather than grilled — they are facts about how Behavioral -actually works, not decisions. - -| Surface | What it does | Relevance | -|---------|--------------|-----------| -| **Controller floors + schemas-as-data** (`src/controller/`) | `isInvalidTrigger`/`detectXssVectors` as hardcoded invariants; `html.schemas.ts`/`css.schemas.ts` as pure schema data (classifier context, not compiled validators); . | Deterministic floor + probabilistic ceiling — the retired html tools' validation story, decomposed. | -| **Controller** (browser, `src/controller/controller.ts`) | WebSocket-push-driven; binds `b-trigger`/`b-form` in light DOM; applies `render`/`attrs`/`dispatch_custom_event`/`navigate`/`scale_check`. Swaps fragments via `