diff --git a/architecture.md b/architecture.md index 734a1f10e..f2a72a543 100644 --- a/architecture.md +++ b/architecture.md @@ -3746,6 +3746,23 @@ by registration order. They may refuse; they may not complete. ### Capability-backed execution +`Execution.init(): Operation` establishes live dependencies once in each +existing execution-owned scope. Its public default is no-op; middleware installs +ordinary package behavior and delegates. Core invokes it after copying trusted +host contributions and arming their revocation, before provider/installation +callbacks, documentation capture and public execution policy. Setup receives no +installation array, request, entry identifier or receipt and creates no second +execution owner. + +Initialization failure prevents policy, root import and body. Failure and +cancellation join partially acquired teardown before the invocation settles. +Ordinary descendants inherit initialized state; concurrent executions and REPL +entries own distinct state. Completed handle observation initializes nothing. +An isolated run child reinstalls the selected host/Plugin middleware after cwd +assembly; arbitrary parent handlers do not cross that isolation. REPL Agent +attachment and durable preparation are owned and ordered elsewhere, and +initialization neither replaces nor reorders them. + Canonical core is authoritative for document execution. It invokes the stable `Execution` middleware through a private, per-invocation same-name Api whose instance-owned default handler is the authoritative terminal. A stable name @@ -5067,11 +5084,11 @@ cannot repair a missing or multiply selected domain. Paired producers retain their lexical site route and operational permission. A trusted provider may therefore delegate and register canonical self-closing -`` as a component answer. Its bare form reports the admitted vocabulary; -its named form renders the enclosing authoring documentation with narrowed -availability. Verified protected documentation provenance survives sealing even -when the delegating provider has a different origin. That provenance is -descriptive data and never a callable lookup key. +`` as a component answer. Its default bare form reports the admitted +vocabulary; its named form renders the enclosing authoring documentation with +narrowed availability. Verified protected documentation provenance survives +sealing even when the delegating provider has a different origin. That +provenance is descriptive data and never a callable lookup key. Routes, projection edges, bodies, domains, callable sets and site operations belong to one live execution. Child closure and execution teardown invalidate @@ -5233,40 +5250,58 @@ declared origin and digest as its origin. Its private closure contributes nothing: those names are not syntax a document may write, so a set of symbols that listed them would describe an environment that does not exist. -**An execution carries one of its own, lexically.** The symbols a running -document is shown travel by value on canonical core's private expansion -environment, beside component resolution and the identity domains — not through a -Context, because a context resolves by name and a name is not a secret, so a -document could build one and answer for the vocabulary it is shown. - -Canonical core builds it at the execution root from the selection inputs that -execution captured before any installation, middleware or document code ran: the -includes it resolves against, the registry it started with, the identity -components and exact Markdown its host declared, and the component bundle it is -closed over when it has one. A bundled name is described from the pinned bytes -already in hand — nothing is imported, executed, or read from a file to describe -one — and reported at the canonical repository-relative path that blob has in the -commit, which is where a reader of a workflow run looks for it. - -A trusted host may state the symbols its profile describes instead, on the terms -every other trusted-host value travels: captured by value with the rest of the -installation, before any installed code exists. One execution accepts one, and -two are refused rather than ordered, because ordering them would make which -profile a document is shown depend on assembly order. `xmd plan` states one — a -Plan is a program a later `xmd run` executes, so the vocabulary the agent must be -shown is that profile's rather than the authorship execution's, which searches no -repository and refuses almost every capability. An ordinary run states none and -describes itself. - -Nothing is built until an occurrence asks, so a run whose document never writes -`` enumerates no includes, parses no component and reads no -frontmatter. Each ask builds afresh, which is what makes an occurrence's retained -symbols its own rather than a copy of whichever one ran first. - -A trusted canonical evaluation boundary may install a narrower reference for -the subtree it evaluates, and leaving that subtree restores the enclosing one. It -adds nothing: the symbols it installs are the ones its own admission already -selected, so an entry absent from that admission is absent from the reference. +**Live descriptions compose through Syntax.** Core exports `Syntax` and +`SyntaxApi` from `@executablemd/core/api`. Its `symbols` and `text` properties +are `Operation` and `Operation`; callers yield the +properties without parentheses. `documentation(names)` is an Operation-valued +method. Defaults are reusable lazy Operations, so each live read starts fresh +work. Default text renders the effective contextual symbols through the shared +Markdown renderer. Middleware may answer, delegate or transform these views. + +The execution-derived private reference supplies the fallback. It is built from +the selection inputs that execution captured before any installation, +middleware or document code ran: the includes it resolves against, the registry +it started with, the identity components and exact Markdown its host declared, +and the component bundle it is closed over when it has one. A bundled name is +described from the pinned bytes already in hand — nothing is imported, executed, +or read from a file to describe one — and reported at the canonical +repository-relative path that blob has in the commit, which is where a reader of +a workflow run looks for it. It is bound as the Api's fallback rather than as +inner middleware, so it does not shadow an enclosing host profile. + +`useSyntax(source)` from `@executablemd/core/host` binds a host's lazy complete +descriptive profile and snapshots the documentation contributions already in its +calling scope. It installs ordinary symbols and documentation +providers at `min` without reading the source. There is no installation-member +count rule for descriptive providers. + +Standalone Plan enters the ordinary run profile's declarative bootstraps and +installs this helper before its constrained authorship execution. Its source +inspects captured target registry/declaration inputs lazily, so named metadata +also describes the program it is preparing. An embedded Plan keeps its enclosing +profile. CLI inspection uses the same bounded declarative assembly and helper, +without running execution initialization or operational providers. + +Named documentation retains the enclosing catalog for metadata and long-form +text and reads effective contextual symbols once for availability. Each +occurrence resolves its authoring basis once; its default symbols provider +reuses that basis rather than inspecting a moving root twice. Later occurrences +resolve independently. A symbols filter can hide an entry from the bare view +while named documentation still explains it as unavailable. Text-only +middleware changes neither structured symbols nor named lookup. + +Generated producer and body scopes narrow symbols through ordinary middleware +while keeping the enclosing named-documentation owner. Leaving those scopes +restores the enclosing view. Synthetic answers that do not delegate perform no +default inspection. Descriptions register, resolve and authorize nothing, so a +custom view may describe a forbidden name without admitting it, or hide an +admitted name without disabling it. + +The protected Syntax component records the final middleware answer inside its +existing durable live callback. Completed replay restores that final text +without consulting any live symbols, text or catalog provider. Its authenticated +invocation, safe canonical named refusal, publication and secret gate remain +separate from descriptive composition. **Inspection is observation, never execution.** Producing symbols installs only the declarative registration layer selection needs. It does not enter diff --git a/packages/cli/src/cli.ts b/packages/cli/src/cli.ts index 0cea3443a..8b90a99de 100755 --- a/packages/cli/src/cli.ts +++ b/packages/cli/src/cli.ts @@ -142,8 +142,9 @@ import { componentSearchPath, resolveTestTarget } from "./test-target.ts"; import { renderSyntaxDocumentation, renderSyntaxJson, - renderSyntaxMarkdown, + syntaxText, syntaxSymbols, + useCommandSyntax, } from "./syntax.ts"; import { assembleRunProfile } from "./run-profile.ts"; import type { PluginModuleLoader } from "./plugin-loader.ts"; @@ -2858,7 +2859,11 @@ function* dispatch( { ...(sessions === undefined ? {} : { sessions }), plugins, - symbols: (includes) => syntaxSymbols(includes, plugins), + // What the Plan being written will be run under. The same declarative + // assembly `xmd syntax` prints from, entered inside the authorship + // scope so the documentation this host's packages contribute is + // captured with it. + installSyntax: (includes) => useCommandSyntax(includes, plugins), // The two facts about this process's own stderr that nothing further // in may go and read: whether it is a terminal, and whether it took // what it was handed. The approved Plan's sinks are stdout and @@ -3083,10 +3088,9 @@ function* dispatch( // The compact list of symbols, unchanged: routine discovery output and // every default Plan prompt read it, and long documentation would make // both unnecessarily large. - const catalog = yield* syntaxSymbols(command.config.include, plugins); rendered = command.config.json - ? renderSyntaxJson(catalog) - : renderSyntaxMarkdown(catalog); + ? renderSyntaxJson(yield* syntaxSymbols(command.config.include, plugins)) + : yield* syntaxText(command.config.include, plugins); } else { // The same selection, index and renderer `` uses, so // the command and the component cannot describe one component two diff --git a/packages/cli/src/plan-writer-profile.ts b/packages/cli/src/plan-writer-profile.ts index d8f2b8b27..de5648f81 100644 --- a/packages/cli/src/plan-writer-profile.ts +++ b/packages/cli/src/plan-writer-profile.ts @@ -45,7 +45,7 @@ import { useTerminalOutput, } from "@executablemd/core"; import type { AgentProviderOptions, Json } from "@executablemd/core"; -import type { SyntaxSymbolsProvider, MarkdownComponent } from "@executablemd/core/host"; +import type { MarkdownComponent } from "@executablemd/core/host"; import { executeInstalled, installInvocationAgentProvider } from "@executablemd/core/host"; import { createAcpxProvider } from "@executablemd/acp"; import type { AcpxProviderDependencies } from "@executablemd/acp"; @@ -170,15 +170,16 @@ export interface PlanWriterProfile { */ declaration: MarkdownComponent; /** - * The symbols this authorship describes. + * Install the description this authorship shows. * - * The `run` profile's, because a Plan is a program a later `xmd run` executes: - * deriving them from this execution — which searches no repository and - * refuses almost every capability — would describe a vocabulary the approved - * program would not have. Captured with the rest of the installation, before - * any installed code, middleware or document code runs. + * Invoked inside the bounded scope below, before the execution exists, which + * is the only place a target-run profile can be both bootstrapped and + * captured: the helper core exports snapshots the documentation contributions + * that are in scope when it is called, so a host that bootstrapped elsewhere + * and handed over a bare symbols callback would lose them, and named metadata + * would fall back to this restricted authorship execution's own catalog. */ - symbols: SyntaxSymbolsProvider; + installSyntax(): Operation; } /** What building the constrained provider needs, and nothing more. */ @@ -428,84 +429,88 @@ export function* runPlanCommandDocument(profile: PlanWriterProfile): Operation` the Component writes reads the profile the Plan + // is being written for. Nothing further in may state another. + yield* profile.installSyntax(); + const source = yield* readPackagedDocument(PLAN_COMMAND_DOCUMENT); + try { + // The presentation every phase this command renders passes through. A raw + // capture would show an operator whitespace nobody wrote, and terminal + // formatting belongs to a terminal — which is the host's fact about its + // own stream, not something this module may go and detect. + yield* useNormalizedOutput(); + if (profile.progress.terminal) { + yield* useTerminalOutput(); + } - // The command root is a thin adapter: it projects the request into - // `` and returns what comes back. Everything that used to be - // installed around it — the directory, the provider, the Elicitation, the - // refusals — is installed by the Component itself, inside the invocation that - // owns it, so the command and an ordinary document run one workflow rather - // than two arrangements of one. - const execution = yield* executeInstalled( - { - ...retainedSource(PLAN_COMMAND_IDENTITY, source), - stream: profile.stream, - // No repository component search. What the adapter may name is - // what this command declares, so a file in the caller's tree - // cannot answer for `` or anything else. - includes: [], - // The one secret boundary this command has. Every live durable event - // crosses the serialized pre-append gate, so a draft or a diagnostic - // becomes readable — to the journal and to the phase that presents it - // — only after the event supplying it has cleared and committed. - secretDetection: true, - props: { - request: profile.request, - session: profile.session, - }, - }, - [ + // The command root is a thin adapter: it projects the request into + // `` and returns what comes back. Everything that used to be + // installed around it — the directory, the provider, the Elicitation, the + // refusals — is installed by the Component itself, inside the invocation that + // owns it, so the command and an ordinary document run one workflow rather + // than two arrangements of one. + const execution = yield* executeInstalled( { - components: agentIdentityComponents(), - declarations: [profile.declaration], - // What a Plan may write is the `run` profile's vocabulary, not this - // authorship execution's — the adapter searches no repository, the - // Component installs a ceiling that refuses almost everything, and - // symbols derived from *that* would describe a program nobody can - // run. Captured before any installation runs, so no prop, binding, - // middleware answer or projected content reaches it. - symbols: profile.symbols, - // The ordinary run's evaluation ceiling, so an information request - // this authorship evaluates reads through exactly what `xmd run` - // states — File, Glob and canonical Syntax, with the write table - // present and never selected. The host installs it, not ``: - // a component that installed its own would be choosing its - // ceiling instead of being given one. - evaluation: ordinaryEvaluationProfile(), + ...retainedSource(PLAN_COMMAND_IDENTITY, source), + stream: profile.stream, + // No repository component search. What the adapter may name is + // what this command declares, so a file in the caller's tree + // cannot answer for `` or anything else. + includes: [], + // The one secret boundary this command has. Every live durable event + // crosses the serialized pre-append gate, so a draft or a diagnostic + // becomes readable — to the journal and to the phase that presents it + // — only after the event supplying it has cleared and committed. + secretDetection: true, + props: { + request: profile.request, + session: profile.session, + }, }, - ], - ); - - // Drains while the document is still producing. A destination that fails - // raises here, and leaving this scope cancels the execution and waits for - // its teardown before anything is reported. - yield* forEach(function* (chunk: string) { - const written = yield* profile.progress.write(chunk); - if (!written.ok) { - throw new ProgressDeliveryError(written.error); + [ + { + components: agentIdentityComponents(), + declarations: [profile.declaration], + // The ordinary run's evaluation ceiling, so an information request + // this authorship evaluates reads through exactly what `xmd run` + // states — File, Glob and canonical Syntax, with the write table + // present and never selected. The host installs it, not ``: + // a component that installed its own would be choosing its + // ceiling instead of being given one. + evaluation: ordinaryEvaluationProfile(), + }, + ], + ); + + // Drains while the document is still producing. A destination that fails + // raises here, and leaving this scope cancels the execution and waits for + // its teardown before anything is reported. + yield* forEach(function* (chunk: string) { + const written = yield* profile.progress.write(chunk); + if (!written.ok) { + throw new ProgressDeliveryError(written.error); + } + }, execution.output); + + const completed = yield* execution; + if (!completed.ok) { + return completed; } - }, execution.output); - - const completed = yield* execution; - if (!completed.ok) { - return completed; - } - if (typeof completed.value !== "string") { - return Err(new Error("the plan command document returned something that is not a Plan")); + if (typeof completed.value !== "string") { + return Err(new Error("the plan command document returned something that is not a Plan")); + } + return Ok(completed.value); + } catch (error) { + return Err(error instanceof Error ? error : new Error(String(error))); } - return Ok(completed.value); - } catch (error) { - return Err(error instanceof Error ? error : new Error(String(error))); - } + }); }); } diff --git a/packages/cli/src/plan.ts b/packages/cli/src/plan.ts index 4c0d82aa1..8659b8112 100644 --- a/packages/cli/src/plan.ts +++ b/packages/cli/src/plan.ts @@ -40,7 +40,6 @@ import { randomUUID } from "node:crypto"; import { resolve } from "node:path"; import process from "node:process"; -import type { SyntaxSymbols } from "@executablemd/core"; import type { AcpxProviderDependencies } from "@executablemd/acp"; import { InMemoryStream } from "@executablemd/durable-streams"; import type { DurableStream } from "@executablemd/durable-streams"; @@ -89,8 +88,21 @@ export interface PlanDependencies { sessions?: MachineSessionAssembly; /** What the profile's provider is built on, beyond the host's own assembly. */ acp?: AcpxProviderDependencies; - /** The run profile's complete structured vocabulary. */ - symbols(includes: readonly string[]): Operation; + /** + * Install the description this authorship shows, in the scope it is given. + * + * A setup operation rather than a symbols callback, because binding the + * profile and building it are two different moments: the host enters the + * target run's declarative bootstraps, so the documentation its packages + * contribute is in scope when `useSyntax()` snapshots it, and only the lazy + * source it binds there runs later, when an occurrence renders. + * + * Production installs the `run` profile's: a Plan is a program a later + * `xmd run` executes, so the vocabulary the agent must be shown is that + * profile's rather than the authorship execution's, which searches no + * repository and refuses almost every capability. + */ + installSyntax(includes: readonly string[]): Operation; /** Who answers the review question. */ installElicitation(): Operation; /** @@ -232,13 +244,7 @@ export function* runPlan(command: PlanCommand, deps: PlanDependencies): Operatio declaration, stream, progress: deps.progress, - // The vocabulary this authorship describes is the `run` profile's, not - // this execution's: a Plan is a program a later `xmd run` executes, so the - // symbols the Agent must be shown are the ones that run will have. Stated - // at the execution boundary and captured before any installed code — no - // prop on the thin adapter, and nothing the Component projects, could - // supply another. - symbols: () => deps.symbols(command.include), + installSyntax: () => deps.installSyntax(command.include), }); } catch (error) { console.error(describeError(error)); diff --git a/packages/cli/src/repl/expansion.ts b/packages/cli/src/repl/expansion.ts index 5725f7f30..c44d4944a 100644 --- a/packages/cli/src/repl/expansion.ts +++ b/packages/cli/src/repl/expansion.ts @@ -112,15 +112,17 @@ export const COMPONENT_BOUNDARIES: BoundaryPartition = { * exits until the run is over, which makes it the right anchor for *which* * expansion is being paused and a useless place to rest. * - * `execute` is not an expansion boundary. It is the request-policy seam, asked - * once before a document exists, and canonical core dispatches it through a - * private instance — so holding it would delay building the run rather than - * stopping its expansion, and would not reliably be reached at all. + * Neither `init` nor `execute` is an expansion boundary. `execute` is the + * request-policy seam, asked once before a document exists, and canonical core + * dispatches it through a private instance — so holding it would delay building + * the run rather than stopping its expansion, and would not reliably be reached + * at all. `init` is earlier still: it establishes the run's live dependencies + * before there is any expansion to step. */ export const EXECUTION_BOUNDARIES: BoundaryPartition = { walks: ["document"], gates: [], - unheld: ["execute"], + unheld: ["execute", "init"], }; /** diff --git a/packages/cli/src/syntax.ts b/packages/cli/src/syntax.ts index 1a860891b..45de9fdc4 100644 --- a/packages/cli/src/syntax.ts +++ b/packages/cli/src/syntax.ts @@ -29,15 +29,13 @@ import { planComponentDescription } from "./plan-component.ts"; import { scoped } from "effection"; +import { Component, Syntax } from "@executablemd/core/api"; +import { useSyntax } from "@executablemd/core/host"; import type { Operation } from "effection"; import { agentIdentityComponents, - capturedDocumentation, - documentationIndexFor, inspectSyntax, - renderSelectedDocumentation, renderSyntaxMarkdown, - selectDocumented, useAgentComponents, } from "@executablemd/core"; import type { SyntaxSymbols } from "@executablemd/core"; @@ -74,24 +72,58 @@ export function* syntaxSymbols( plugins: CommandPlugins = NO_PLUGINS, ): Operation { return yield* scoped(function* () { - yield* useCommandComponents(); - return yield* profileSymbols(includes, plugins); + yield* useCommandSyntax(includes, plugins); + return yield* Syntax.operations.symbols; }); } -/** The profile's symbols, inside a scope that has already bootstrapped it. */ -function* profileSymbols( +/** + * Describe this scope with the command's profile. + * + * The one descriptive setup every `xmd syntax` projection and `xmd plan` + * authorship goes through, so an operator printing a profile and an agent being + * told to write against it are shown one construction. + * + * Capture now, inspect later. The bootstraps are entered and their registry and + * declarations are read here, into one object the source closes over, because + * those are the *target* inputs: what this command installed, before anything + * else registers. Inspection itself is deferred to the source, so a command + * that never renders a description enumerates no includes and parses no + * component. Calling the helper in this same scope is also what captures the + * documentation those bootstraps contributed. + */ +export function* useCommandSyntax( includes: readonly string[], - plugins: CommandPlugins, -): Operation { - return yield* inspectSyntax({ - includes, + plugins: CommandPlugins = NO_PLUGINS, +): Operation { + yield* useCommandComponents(); + const inputs = { + includes: [...includes], + registry: new Map(yield* Component.operations.registry), components: agentIdentityComponents(), // `` is part of the run profile, so symbols that left it out would // describe a vocabulary no run has. Described from the packaged bytes: // inspection mints nothing, so it reports the Component's identity and // contract without building the capabilities only a run can build. declarations: commandDeclarations(plugins, yield* planComponentDescription()), + }; + yield* useSyntax(() => inspectSyntax(inputs)); +} + +/** + * The profile's symbols as Markdown, through the contextual construction. + * + * Read from the Api rather than rendered from the value beside it, so this + * command prints what a document writing `` under the same profile + * would be shown — including whatever middleware the profile installed. + */ +export function syntaxText( + includes: readonly string[], + plugins: CommandPlugins = NO_PLUGINS, +): Operation { + return scoped(function* () { + yield* useCommandSyntax(includes, plugins); + return yield* Syntax.operations.text; }); } @@ -185,9 +217,7 @@ export function* renderSyntaxDocumentation( plugins: CommandPlugins = NO_PLUGINS, ): Operation { return yield* scoped(function* () { - yield* useCommandComponents(); - const catalog = yield* profileSymbols(includes, plugins); - const index = documentationIndexFor(yield* capturedDocumentation()); - return renderSelectedDocumentation(selectDocumented(catalog, catalog, names, index)); + yield* useCommandSyntax(includes, plugins); + return yield* Syntax.operations.documentation(names); }); } diff --git a/packages/cli/tests/plan-cli.test.ts b/packages/cli/tests/plan-cli.test.ts index ad759b24f..705baa31d 100644 --- a/packages/cli/tests/plan-cli.test.ts +++ b/packages/cli/tests/plan-cli.test.ts @@ -36,6 +36,7 @@ import { fileURLToPath } from "node:url"; import process from "node:process"; import * as cliModule from "../src/cli.ts"; +import { syntaxSymbols } from "../src/syntax.ts"; import { runPlan } from "../src/plan.ts"; import type { PlanCommand } from "../src/plan.ts"; import type { PlanWriterStack } from "../src/agent-stack.ts"; @@ -1805,19 +1806,22 @@ describe( it("PO15: the catalog is built once, from inside the command document", function* () { yield* useWorkingDirectory(function* (dir, planWriterRoot) { - const harness = createPlanHarness({ planWriterRoot }); - harness.fake.script({ reply: PLAIN }); - harness.script({ decision: "Approve" }); - // Recorded against the progress the drain had at the time, so what this // observes is that Preparing reached the operator before the catalog - // was read rather than merely that both happened. + // was read rather than merely that both happened. Observed at the lazy + // source rather than at the setup that binds it: binding happens when + // the authorship scope opens, and what this case is about is when the + // description is *built*. const before: string[][] = []; - const catalog = harness.deps.symbols; - harness.deps.symbols = function* (includes) { - before.push(phasesOf(harness.progress.join(""))); - return yield* catalog(includes); - }; + const harness = createPlanHarness({ + planWriterRoot, + *symbols(includes) { + before.push(phasesOf(harness.progress.join(""))); + return yield* syntaxSymbols(includes); + }, + }); + harness.fake.script({ reply: PLAIN }); + harness.script({ decision: "Approve" }); const { value } = yield* delivered(harness, () => runPlan(planning(dir), harness.deps)); diff --git a/packages/cli/tests/plan-command-document.test.ts b/packages/cli/tests/plan-command-document.test.ts index a084d27eb..0f182934a 100644 --- a/packages/cli/tests/plan-command-document.test.ts +++ b/packages/cli/tests/plan-command-document.test.ts @@ -53,6 +53,7 @@ import type { } from "@executablemd/core/host"; import { ordinaryEvaluationProfile } from "../src/evaluation-profile.ts"; import { elicitWriteEntry } from "@executablemd/core/host"; +import { useSyntax } from "@executablemd/core/host"; import { Elicitation } from "@executablemd/core"; import { recordedFiles, useRecordedFiles } from "../../core/tests/support/fragment-files.ts"; import type { RecordedFiles } from "../../core/tests/support/fragment-files.ts"; @@ -229,6 +230,7 @@ function* runDocument(options: RunOptions = {}): Operation { // would show an operator whitespace nobody wrote. yield* useNormalizedOutput(); try { + yield* useSyntax(harness.symbols); const execution = yield* executeInstalled( { ...retainedSource(PLAN_COMMAND_IDENTITY, source), @@ -244,7 +246,6 @@ function* runDocument(options: RunOptions = {}): Operation { { components: agentIdentityComponents(), declarations: [harness.declaration], - symbols: harness.symbols, // The ceiling the command installs, so an information request this // document evaluates reads through exactly what `xmd run` states. // A case that supplies its own recorder gets that instead, which is @@ -569,6 +570,7 @@ describe("the packaged plan command document", () => { yield* scoped(function* () { yield* installAgentComponents({ defaultAgent: AGENT, permissionMode: "deny-all" }); yield* useNormalizedOutput(); + yield* useSyntax(harness.symbols); const execution = yield* executeInstalled( { ...retainedSource(PLAN_COMMAND_IDENTITY, source), @@ -581,7 +583,6 @@ describe("the packaged plan command document", () => { { components: agentIdentityComponents(), declarations: [installed], - symbols: harness.symbols, }, ], ); diff --git a/packages/cli/tests/plan-component.test.ts b/packages/cli/tests/plan-component.test.ts index 545ba54eb..fcc9a1daa 100644 --- a/packages/cli/tests/plan-component.test.ts +++ b/packages/cli/tests/plan-component.test.ts @@ -40,6 +40,7 @@ import { syntaxReadEntry, } from "@executablemd/core/host"; import type { FragmentEvaluationInput } from "@executablemd/core/host"; +import { useSyntax } from "@executablemd/core/host"; import { API, Permissions } from "@executablemd/runtime"; import { recordedFiles, useRecordedFiles } from "../../core/tests/support/fragment-files.ts"; import type { RecordedFiles } from "../../core/tests/support/fragment-files.ts"; @@ -192,6 +193,7 @@ function* runDocument(options: { yield* useNormalizedOutput(); } try { + yield* useSyntax(harness.symbols); const execution = yield* executeInstalled( { ...retainedSource(ROOT, options.source), @@ -206,7 +208,6 @@ function* runDocument(options: { // Where the profile a document observes is settled now: the // `` the packaged Plan writes is canonical core's public // component, and what it answers with is this execution's. - symbols: harness.symbols, ...(options.evaluation === undefined ? {} : { evaluation: options.evaluation }), ...(options.files === undefined && options.installing === undefined ? {} diff --git a/packages/cli/tests/repl-execution.test.ts b/packages/cli/tests/repl-execution.test.ts index 7fd51da3b..fe9a1369e 100644 --- a/packages/cli/tests/repl-execution.test.ts +++ b/packages/cli/tests/repl-execution.test.ts @@ -18,7 +18,7 @@ import { expect } from "@executablemd/test-support/expect"; import { useTempFileCompiler } from "@executablemd/core"; import { InMemoryStream, serializeDurableEvent } from "@executablemd/durable-streams"; import type { DurableEvent } from "@executablemd/durable-streams"; -import { ensure, race, scoped, sleep, spawn, until } from "effection"; +import { createContext, ensure, race, scoped, sleep, spawn, until } from "effection"; import type { Operation, Result } from "effection"; import { ensureDir, rm, writeTextFile } from "@effectionx/fs"; import { API } from "@executablemd/runtime"; @@ -919,3 +919,43 @@ describe("REPL execution: one execution's file", () => { expect(yield* opened.stream.readAll()).toEqual([]); }); }); + +describe("execution initialization in REPL entries", () => { + it("EI-REPL: two entries own fresh state below the same session attachment", function* () { + const State = createContext("executablemd.test.repl.init", undefined); + const initialized: object[] = []; + const observed: object[] = []; + yield* Execution.around({ + *init(_args, next) { + const state = {}; + initialized.push(state); + yield* State.set(state); + yield* next(); + }, + *document([request], next) { + const state = yield* State.expect(); + if (state === undefined) { + throw new Error("entry has no initialized state"); + } + observed.push(state); + yield* next(request); + }, + }); + const session = opened( + yield* submitReplEntry({ ...options(execution()), source: "first entry\n" }), + ); + yield* session.join(); + const submitted = yield* session.submit("second entry\n"); + if (!submitted.ok) { + throw submitted.error; + } + yield* session.join(); + expect(initialized).toHaveLength(2); + expect(observed).toEqual(initialized); + expect(initialized[0]).not.toBe(initialized[1]); + expect(yield* State.get()).toBe(undefined); + expect(session.model.entries).toHaveLength(2); + yield* session.join(); + expect(initialized).toHaveLength(2); + }); +}); diff --git a/packages/cli/tests/support/plan-harness.ts b/packages/cli/tests/support/plan-harness.ts index 943e5320c..5725ce662 100644 --- a/packages/cli/tests/support/plan-harness.ts +++ b/packages/cli/tests/support/plan-harness.ts @@ -29,7 +29,8 @@ import { planComponentDeclaration } from "../../src/plan-component.ts"; import type { PlanSurface, StructuralValidation } from "../../src/plan-component.ts"; import { planAgentContext } from "../../src/plan-writer-profile.ts"; import type { PlanWriterStack } from "../../src/agent-stack.ts"; -import type { SyntaxSymbolsProvider, MarkdownComponent } from "@executablemd/core/host"; +import { useSyntax } from "@executablemd/core/host"; +import type { MarkdownComponent } from "@executablemd/core/host"; import type { PlanDependencies } from "../../src/plan.ts"; import { createFakeAcp, makeRegistry, makeStore } from "./fake-acp.ts"; import type { FakeAcp, FakeStore } from "./fake-acp.ts"; @@ -184,9 +185,14 @@ export function createPlanHarness(options: { sessionStore: options.store ?? makeStore(), agentRegistry: makeRegistry({ [AGENT]: `${AGENT}-cmd` }), }, - *symbols(includes) { - symbolCalls.push([...includes]); - return yield* (options.symbols ?? syntaxSymbols)(includes); + *installSyntax(includes) { + // The harness's own lazy source, bound through the helper production + // uses — so what a case replaces is the description, not the way it + // reaches the execution. + yield* useSyntax(function* () { + symbolCalls.push([...includes]); + return yield* (options.symbols ?? syntaxSymbols)(includes); + }); }, planWriterRoot: options.planWriterRoot, *installElicitation() { @@ -287,7 +293,7 @@ export interface PlanDeclarationHarness { * public, canonical core owns it, and what it answers with is the execution's * own. */ - symbols: SyntaxSymbolsProvider; + symbols: () => Operation; /** How many times that provider was asked. */ symbolCalls: number; /** Review answers, taken in order. Running out is a test defect, not a case. */ diff --git a/packages/cli/tests/testing-execution-host.test.ts b/packages/cli/tests/testing-execution-host.test.ts index c07b20c12..9359a7139 100644 --- a/packages/cli/tests/testing-execution-host.test.ts +++ b/packages/cli/tests/testing-execution-host.test.ts @@ -19,7 +19,7 @@ import { expect } from "@executablemd/test-support/expect"; import { Err, Ok, ensure, resource, scoped, spawn, withResolvers } from "effection"; import type { Operation } from "effection"; import { ensureDir, readdir, rm, writeTextFile } from "@effectionx/fs"; -import { mkdtemp } from "node:fs/promises"; +import { mkdtemp, realpath } from "node:fs/promises"; import { until } from "effection"; import { join } from "node:path"; import { tmpdir } from "node:os"; @@ -1097,3 +1097,53 @@ describe("deterministic dependencies declared for a nested run", () => { expect(result.code).toBe(0); }); }); + +describe("execution initialization in isolated run children", () => { + it("EI-child: chosen Plugin setup is reinstalled after cwd and parent middleware stays outside", function* () { + const project = yield* useProject({}); + const api = new URL("../../core/api.ts", import.meta.url).href; + const runtime = new URL("../../runtime/mod.ts", import.meta.url).href; + // ESM by extension, because the project is a bare directory with no + // `package.json`: a `.ts` plugin there loads as CommonJS under Node, and + // the dependencies core reaches through publish an import condition only. + yield* writeTextFile( + join(project, "init-plugin.mjs"), + [ + `import { Execution, Plugin, Syntax } from ${JSON.stringify(api)};`, + `import { cwd } from ${JSON.stringify(runtime)};`, + "const calls = [];", + "export default Plugin({ name: 'init-evidence', *install(request) {", + " yield* Execution.around({ *init(_args, next) {", + " calls.push(`${request.command}@${yield* cwd()}`);", + " yield* Syntax.around({ *text() { return calls.join(','); } }, { at: 'min' });", + " yield* next();", + " } });", + " return undefined;", + "} });", + ].join("\n"), + ); + const canonical = yield* until(realpath(project)); + const expected = `test@${canonical},run@${canonical}`; + yield* writeTextFile( + join(project, "README.md"), + doc( + '', + '\\n"} as="child">', + '', + '', + "", + ``, + "", + "", + ), + ); + const result = yield* runCli(["test", "--plugin", "./init-plugin.mjs", "README.md"], { + cwd: project, + }).join(); + if (result.code !== 0) { + throw new Error(result.stdout + result.stderr); + } + expect(result.code).toBe(0); + expect(result.stdout + result.stderr).not.toContain("❌"); + }); +}); diff --git a/packages/core/api.ts b/packages/core/api.ts index fee42b6e1..531aaec45 100644 --- a/packages/core/api.ts +++ b/packages/core/api.ts @@ -128,3 +128,29 @@ export type { ComponentExpansionRequest, } from "./src/component-api.ts"; export type { Expansion } from "./src/expansion.ts"; + +/** + * The execution a host composes around, and the requests its layers are handed. + * + * Three seams. `init` is the one a command host installs on: it establishes + * the live dependencies this execution runs on, once, inside the scope the + * invocation already owns. `execute` is the request-policy seam and `document` + * wraps the whole expansion; both of those defaults refuse, because only + * canonical core completes a document. + */ +export { Execution } from "./src/execute.ts"; +export type { ExecutionApi } from "./src/execute.ts"; +export type { ExecutionRequest } from "./src/execution-request.ts"; +export type { DocumentRequest } from "./src/document-request.ts"; + +/** + * The description in effect, as the thing a consumer composes on. + * + * The same descriptor the root exports, reachable from here as well because a + * consumer installing middleware imports from `./api`. What the three + * properties answer, and what composing on them does and does not change, is + * in `src/syntax-api.ts`; `./host`'s `useSyntax` is how a host supplies one. + */ +export { Syntax } from "./src/syntax-api.ts"; +export type { SyntaxApi } from "./src/syntax-api.ts"; +export type { SyntaxSymbols } from "./src/inspect.ts"; diff --git a/packages/core/host.ts b/packages/core/host.ts index f9d3e8ac4..422701678 100644 --- a/packages/core/host.ts +++ b/packages/core/host.ts @@ -189,12 +189,6 @@ export { generatedRequestRefusal } from "./src/generated-request-refusal.ts"; */ export { classifyPlanResponse } from "./src/plan-response.ts"; export type { PlanResponseKind } from "./src/plan-response.ts"; -/** - * The symbols a host's profile describes, when they are not the ones the - * execution would derive from its own captured inputs — see - * `src/syntax-reference.ts`. - */ -export type { SyntaxSymbolsProvider } from "./src/syntax-reference.ts"; export type { DurablePreparation } from "./src/document-request.ts"; /** @@ -407,3 +401,18 @@ export type { AnswerConfiguration, AnswerMatcher } from "./src/answers.ts"; */ export { DeclarationScan } from "./src/declaration-scan.ts"; export type { AnswersPlacement, DeclarationScanner } from "./src/declaration-scan.ts"; + +/** + * Where a trusted host states the description its profile shows. + * + * A setup operation rather than a value on the installation, because what it + * binds is two things: the lazy source, and the documentation contributions + * that are in scope where it is called. `xmd plan` needs both — a Plan is a + * program a later `xmd run` executes, so the vocabulary the agent must be shown + * is that profile's, and so is the long-form prose a named lookup renders. + * + * It reads nothing when called and installs ordinary providers, so nearer + * middleware still answers, delegates or transforms. Stating a description + * confers no execution authority. + */ +export { useSyntax } from "./src/use-syntax.ts"; diff --git a/packages/core/mod.ts b/packages/core/mod.ts index be32a01a2..8e898209f 100644 --- a/packages/core/mod.ts +++ b/packages/core/mod.ts @@ -256,7 +256,8 @@ export { } from "./src/documentation-api.ts"; export type { DocumentationApi } from "./src/documentation-api.ts"; export { select as selectDocumented } from "./src/syntax-reference.ts"; -export { NO_DOCUMENTATION, UnknownComponentError } from "./src/documentation-index.ts"; +export { UnknownComponentError } from "./src/documentation-index.ts"; +export { NO_DOCUMENTATION } from "./src/syntax-markdown.ts"; export type { DocumentationIndex } from "./src/documentation-index.ts"; export { PROTECTED_COMPONENT_NAMES, ProtectedComponentError } from "./src/components/protected.ts"; export { SYNTAX_COMPONENT } from "./src/components/Syntax.ts"; @@ -450,3 +451,10 @@ export type { FileSecretFinding } from "./src/secrets/files.ts"; // here can disable, replace, or outlive the detection an execution runs under. export { scanSecrets, secretPolicy } from "./src/secrets/policy.ts"; export type { SecretPolicy } from "./src/secrets/policy.ts"; + +// The description in effect here. The same descriptor `./api` exports — one +// stable name, so middleware composes across separately loaded copies — kept at +// the root as well because reading a description is ordinary work. What the +// properties answer is in `src/syntax-api.ts`. +export { Syntax } from "./src/syntax-api.ts"; +export type { SyntaxApi } from "./src/syntax-api.ts"; diff --git a/packages/core/src/components/Syntax.ts b/packages/core/src/components/Syntax.ts index 6982b8e55..6b4e47c07 100644 --- a/packages/core/src/components/Syntax.ts +++ b/packages/core/src/components/Syntax.ts @@ -3,39 +3,41 @@ * * An author asking "which components do I have here?" and an agent being told * what to write are the same question, and `xmd syntax` already answers it from - * outside. This is the same answer from inside: the symbols for the site the - * element was written at, as the Markdown that command prints. + * outside. This is the same answer from inside: the description in effect at + * the site the element was written at, as the Markdown that command prints. * * ## Why canonical core owns it * - * The symbols describe the vocabulary an execution actually has. A repository - * `Syntax.md`, a bundled `Syntax`, a registration, an import handler or a - * second loaded copy answering for the name would each describe a vocabulary - * the run does not have — to whoever is reading, and to whichever agent is - * being told what to write next. So the name is reserved by the canonical - * protected tier, ahead of every host and author tier, and the definition - * canonical core selected is what runs. + * The description is what an agent is told to write next, so whoever answers + * for it decides what gets written. A repository `Syntax.md`, a bundled + * `Syntax`, a registration, an import handler or a second loaded copy answering + * for the name would each do that from outside the composition the run + * assembled. So the name is reserved by the canonical protected tier, ahead of + * every host and author tier, and the definition canonical core selected is + * what runs. * - * Protection is about the *answer*, not about power. The component receives one - * reference that renders symbol text and nothing else: no definitions, no - * import witness, no invocation capability, no policy table, no provider and no - * registration handle. Naming a component in the symbols is not permission to - * run it. + * Protection is about *who answers*, not about power. The description itself is + * ordinary composed state: a host binds its own profile, a trusted boundary + * narrows it, and middleware may transform the text — none of which registers, + * resolves or admits anything. Naming a component in a description is not + * permission to run it, and hiding one does not disable it. * * ## What one occurrence does * - * It authenticates the occurrence identity this execution minted, renders - * once, and retains exactly what it rendered. A continuation reads that record - * and hands the same text back without consulting the filesystem, the - * registry, the bundle, the host or the lexical reference again — so an agent - * resuming authorship is shown the vocabulary the run actually showed it, not - * one rebuilt from a tree that has moved since. + * It authenticates the occurrence identity this execution minted, asks the + * contextual `Syntax` Api inside its own durable live callback, and retains + * exactly the final answer that came back. A continuation reads that record and + * hands the same text back without consulting the filesystem, the registry, the + * bundle, the host, or any live symbols, text or catalog provider again — so an + * agent resuming authorship is shown the vocabulary the run actually showed it, + * not one rebuilt from a tree, or a composition, that has moved since. */ import { createDurableOperation, StaleInputError } from "@executablemd/durable-streams"; import type { Json as DurableJson, Workflow } from "@executablemd/durable-streams"; import type { Operation } from "effection"; +import { Syntax } from "../syntax-api.ts"; import { getExpansion } from "../expansion.ts"; import { markGeneratedRequestRefusal } from "../generated-request-refusal.ts"; import { SyntaxSelectionRefusal } from "../syntax-refusal.ts"; @@ -132,7 +134,6 @@ function syntax(claim: IdentityClaimant): ProtectedBody { invocation: ComponentInvocation, site: ProtectedSite, ): Operation { - const reference = site.syntax; // Read off the issuance the engine holds rather than off a method the // caller could have written, and answered before anything is claimed or // rendered: a paired spelling is a document asking for something this @@ -155,7 +156,12 @@ function syntax(claim: IdentityClaimant): ProtectedBody { // that somebody validated them. const names = requestedNames(props.names); const id = yield* claim(invocation); - if (reference === undefined) { + // Site validation, kept where the element is: an expansion driven outside + // an execution established no description at all, so there is nothing for + // the contextual Api to answer from. Answering that here rather than + // leaving it to the Api's own default is what makes the refusal name the + // element a reader is looking at. + if (site.syntax === undefined) { throw new Error(NO_REFERENCE_REFUSAL); } const expansion = yield* getExpansion(); @@ -169,8 +175,8 @@ function syntax(claim: IdentityClaimant): ProtectedBody { return { symbols: names === undefined - ? yield* reference.symbols() - : yield* reference.documentation(names), + ? yield* Syntax.operations.text + : yield* Syntax.operations.documentation(names), }; } catch (error) { if (error instanceof SyntaxSelectionRefusal) { diff --git a/packages/core/src/documentation-index.ts b/packages/core/src/documentation-index.ts index 63a1cd07a..819bf7020 100644 --- a/packages/core/src/documentation-index.ts +++ b/packages/core/src/documentation-index.ts @@ -113,9 +113,6 @@ export interface DocumentationIndex { bundleDocumentation(origin: ComponentOrigin): string | undefined; } -/** What a component with no authored documentation renders instead of prose. */ -export const NO_DOCUMENTATION = "No long-form documentation is available for this component."; - /** A level-two heading, captured without its marker. */ const HEADING = /^##\s+(.+?)\s*$/; /** Any ATX heading, so a deeper one can be told from a section boundary. */ diff --git a/packages/core/src/execute.ts b/packages/core/src/execute.ts index 978be5411..37785e884 100644 --- a/packages/core/src/execute.ts +++ b/packages/core/src/execute.ts @@ -158,6 +158,7 @@ import { import { coreComponentDefinition, CORE_ORIGIN } from "./components/registry.ts"; import { CORE_REVISION } from "./generated-xmd.ts"; import { rootSyntaxReference } from "./syntax-reference.ts"; +import { useRootSyntaxReference } from "./syntax-api.ts"; import { capturedDocumentation } from "./documentation-api.ts"; import { CANONICAL_PROFILE_ANSWERS, @@ -176,7 +177,6 @@ import { componentAnswerRegistrar } from "./component-answers.ts"; import type { ComponentAnswerInstallation } from "./component-answers.ts"; import { packagedAssetReader } from "./component-documentation.ts"; import type { DocumentationContribution, DocumentationReader } from "./component-documentation.ts"; -import type { SyntaxSymbolsProvider } from "./syntax-reference.ts"; import type { WorkflowComponentBundle, WorkflowComponentCatalog } from "./components/bundle.ts"; import type { CodeBlockContext, CodeBlockResult, EvalEnv } from "./types.ts"; import { readRootSource, rootSourcePath } from "./root-source.ts"; @@ -2417,13 +2417,12 @@ function* executeDocument( bundles: readonly WorkflowComponentBundle[] = [], identityComponents: readonly IdentityComponent[] = [], installed: readonly RetainedInstallation[] = [], - providers: readonly SyntaxSymbolsProvider[] = [], /** * The documentation each bootstrapped package contributed. * - * Carried by value from the collection boundary, like the symbols provider - * beside it, so the index a document's own `` reads is the - * index the profile actually assembled. + * Carried by value from the collection boundary, so the index a document's + * own `` reads is the index the profile actually + * assembled. */ documentation: readonly DocumentationContribution[] = [], /** This execution's packaged-asset reader, carried by value from the caller. */ @@ -2442,8 +2441,8 @@ function* executeDocument( * The one evaluation profile this host stated, copied and bound, when it * stated one. * - * Carried by value like the symbols provider beside it. It is sealed below, - * once the provider-backed names it admits have been resolved, and the sealed + * Carried by value from the collection boundary. It is sealed below, once + * the provider-backed names it admits have been resolved, and the sealed * result is handed to core's own expansion on the private environment rather * than through any context: it is the ceiling `` narrows from, and a * document that could reach it could raise it. @@ -2702,6 +2701,25 @@ function* executeDocument( return imports === undefined ? definition : imports.issue(name, definition); } + // Built from what this execution captured before any installation, + // middleware or document code ran, and asked only when an occurrence + // renders: a run whose document never writes `` enumerates + // nothing. + const derivedSyntax = rootSyntaxReference( + { + includes, + registry: startingRegistry, + components: identityComponents, + declarations: installed.flatMap((one) => [...one.declarations]), + ...(bundle === undefined ? {} : { workflow: bundle }), + }, + documentation, + ); + // Bound as the contextual Api's fallback, not installed in front of + // anything: a host that stated its own profile with `useSyntax()` still + // answers, and this is what answers when none did. + yield* useRootSyntaxReference(derivedSyntax); + const environment: ExecutionEnvironment = { componentResolution: imports, // Everything this host declared, admitted above. Expansion asks it @@ -2716,21 +2734,7 @@ function* executeDocument( // document, a component, middleware or a separately loaded copy can // name reaches this object. sourceSegments: createExactSource(), - // Built from what this execution captured before any installation, - // middleware or document code ran, and asked only when an occurrence - // renders: a run whose document never writes `` enumerates - // nothing. - syntax: rootSyntaxReference( - { - includes, - registry: startingRegistry, - components: identityComponents, - declarations: installed.flatMap((one) => [...one.declarations]), - ...(bundle === undefined ? {} : { workflow: bundle }), - }, - providers[0], - documentation, - ), + syntax: derivedSyntax, // The ceiling a generated fragment is evaluated under, when this host // offers evaluation at all. Absent is a host that offers none, and // `` refuses on that rather than inventing one. @@ -3008,24 +3012,6 @@ export interface ExecutionInstallation { * operation after its invocation. */ readonly components?: readonly IdentityComponent[]; - /** - * The symbols this host's profile describes, when its profile is not the one - * the execution itself would derive. - * - * Omitted is the ordinary case — canonical core derives the symbols from the - * selection inputs this execution captured, which is what makes an ordinary - * run's reference the run's own. - * - * `xmd plan` states one, because the Plan being written is a program a later - * `xmd run` executes: the vocabulary the agent must be shown is that profile's - * rather than the authorship execution's. One execution accepts one. - * - * Documentation does not travel here. It is additive — a profile installing - * four packages has four boundaries — so each package's bootstrap contributes - * its own through the `Documentation` Api, and this execution collects them - * once at the same boundary. - */ - readonly symbols?: SyntaxSymbolsProvider; /** * The maximum permission a generated fragment may be evaluated under here. * @@ -3055,13 +3041,41 @@ export interface ExecutionInstallation { /** * Execution Api — a policy surface around document execution. * - * A handler is given an `ExecutionRequest`, not the execution. It may inspect - * the options, narrow or replace them with `withOptions()`, register an - * additive completion failure, install contextual behavior the document will - * inherit, refuse by throwing, and delegate. It returns nothing, and whatever - * it returns is ignored: only canonical execution completes a document. + * No handler is given the execution, and whatever a handler returns is + * ignored: only canonical execution completes a document. What a handler *is* + * given differs by member — `init` nothing, `execute` an `ExecutionRequest`, + * `document` a `DocumentRequest` — and each one below says what it may decide + * with what it was handed. Any of them may install contextual behavior the + * document will inherit, refuse by throwing, and delegate. */ export interface ExecutionApi { + /** + * Establish the live dependencies this execution runs on. + * + * The seam a command host uses to install what it provides — a Files + * provider, a recorder, a service — once per execution, inside the scope the + * invocation already owns. So a resource acquired here belongs to this + * execution, ordinary descendants inherit it, two concurrent executions get + * their own, and teardown is joined before the invocation settles. + * + * It receives nothing. There is no request to brand, no installation array + * to read and no receipt to return, because initialization is not a decision + * about this run: it is the run's dependencies being put in place. A layer + * establishes its state and delegates with `yield* next()`. + * + * Invoked after the trusted host contributions are copied and their + * revocation armed, and before installation callbacks, documentation capture + * and `execute`'s policy — so what a host installs here is visible to all + * three, and nothing a host installs here can reach back and change what was + * already captured. + */ + init(): Operation; + /** + * The policy seam for one execution request. + * + * A handler may inspect the options, narrow or replace them with + * `withOptions()`, register an additive completion failure, and delegate. + */ execute(request: ExecutionRequest): Operation; /** * The document's expansion, as `durableRun` runs it. @@ -3082,6 +3096,11 @@ export interface ExecutionApi { * handed to it. Canonical core dispatches through a private instance instead. */ export const Execution: Api = createApi("Execution", { + // No-op rather than a refusal, unlike the two below. Initialization is not a + // protocol a caller can complete on core's behalf — there is nothing to + // settle and nothing to record — so an execution with no host middleware + // simply has no live dependencies to establish, and invents none. + *init() {}, // deno-lint-ignore require-yield *execute(_request: ExecutionRequest): Operation { throw new ExecutionProtocolError("invoked execution outside canonical core"); @@ -3098,10 +3117,11 @@ export const Execution: Api = createApi("Execution", * * The order is the contract. Admissions are copied and frozen first, so what * ends up authoritative is fixed before any installation, any middleware and - * any document code exists. Installations then run, then the chain is invoked - * with one opaque request, then the request must have reached the terminal - * exactly once, and only then does canonical core execute the document with the - * options the terminal recorded. + * any document code exists. Initialization then establishes the live + * dependencies, installations run, and the chain is invoked with one opaque + * request; then the request must have reached the terminal exactly once, and + * only then does canonical core execute the document with the options the + * terminal recorded. * * One structured owner task holds the invocation scope. Everything an * installation established, and every child the document spawned, lives inside @@ -3528,23 +3548,6 @@ function* invoke( }), ); - // Read once and frozen with the rest, and before any installation runs: which - // profile a document is shown is settled before anything can replace it. Two - // are refused rather than ordered — symbols chosen by installation order - // would make what an agent is told to write depend on assembly order. - const providers = Object.freeze( - installations.flatMap((installation) => { - const provider = installation.symbols; - return provider === undefined ? [] : [provider]; - }), - ); - if (providers.length > 1) { - throw new Error( - "two installations stated the symbols this execution describes. One execution describes " + - "one vocabulary, so which profile a document is shown is never a question of order.", - ); - } - // What each installation will actually be asked to do, read once and in // order, before the first of them runs. Both members are read here for the // reason everything else on the installation is: a property that answered @@ -3604,6 +3607,15 @@ function* invoke( }); canonicalImports.activate(); + // The host's live dependencies, established here: after everything trusted + // has been copied and its revocation armed, so initialization cannot reach + // back and change what is already authoritative, and before the installation + // callbacks and documentation capture below, so what it establishes is there + // for all of them. In this scope, which the owner task holds — so what a + // layer acquires is torn down with this execution and joined before it + // settles, and a failure here reaches no policy, no root import and no body. + yield* Execution.operations.init(); + // Each provider installer first, then that installation's ordinary `install`, // in the order the installations were captured. A provider registers // middleware through its registrar, and each invocation states what it is @@ -3659,7 +3671,6 @@ function* invoke( bundles, identityComponents, installed, - providers, documentation, readAsset, canonicalImports, diff --git a/packages/core/src/expand.ts b/packages/core/src/expand.ts index 648f57e48..aefab6143 100644 --- a/packages/core/src/expand.ts +++ b/packages/core/src/expand.ts @@ -122,6 +122,7 @@ import { isPrivateImplementation, issueInvocation } from "./invocation-identity. import type { IdentityDomain } from "./invocation-identity.ts"; import { protectedContentLease } from "./protected-content.ts"; import type { SyntaxReference } from "./syntax-reference.ts"; +import { useNarrowedSyntax } from "./syntax-api.ts"; import { withInvocation } from "./invocation.ts"; import { observeExpansion } from "./component-expansion.ts"; import type { ObservedExpansion } from "./component-expansion.ts"; @@ -369,6 +370,17 @@ interface ProjectionState { */ checkedFailures: CheckedFailures | undefined; environment: ExecutionEnvironment | undefined; + /** + * The admission a protected producer's content is narrowed to, when one + * narrowed it. + * + * Set only where a boundary replaced the reference, and reported to the + * contextual Api in the content scope below. The ordinary handle leaves it + * absent on purpose: the execution's own reference is already the Api's + * fallback, and reinstalling it here as middleware would shadow a host that + * stated its own profile. + */ + narrowedSyntax: SyntaxReference | undefined; imports: ComponentImportTerminal | undefined; } @@ -493,6 +505,9 @@ function createProjectionHandle(state: ProjectionState): ProjectionHandle { const task = contentScope.scope.run(function* () { try { yield* ErrorMode.set(options.mode); + if (state.narrowedSyntax !== undefined) { + yield* useNarrowedSyntax(state.narrowedSyntax); + } if (options.segments.length === 0) { outcome.resolve({ segments: [] }); return; @@ -592,6 +607,9 @@ function createProjectionHandle(state: ProjectionState): ProjectionHandle { const task = contentScope.scope.run(function* () { try { yield* ErrorMode.set(mode); + if (state.narrowedSyntax !== undefined) { + yield* useNarrowedSyntax(state.narrowedSyntax); + } if (!slotErrorsEmitted && slots.errors.length > 0) { slotErrorsEmitted = true; for (const slotError of slots.errors) { @@ -3074,6 +3092,7 @@ function* expandComponent( invocation, enclosing, children, + narrowedSyntax: undefined, caller: { env: capturedCallerEnv, meta: callerMeta, @@ -3587,6 +3606,7 @@ function* expandFunctionComponent( projecting: issued.projecting, enclosing, children, + narrowedSyntax: undefined, caller: { env: undefined, meta: callerMeta, @@ -3741,6 +3761,7 @@ function* expandFunctionComponent( // content scope, not a second expansion. const narrowed = createProjectionHandle({ ...projectionState, + narrowedSyntax: syntax, // Only the reference is replaced, and only where there is an // environment to replace it on: an expansion driven without // one narrows nothing because it admitted nothing. diff --git a/packages/core/src/generated-xmd.ts b/packages/core/src/generated-xmd.ts index 0b4101818..818e7ffa3 100644 --- a/packages/core/src/generated-xmd.ts +++ b/packages/core/src/generated-xmd.ts @@ -157,6 +157,7 @@ import { installFormSyntax, invocationForm } from "./invocation-identity.ts"; import type { FormSyntax, ComponentRouting } from "./invocation-identity.ts"; import type { ComponentInvocation } from "./invocation-identity.ts"; import type { SyntaxReference } from "./syntax-reference.ts"; +import { useNarrowedSyntax } from "./syntax-api.ts"; import type { ComponentElement, FunctionComponentDefinition, @@ -2443,6 +2444,12 @@ function expand( ): Operation { return scoped(function* () { yield* ErrorMode.set("throw"); + // The admission this fragment was evaluated under, reported for the length + // of the fragment. Named documentation stays with the enclosing owner, so a + // generated body can still be told about a component it may not run. + if (syntax !== undefined) { + yield* useNarrowedSyntax(syntax); + } // Every expression this expansion resolves — an ordinary prop and a // declared capture alike — reads the declarative grammar rather than the // trusted-document evaluator. Set on this scope, so it ends with the diff --git a/packages/core/src/syntax-api.ts b/packages/core/src/syntax-api.ts new file mode 100644 index 000000000..84b4299ff --- /dev/null +++ b/packages/core/src/syntax-api.ts @@ -0,0 +1,131 @@ +/** + * The description in effect here, as a contextual Api. + * + * ``, `xmd syntax` and a Plan's authorship all ask the same three + * questions — what are the symbols, what is the text, what do these names + * mean — of one descriptor, dispatched by name, so a host, a trusted boundary + * or ordinary middleware can answer, delegate or transform any of them. + * + * ## Why the properties are Operations + * + * `symbols` and `text` are values a caller reads, not calls it makes, and they + * are also work: a description is inspected lazily, from a filesystem, on + * demand. `Operation`-valued properties are both at once. A caller writes + * `yield* Syntax.operations.text` with no parentheses, and a provider's body + * may suspend, read a file and delegate — with no cast hiding an operation + * behind a string. + * + * The defaults are *reusable* operations. `call(factory)` starts fresh + * generator work on each read, so two reads of one property are two answers + * rather than one consumed generator and then nothing. + * + * ## The fallback is not middleware + * + * An execution derives a description from what it captured + * (`syntax-reference.ts`), and that has to be what answers when nothing else + * does — while still losing to a host that stated its own profile. Those are + * different slots, and conflating them is the bug this module exists to avoid: + * + * - {@link useRootSyntaxReference} binds the execution's own reference as the + * value the *default* reads. A default is the furthest thing from the + * caller, so an enclosing `useSyntax()` profile still wins. + * - {@link useNarrowedSyntax} installs real middleware at `min` for a + * boundary that has admitted less than its enclosing site. That one is meant + * to shadow: it is reporting an admission, not offering a fallback, and it + * ends with the scope that installed it. + * + * Neither grants anything, and neither takes anything away: a description that + * names a component does not admit it, and one that hides a component does not + * disable it. Component selection, invocation identity and durable recording + * stay canonical whatever the description says, which is why composing on this + * Api is ordinary rather than privileged work. The boundary itself is + * architecture.md's *The syntax symbols boundary*. + */ + +import { type Api, createApi } from "@effectionx/context-api"; +import { call, createContext, type Operation } from "effection"; +import type { SyntaxReference } from "./syntax-reference.ts"; +import type { SyntaxSymbols } from "./inspect.ts"; +import { renderSyntaxMarkdown } from "./syntax-markdown.ts"; + +/** The three questions anything describing this site answers. */ +export interface SyntaxApi { + /** The effective structured description of this site. */ + readonly symbols: Operation; + /** + * The effective text, which is what `xmd syntax` prints. + * + * Its *default* renders {@link SyntaxApi.symbols}. Middleware may answer with + * text of its own, so the two are not two views of one value by construction. + */ + readonly text: Operation; + /** These components' metadata and long-form documentation. */ + documentation(names: readonly string[]): Operation; +} + +/** + * The execution's own reference, read by the defaults below. + * + * A context rather than installed middleware, because this is the answer of + * last resort: middleware would sit in front of an enclosing host profile and + * shadow it, which is precisely backwards. + */ +const DefaultReference = createContext( + "executablemd.core.syntax.defaultReference", + undefined, +); + +function* reference(): Operation { + const value = yield* DefaultReference.get(); + if (value === undefined) { + throw new Error("Syntax has no descriptive reference in this scope."); + } + return value; +} + +export const Syntax: Api = createApi("executablemd.core.syntax", { + symbols: call(function* (): Operation { + return yield* (yield* reference()).symbols(); + }), + // Through the Api, not through the reference: a nearer provider that replaced + // the symbols is describing this site, so the text has to render what that + // provider said rather than what the fallback would have. + text: call(function* (): Operation { + return renderSyntaxMarkdown(yield* Syntax.operations.symbols); + }), + documentation(names): Operation { + return call(function* () { + return yield* (yield* reference()).documentation(names); + }); + }, +}); + +/** + * The authoring basis one named lookup is resolving, while it resolves it. + * + * Set by the reference that owns the lookup and read by its own default symbols + * provider, so an occurrence that needs both its metadata and its availability + * inspects a moving root once rather than twice. Nothing outside that one + * lookup's scope sees it. + */ +export const NamedBasis = createContext( + "executablemd.core.syntax.namedBasis", + undefined, +); + +/** Bind the reference this execution derived as the Api's fallback answer. */ +export function* useRootSyntaxReference(value: SyntaxReference): Operation { + // A fresh execution resolves its own basis. Started inside a named lookup — + // an isolated child, or a fragment evaluated while one is open — it would + // otherwise inherit that occurrence's basis as its root symbols. + yield* NamedBasis.set(undefined); + yield* DefaultReference.set(value); +} + +/** Report a narrowed admission for the scope a boundary is evaluating. */ +export function useNarrowedSyntax(value: SyntaxReference): Operation { + // Symbols only. Named documentation stays with the enclosing owner, because a + // subtree that may run less is still entitled to read about what it may not + // run — and the entry it reads says so. + return Syntax.around({ symbols: () => value.symbols() }, { at: "min" }); +} diff --git a/packages/core/src/syntax-markdown.ts b/packages/core/src/syntax-markdown.ts index 730fad365..eb1396c2e 100644 --- a/packages/core/src/syntax-markdown.ts +++ b/packages/core/src/syntax-markdown.ts @@ -24,9 +24,20 @@ import type { StructuralSyntaxEntry, SyntaxSymbols, } from "./inspect.ts"; -import { NO_DOCUMENTATION } from "./documentation-index.ts"; import type { ComponentOrigin, Json, PropsSchema } from "./types.ts"; +/** + * What a component with no authored documentation renders instead of prose. + * + * Here, with the renderer that emits it, rather than with the index that + * returns nothing for such a component: an absent entry and the sentence + * standing in for one are different facts, and only the renderer needs the + * second. Keeping it here is also what lets this module stay a leaf of the + * documentation index, so the contextual `Syntax` Api can reach the renderer + * without the component registry reaching back through it. + */ +export const NO_DOCUMENTATION = "No long-form documentation is available for this component."; + /** The three category kinds, taken from the symbols rather than restated. */ type CategoryKind = SyntaxSymbols["categories"][number]["kind"]; diff --git a/packages/core/src/syntax-reference.ts b/packages/core/src/syntax-reference.ts index 0169f2adf..73a9c33ed 100644 --- a/packages/core/src/syntax-reference.ts +++ b/packages/core/src/syntax-reference.ts @@ -1,38 +1,40 @@ /** * What a document may write here, as one thing an execution carries. * - * `xmd syntax` answers that question for an environment nobody is running. - * Canonical `` answers it for the site an element was actually written - * at, and the two have to be the same answer — symbols an agent is shown and - * symbols an operator prints describe one vocabulary or they describe none. + * `xmd syntax` answers that question for an environment nobody is running, and + * canonical `` answers it for the site an element was actually written + * at. Both reach the answer through the contextual `Syntax` Api, so there is + * one construction and one renderer, and this module owns what that Api falls + * back to when nothing nearer answers. * - * So there is one construction and one renderer, and this module is where an - * execution keeps its own use of them. The reference is built from the - * selection inputs the execution captured before any installation, middleware or - * document code ran: the includes it resolves against, the registry it started - * with, the identity components and exact Markdown its host declared, and the - * component bundle it is closed over when it has one. Nothing is read from a - * context, a registry answer, or anything a document can reach. + * Two references are built here. The execution's own is built from the + * selection inputs it captured before any installation, middleware or document + * code ran: the includes it resolves against, the registry it started with, the + * identity components and exact Markdown its host declared, and the component + * bundle it is closed over when it has one. A host's own is built from the lazy + * source `useSyntax()` was handed, closed over the documentation contributions + * that were in scope when it was called. `xmd plan` uses that one, because a + * Plan is a program a later `xmd run` executes: the vocabulary the agent must + * be shown is that profile's rather than the authorship execution's. * - * A trusted host may state the symbols for its own profile instead. `xmd plan` - * does: a Plan is written to be run by `xmd run`, so the vocabulary the agent - * must be shown is the run profile's rather than the authorship execution's. - * That provider is captured with the rest of the installation, before any - * installed code exists, and one execution accepts one — two are refused rather - * than ordered, because ordering them would make which profile a document is - * shown depend on installation order. + * Either way the reference holds the *default* answer, not the effective one. + * It is reached through ordinary contextual dispatch, so nearer middleware can + * answer instead, delegate to it, or transform what it returns — which is why + * `documentation()` asks the Api for the symbols availability is decided + * against rather than deciding that for itself. * - * The reference decides nothing at all. It answers with text. Seeing a - * component named in the symbols neither registers it, resolves it, nor + * The reference admits nothing at all. It answers with symbols and with text. + * Seeing a component named in either neither registers it, resolves it, nor * admits it: what a name means is still `selectComponent()`'s decision, and * what may run is still the execution's. */ -import type { Operation } from "effection"; +import { type Operation, scoped } from "effection"; +import { NamedBasis, Syntax } from "./syntax-api.ts"; import { inspectSyntax } from "./inspect.ts"; import type { SyntaxSymbols } from "./inspect.ts"; -import { renderSelectedDocumentation, renderSyntaxMarkdown } from "./syntax-markdown.ts"; +import { renderSelectedDocumentation } from "./syntax-markdown.ts"; import type { SelectedEntry } from "./syntax-markdown.ts"; import { documentationIndexFor } from "./component-documentation.ts"; import type { DocumentationContribution } from "./component-documentation.ts"; @@ -45,21 +47,23 @@ import { SyntaxSelectionRefusal } from "./syntax-refusal.ts"; import type { ComponentOrigin, ComponentRegistry } from "./types.ts"; /** - * The symbols in scope for the segments being expanded. + * The default description for the segments being expanded. * * Held by the execution and handed to core's own expansion by value, beside the - * component resolution and the identity domains. It is not a Context: a context - * resolves by name, and a name is not a secret, so a document could build one - * and answer for the vocabulary it is shown. + * component resolution and the identity domains: this is what the public + * `Syntax` Api answers from when nothing nearer does, not a second place where + * anything is decided. Composing in front of it is ordinary and expected. What + * protection keeps canonical is component selection, invocation identity and + * durable recording — none of which a description reaches. */ export interface SyntaxReference { - /** The symbols this site describes, rendered as Markdown. */ - symbols(): Operation; + /** The symbols this site describes, structured. */ + symbols(): Operation; /** * The selected components' metadata and long-form documentation. * * Two inputs, not one, and this is the reason the reference is an object - * rather than a string. *What may I write here* and *what may I read about* + * rather than a value. *What may I write here* and *what may I read about* * are different questions, and a narrowing evaluation boundary answers them * differently on purpose: the vocabulary it admits is smaller than the * vocabulary an author is entitled to understand. @@ -69,6 +73,10 @@ export interface SyntaxReference { * and each rendered entry states whether it is available in the current * evaluation. Collapsing the two would either hide reference material an * author needs or imply a permission they do not have. + * + * Availability is decided against the *effective* symbols, read once through + * the Api, so whatever customized the descriptive view here is the same thing + * that decides what this annotation says. */ documentation(names: readonly string[]): Operation; /** @@ -89,16 +97,6 @@ export interface SyntaxReference { available(symbols: SyntaxSymbols): SyntaxReference; } -/** - * A trusted host's statement of the symbols its profile describes. - * - * Captured by value with the rest of the installation, before any installed - * code, middleware or document code runs. It returns the symbols and core - * renders them, so a host cannot make its profile print differently from the - * way `xmd syntax` prints the same symbols. - */ -export type SyntaxSymbolsProvider = () => Operation; - /** The selection inputs an execution captured, as symbol construction reads them. */ export interface CapturedSymbolInputs { readonly includes: readonly string[]; @@ -124,7 +122,6 @@ export interface CapturedSymbolInputs { */ export function rootSyntaxReference( inputs: CapturedSymbolInputs, - provider: SyntaxSymbolsProvider | undefined, /** * The documentation the installed packages contribute. * @@ -135,9 +132,6 @@ export function rootSyntaxReference( */ contributions: readonly DocumentationContribution[] = [], ): SyntaxReference { - function* current(): Operation { - return provider === undefined ? yield* derived(inputs) : yield* provider(); - } // Snapshotted once, here, so the contributions a reference reads are the // ones the collection boundary captured rather than whatever the caller's // objects hold by the time a document asks. @@ -145,7 +139,7 @@ export function rootSyntaxReference( // No admission at a root: nothing has narrowed what may execute, so the one // set of symbols this resolves is both what a document may write and what it // may read about. - return referencing(current, undefined, captured); + return referencing(() => derived(inputs), undefined, captured); } /** @@ -171,31 +165,40 @@ function referencing( contributions: readonly DocumentationContribution[], ): SyntaxReference { return { - *symbols(): Operation { + *symbols(): Operation { // A narrowed reference reports its admission and asks the enclosing - // symbols for nothing — the bare form is about what runs. - return renderSyntaxMarkdown(admitted ?? (yield* authoring())); + // symbols for nothing — the bare form is about what runs. Otherwise this + // is the default answer, and the basis below is consulted first so that + // the one occurrence already resolving a basis gets that same value back + // instead of resolving a moving root a second time. + return admitted ?? (yield* NamedBasis.get()) ?? (yield* authoring()); }, - *documentation(names: readonly string[]): Operation { - // One resolution, both decisions. The provider is asked first and its - // failures propagate untouched: a symbols provider that throws is this - // run's infrastructure, whatever it happens to call its error. - const readable = yield* authoring(); - const runnable = admitted ?? readable; - const index = documentationIndexFor(contributions); - // Only the selection this module performs itself is restated under core's - // own namespaced identity. The provider has already returned, so nothing - // it raised can reach this. - let selected; - try { - selected = select(readable, runnable, names, index); - } catch (error) { - if (error instanceof UnknownComponentError) { - throw new SyntaxSelectionRefusal(error.message); + documentation(names: readonly string[]): Operation { + return scoped(function* () { + // One basis per occurrence, resolved here and published for the length + // of this lookup only. Availability is then read through the Api, so a + // host or boundary that customized what runs decides the annotation — + // and when that read reaches *this* reference's default, it comes back + // with the basis above rather than inspecting the root again. Two + // occurrences enter two scopes and resolve independently. + const readable = yield* authoring(); + yield* NamedBasis.set(readable); + const runnable = yield* Syntax.operations.symbols; + const index = documentationIndexFor(contributions); + // Only the selection this module performs itself is restated under + // core's own namespaced identity. Both symbol reads have already + // returned, so nothing either of them raised can reach this. + let selected; + try { + selected = select(readable, runnable, names, index); + } catch (error) { + if (error instanceof UnknownComponentError) { + throw new SyntaxSelectionRefusal(error.message); + } + throw error; } - throw error; - } - return renderSelectedDocumentation(selected); + return renderSelectedDocumentation(selected); + }); }, available(next: SyntaxSymbols): SyntaxReference { // The enclosing authoring symbols and the enclosing contributions, @@ -384,3 +387,22 @@ export function syntaxReference( captured, ); } + +/** + * The reference a host's own profile carries. + * + * What `useSyntax()` binds. The source is lazy for the reason the derived one + * is — a command that never renders a description inspects nothing — and the + * contributions are snapshotted here, from the scope that called the helper, + * because a host that bootstrapped a profile and then handed over a bare + * symbols callback would lose the index its own packages contributed. + * + * No admission: a host states one profile, and what narrows it afterwards is a + * canonical evaluation boundary rather than this. + */ +export function profileSyntaxReference( + source: () => Operation, + contributions: readonly DocumentationContribution[], +): SyntaxReference { + return referencing(source, undefined, snapshotContributions(contributions)); +} diff --git a/packages/core/src/use-syntax.ts b/packages/core/src/use-syntax.ts new file mode 100644 index 000000000..fb9688160 --- /dev/null +++ b/packages/core/src/use-syntax.ts @@ -0,0 +1,42 @@ +/** + * Binding a host's own description, as one setup operation. + * + * The thing a host actually has is not a set of symbols — it is a profile it + * can inspect, plus the documentation its packages contributed. Both have to be + * taken here, in the scope the host assembled, because only one of them is + * lazy: the source runs later, when an occurrence renders, but the + * contributions are gone as soon as that assembly's scope ends. A host that + * bootstrapped its profile and then handed over a bare symbols callback would + * keep the first and lose the second, and a named lookup would fall back to + * whichever catalog the execution happened to have. + */ + +import type { Operation } from "effection"; +import type { SyntaxSymbols } from "./inspect.ts"; +import { capturedDocumentation } from "./documentation-api.ts"; +import { Syntax } from "./syntax-api.ts"; +import { profileSyntaxReference } from "./syntax-reference.ts"; + +/** + * Describe this scope with the host's own profile. + * + * The source is not read here. Installing is binding, not inspecting, so a + * command that prints nothing and a run whose document never writes + * `` both enumerate nothing. + * + * Installed at `min`, which is the implementation slot: this is a host stating + * what it has, so ordinary middleware nearer the caller still answers, + * delegates to it or transforms what it returns. Symbols and named + * documentation both, because a host supplying one and not the other would + * describe its own vocabulary with somebody else's prose. + */ +export function* useSyntax(source: () => Operation): Operation { + const value = profileSyntaxReference(source, yield* capturedDocumentation()); + yield* Syntax.around( + { + symbols: () => value.symbols(), + documentation: ([names]) => value.documentation(names), + }, + { at: "min" }, + ); +} diff --git a/packages/core/tests/evaluate-component.test.ts b/packages/core/tests/evaluate-component.test.ts index d3a8aaa74..f16d00f28 100644 --- a/packages/core/tests/evaluate-component.test.ts +++ b/packages/core/tests/evaluate-component.test.ts @@ -34,6 +34,7 @@ import { import type { ExecutionInstallation, FragmentEvaluationInput } from "../host.ts"; import { registerComponents } from "../src/components/registration.ts"; import { elicitWriteEntry } from "../host.ts"; +import { useSyntax } from "../host.ts"; import { Elicitation } from "../src/elicitation-api.ts"; import type { ElicitationRequest } from "../src/elicitation-api.ts"; import { retainedSource } from "../src/root-source.ts"; @@ -1922,10 +1923,12 @@ describe("Tier FE34 — the shared read profile", () => { shared(files), { // deno-lint-ignore require-yield - *symbols(): Operation { - const error = new Error("the provider could not build the symbols"); - error.name = "executablemd.core.syntax-selection-refusal"; - throw error; + *install(): Operation { + yield* useSyntax(function* (): Operation { + const error = new Error("the provider could not build the symbols"); + error.name = "executablemd.core.syntax-selection-refusal"; + throw error; + }); }, }, ]); diff --git a/packages/core/tests/execution-init.test.ts b/packages/core/tests/execution-init.test.ts new file mode 100644 index 000000000..6b0696834 --- /dev/null +++ b/packages/core/tests/execution-init.test.ts @@ -0,0 +1,258 @@ +import { describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import { createContext, ensure, scoped, spawn, suspend, withResolvers } from "effection"; +import type { Operation } from "effection"; +import { InMemoryStream } from "@executablemd/durable-streams"; +import { Execution } from "../api.ts"; +import { collect, execute, Execution as RootExecution, inlineSource } from "../mod.ts"; +import { executeInstalled, fileReadEntry, Markdown, sourceDigest } from "../host.ts"; +import type { JournalAdmission } from "../host.ts"; +import { Documentation } from "../src/documentation-api.ts"; +import { recordedFiles, useRecordedFiles } from "./support/fragment-files.ts"; + +const State = createContext<{ id: number } | undefined>("executablemd.test.init.state", undefined); + +function* run(text = "hello\n"): Operation { + return String( + yield* collect( + yield* execute({ ...inlineSource(text), stream: new InMemoryStream(), includes: [] }), + ), + ); +} + +function* failure(operation: Operation): Operation { + try { + yield* operation; + return undefined; + } catch (error) { + return error; + } +} + +describe("Tier EI — execution-owned initialization", () => { + it("EI1: the public descriptors agree, default init is no-op and each fresh owner dispatches once", function* () { + expect(Execution).toBe(RootExecution); + yield* Execution.operations.init(); + expect(yield* State.get()).toBe(undefined); + let calls = 0; + yield* Execution.around({ + *init(_args, next) { + calls += 1; + yield* State.set({ id: calls }); + yield* next(); + }, + }); + const execution = yield* execute({ + ...inlineSource("hello\n"), + stream: new InMemoryStream(), + includes: [], + }); + expect(String(yield* collect(execution))).toContain("hello"); + const completed = yield* execution; + expect(yield* execution).toBe(completed); + expect(calls).toBe(1); + yield* run(); + expect(calls).toBe(2); + expect(yield* State.get()).toBe(undefined); + }); + + it("EI2: initialization precedes assembly, captured documentation and execution/document policy", function* () { + const order: string[] = []; + yield* Execution.around({ + *init(_args, next) { + order.push("init"); + yield* State.set({ id: 7 }); + yield* next(); + }, + *execute([request], next) { + expect((yield* State.expect())?.id).toBe(7); + order.push("execute"); + yield* next(request); + }, + *document([request], next) { + expect((yield* State.expect())?.id).toBe(7); + order.push("document"); + yield* next(request); + }, + }); + yield* Documentation.around({ + *contributions([read], next) { + expect((yield* State.expect())?.id).toBe(7); + order.push("documentation"); + return yield* next(read); + }, + }); + const execution = yield* executeInstalled( + { ...inlineSource("\n"), includes: [], stream: new InMemoryStream() }, + [ + { + *install() { + expect((yield* State.expect())?.id).toBe(7); + order.push("install"); + }, + }, + ], + ); + expect(String(yield* collect(execution))).toContain("### ``"); + expect(order).toEqual(["init", "install", "documentation", "execute", "document"]); + }); + + it("EI3: init cannot rewrite captured admissions, declarations or generated profile tables", function* () { + const admitted: string[] = []; + const admissions: JournalAdmission[] = [ + function* () { + admitted.push("original"); + }, + ]; + const source = "CAPTURED DECLARATION\n"; + const declarations = [ + Markdown({ name: "Kept", origin: "test://init/kept", source, digest: sourceDigest(source) }), + ]; + const read = [fileReadEntry()]; + const files = recordedFiles({ "note.md": "CAPTURED PROFILE" }); + yield* Execution.around({ + *init(_args, next) { + admissions.length = 0; + declarations.length = 0; + read.length = 0; + yield* useRecordedFiles(files); + yield* next(); + }, + }); + const text = "\n'} />\n"; + const execution = yield* executeInstalled( + { ...inlineSource(text), includes: [], stream: new InMemoryStream() }, + [{ admissions, declarations, evaluation: { read } }], + ); + const output = String(yield* collect(execution)); + expect(admitted).toEqual(["original"]); + expect(output).toContain("CAPTURED DECLARATION"); + expect(output).toContain("CAPTURED PROFILE"); + expect(files.performed).toEqual(["read note.md"]); + }); + + it("EI4: initializer failure settles only after acquired teardown and reaches no policy or root", function* () { + const entered = withResolvers(); + const release = withResolvers(); + const cleanup = withResolvers(); + const phases: string[] = []; + const stream = new InMemoryStream(); + yield* Execution.around({ + *init() { + yield* ensure(function* () { + cleanup.resolve(); + yield* release.operation; + phases.push("released"); + }); + entered.resolve(); + throw new Error("init refused"); + }, + *execute([request], next) { + phases.push("policy"); + yield* next(request); + }, + }); + let settled = false; + const running = yield* spawn(function* () { + const error = yield* failure( + execute({ ...inlineSource("must not run"), stream, includes: [] }), + ); + settled = true; + return error; + }); + yield* entered.operation; + yield* cleanup.operation; + expect(settled).toBe(false); + expect(stream.snapshot()).toEqual([]); + release.resolve(); + expect(String(yield* running)).toContain("init refused"); + expect(phases).toEqual(["released"]); + }); + + it("EI5: deterministic cancellation during init joins cleanup before halt returns", function* () { + const entered = withResolvers(); + const cleaning = withResolvers(); + const release = withResolvers(); + const phases: string[] = []; + const stream = new InMemoryStream(); + yield* Execution.around({ + *init() { + yield* ensure(function* () { + cleaning.resolve(); + yield* release.operation; + phases.push("released"); + }); + entered.resolve(); + yield* suspend(); + }, + *execute([request], next) { + phases.push("policy"); + yield* next(request); + }, + }); + const running = yield* spawn(() => + execute({ ...inlineSource("must not run"), stream, includes: [] }), + ); + yield* entered.operation; + let halted = false; + const halting = yield* spawn(function* () { + yield* running.halt(); + halted = true; + }); + yield* cleaning.operation; + expect(halted).toBe(false); + expect(stream.snapshot()).toEqual([]); + release.resolve(); + yield* halting; + expect(phases).toEqual(["released"]); + }); + + it("EI6: concurrent owners get independent state inherited by ordinary descendants", function* () { + let counter = 0; + const both = withResolvers(); + const release = withResolvers(); + const seen: number[] = []; + yield* Execution.around({ + *init(_args, next) { + yield* State.set({ id: ++counter }); + yield* next(); + }, + *execute([request], next) { + seen.push((yield* State.expect())?.id ?? 0); + if (seen.length === 2) { + both.resolve(); + } + yield* release.operation; + const parent = yield* State.get(); + expect(yield* scoped(() => State.get())).toBe(parent); + yield* next(request); + }, + }); + const first = yield* spawn(() => run()); + const second = yield* spawn(() => run()); + yield* both.operation; + expect([...seen].sort((a, b) => a - b)).toEqual([1, 2]); + expect(yield* State.get()).toBe(undefined); + release.resolve(); + yield* first; + yield* second; + expect(yield* scoped(() => State.get())).toBe(undefined); + }); + + it("EI7: a no-op initializer invents no Files provider; explicit initialization supplies one", function* () { + expect(String(yield* failure(run('')))).toContain("Files"); + const files = recordedFiles({ "note.md": "INSTALLED BY INIT" }); + const output = yield* scoped(function* () { + yield* Execution.around({ + *init(_args, next) { + yield* useRecordedFiles(files); + yield* next(); + }, + }); + return yield* run(''); + }); + expect(output).toContain("INSTALLED BY INIT"); + expect(files.performed).toEqual(["read note.md"]); + expect(String(yield* failure(run('')))).toContain("Files"); + }); +}); diff --git a/packages/core/tests/protected-content.test.ts b/packages/core/tests/protected-content.test.ts index ffed8de1a..1a66c48c1 100644 --- a/packages/core/tests/protected-content.test.ts +++ b/packages/core/tests/protected-content.test.ts @@ -24,6 +24,7 @@ import type { Operation } from "effection"; import { ComponentInvocationError } from "../src/invocation-identity.ts"; import { protectedContentLease } from "../src/protected-content.ts"; +import type { SyntaxSymbols } from "../api.ts"; import type { SyntaxReference } from "../src/syntax-reference.ts"; /** @@ -36,8 +37,15 @@ import type { SyntaxReference } from "../src/syntax-reference.ts"; */ const REFERENCE: SyntaxReference = { // deno-lint-ignore require-yield - *symbols(): Operation { - return ""; + *symbols(): Operation { + return { + version: 2, + categories: [ + { kind: "structural", entries: [] }, + { kind: "built-in", entries: [] }, + { kind: "user-provided", entries: [] }, + ], + }; }, // deno-lint-ignore require-yield *documentation(): Operation { diff --git a/packages/core/tests/syntax-api.test.ts b/packages/core/tests/syntax-api.test.ts new file mode 100644 index 000000000..0c6e4f982 --- /dev/null +++ b/packages/core/tests/syntax-api.test.ts @@ -0,0 +1,310 @@ +import { describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import { type Api, createApi } from "@effectionx/context-api"; +import { call, ensure, scoped, until } from "effection"; +import type { Operation } from "effection"; +import { readTextFile, rm, writeTextFile } from "@effectionx/fs"; +import { mkdtemp } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { InMemoryStream } from "@executablemd/durable-streams"; +import { Syntax } from "../api.ts"; +import type { SyntaxApi, SyntaxSymbols } from "../api.ts"; +import { + Syntax as RootSyntax, + collect, + inlineSource, + renderSyntaxMarkdown, + UnknownComponentError, +} from "../mod.ts"; +import { executeInstalled, syntaxReadEntry, useSyntax } from "../host.ts"; + +function symbolsOf(...names: string[]): SyntaxSymbols { + return { + version: 2, + categories: [ + { kind: "structural", entries: [] }, + { + kind: "built-in", + entries: names.map((name) => ({ + kind: "component", + name, + origin: { kind: "registered", origin: "test://syntax-api", reserved: false }, + sourceKind: "registered", + inspectability: "complete", + forms: ["self-closing"], + props: { type: "object", properties: {}, additionalProperties: false }, + captures: [], + returnMode: "text", + returns: { type: "string" }, + })), + }, + { kind: "user-provided", entries: [] }, + ], + }; +} + +function without(symbols: SyntaxSymbols, name: string): SyntaxSymbols { + return { + ...symbols, + categories: [ + symbols.categories[0], + { + ...symbols.categories[1], + entries: symbols.categories[1].entries.filter((entry) => entry.name !== name), + }, + symbols.categories[2], + ], + }; +} + +function* run(text: string, stream = new InMemoryStream()): Operation { + return String( + yield* collect( + yield* executeInstalled({ ...inlineSource(text), stream, includes: [] }, [ + { evaluation: { read: [syntaxReadEntry()] } }, + ]), + ), + ); +} + +describe("Tier SA — contextual Syntax", () => { + it("SA1: exact public property types, reusable lazy defaults and a suspending transform", function* () { + const descriptor: Api = Syntax; + expect(descriptor).toBe(RootSyntax); + const directory = yield* until(mkdtemp(join(tmpdir(), "syntax-api-"))); + yield* ensure(() => rm(directory, { recursive: true, force: true })); + const path = join(directory, "marker"); + yield* writeTextFile(path, "First"); + let reads = 0; + yield* useSyntax(function* () { + reads += 1; + return symbolsOf(yield* readTextFile(path), "Hidden"); + }); + expect(reads).toBe(0); + yield* Syntax.around({ + *symbols(_args, next) { + return without(yield* next(), "Hidden"); + }, + }); + const first = yield* descriptor.operations.text; + expect(first).toBe(renderSyntaxMarkdown(without(symbolsOf("First", "Hidden"), "Hidden"))); + yield* writeTextFile(path, "Second"); + expect(yield* descriptor.operations.text).toContain("### ``"); + expect(reads).toBe(2); + }); + + it("SA2: symbols affect default text and named availability; text-only middleware changes neither", function* () { + const all = symbolsOf("Kept", "Hidden"); + let source = 0; + let effective = 0; + yield* useSyntax(function* () { + source += 1; + return all; + }); + yield* Syntax.around({ + *symbols(_args, next) { + effective += 1; + return without(yield* next(), "Hidden"); + }, + }); + const text = yield* Syntax.operations.text; + expect(text).toContain("### ``"); + expect(text).not.toContain("### ``"); + source = effective = 0; + const named = yield* Syntax.operations.documentation(["Hidden", "Kept"]); + expect(named).toContain("### ``"); + expect(named).toContain("**Available in this evaluation:** no"); + expect(named).toContain("**Available in this evaluation:** yes"); + expect(source).toBe(1); + expect(effective).toBe(1); + yield* scoped(function* () { + yield* Syntax.around({ + *text() { + return "custom text"; + }, + }); + expect(yield* Syntax.operations.text).toBe("custom text"); + expect(yield* Syntax.operations.symbols).toEqual(without(all, "Hidden")); + expect(yield* Syntax.operations.documentation(["Hidden", "Kept"])).toBe(named); + }); + expect(yield* Syntax.operations.text).toBe(text); + }); + + it("SA3: synthetic answers skip default source work and leaving the child restores its providers", function* () { + let reads = 0; + yield* useSyntax(function* () { + reads += 1; + return symbolsOf("Outer"); + }); + yield* scoped(function* () { + yield* Syntax.around({ + *symbols() { + return symbolsOf("Synthetic"); + }, + *documentation() { + return "synthetic named answer"; + }, + }); + expect(yield* Syntax.operations.text).toContain("Synthetic"); + expect(yield* Syntax.operations.documentation(["Unknown"])).toBe("synthetic named answer"); + expect(reads).toBe(0); + }); + expect(yield* Syntax.operations.text).toContain("Outer"); + expect(reads).toBe(1); + }); + + it("SA4: moving named occurrences resolve one basis each and record the final transformed answer", function* () { + let reads = 0; + let symbols = 0; + yield* useSyntax(function* () { + return symbolsOf(`Marker${++reads}`); + }); + yield* Syntax.around({ + *symbols(_args, next) { + symbols += 1; + return yield* next(); + }, + *documentation([names], next) { + return `${yield* next(names)}\nFINAL NAMED`; + }, + }); + const stream = new InMemoryStream(); + const source = '\n\n'; + const output = yield* run(source, stream); + expect(output).toContain("Marker1"); + expect(output).toContain("Marker2"); + expect(output.match(/FINAL NAMED/g)).toHaveLength(2); + expect(reads).toBe(2); + expect(symbols).toBe(2); + const records = stream + .snapshot() + .filter((event) => event.type === "yield" && event.description.type === "syntax_symbols"); + expect(records).toHaveLength(2); + for (const record of records) { + if (record.type !== "yield" || record.result.status !== "ok") { + throw new Error("no retained Syntax answer"); + } + expect(JSON.stringify(record.result.value)).toContain("FINAL NAMED"); + } + }); + + it("SA5: complete and partial replay restore bare/named final text without live providers", function* () { + const source = '\n\n'; + const stream = new InMemoryStream(); + const first = yield* scoped(function* () { + yield* useSyntax(function* () { + return symbolsOf("HostOnly"); + }); + yield* Syntax.around({ + *text(_args, next) { + return `${yield* next()}\nFINAL BARE`; + }, + *documentation([names], next) { + return `${yield* next(names)}\nFINAL NAMED`; + }, + }); + return yield* run(source, stream); + }); + expect(first).toContain("FINAL BARE"); + expect(first).toContain("FINAL NAMED"); + const partial = new InMemoryStream(stream.snapshot().filter((event) => event.type !== "close")); + yield* useSyntax(function* (): Operation { + throw new Error("live source reached on replay"); + }); + yield* Syntax.around({ + *symbols(): Operation { + throw new Error("live symbols reached on replay"); + }, + *text(): Operation { + throw new Error("live text reached on replay"); + }, + *documentation(): Operation { + throw new Error("live catalog reached on replay"); + }, + }); + expect(yield* run(source, stream)).toBe(first); + expect(yield* run(source, partial)).toBe(first); + }); + + it("SA6: generated descriptions retain host-only metadata and never grant execution", function* () { + let reads = 0; + yield* useSyntax(function* () { + reads += 1; + return symbolsOf("HostOnly"); + }); + const output = yield* run( + "'} />\n\n", + ); + expect(output).toContain("### ``"); + expect(output).toContain("**Available in this evaluation:** no"); + expect(reads).toBe(2); + yield* Syntax.around({ + *symbols() { + return symbolsOf("HostOnly"); + }, + }); + const refused = yield* run("'} />\n"); + expect(refused).toContain("a component this host did not admit"); + expect(refused).not.toContain("No long-form documentation"); + }); + + it("SA7: a descriptive filter cannot disable an admitted generated component", function* () { + yield* Syntax.around({ + *symbols() { + return symbolsOf(); + }, + }); + yield* Syntax.around({ + *text() { + return "ADMITTED SYNTAX RAN"; + }, + }); + expect(yield* run("'} />\n")).toContain("ADMITTED SYNTAX RAN"); + }); + + it("SA8: canonical unknown names are safe records; provider-thrown UnknownComponentError is terminal", function* () { + const stream = new InMemoryStream(); + yield* useSyntax(function* () { + return symbolsOf("HostOnly"); + }); + yield* run('\n', stream); + const record = stream + .snapshot() + .find((event) => event.type === "yield" && event.description.type === "syntax_symbols"); + expect(record?.type === "yield" ? record.result.status : "absent").toBe("ok"); + expect(JSON.stringify(record)).toContain('"refused"'); + yield* scoped(function* () { + yield* useSyntax(function* (): Operation { + throw new UnknownComponentError("provider failure"); + }); + const failed = new InMemoryStream(); + yield* run('\n', failed); + const retained = failed + .snapshot() + .find((event) => event.type === "yield" && event.description.type === "syntax_symbols"); + expect(retained?.type === "yield" ? retained.result.status : "absent").toBe("err"); + expect(JSON.stringify(retained)).toContain("provider failure"); + }); + }); + + it("SA9: independently constructed same-name descriptors share ordinary middleware", function* () { + const other: Api = createApi("executablemd.core.syntax", { + symbols: call(function* (): Operation { + throw new Error("foreign default must not run"); + }), + text: call(function* (): Operation { + throw new Error("foreign default must not run"); + }), + *documentation() { + throw new Error("foreign default must not run"); + }, + }); + yield* other.around({ + *text() { + return "OTHER DESCRIPTOR"; + }, + }); + expect(yield* run("\n")).toContain("OTHER DESCRIPTOR"); + }); +}); diff --git a/packages/core/tests/syntax-component.test.ts b/packages/core/tests/syntax-component.test.ts index d4ef56982..80b9f7c7c 100644 --- a/packages/core/tests/syntax-component.test.ts +++ b/packages/core/tests/syntax-component.test.ts @@ -65,6 +65,9 @@ import { retainedSource } from "../src/root-source.ts"; import { renderSyntaxMarkdown } from "../src/syntax-markdown.ts"; import { syntaxReference, rootSyntaxReference } from "../src/syntax-reference.ts"; import type { SyntaxReference } from "../src/syntax-reference.ts"; +import { useRootSyntaxReference } from "../src/syntax-api.ts"; +import { useSyntax } from "../host.ts"; +import { Syntax } from "../api.ts"; import { capturedDocumentation, contributeDocumentation, @@ -123,9 +126,11 @@ function symbolsOf(...names: readonly string[]): SyntaxSymbols { function stating(symbols: SyntaxSymbols, calls: { count: number } = { count: 0 }) { const installation: ExecutionInstallation = { // deno-lint-ignore require-yield - *symbols(): Operation { - calls.count += 1; - return symbols; + *install(): Operation { + yield* useSyntax(function* (): Operation { + calls.count += 1; + return symbols; + }); }, }; return { installation, calls }; @@ -321,7 +326,6 @@ function symbolsNamed(name: string, origin: NamedOrigin): SyntaxSymbols { function* rootObservation(): Operation { return rootSyntaxReference( { includes: [], registry: new Map(), components: [], declarations: [] }, - undefined, // What canonical execution hands it: whatever the scope bootstrapped, // terminating in core's own. Passing nothing here would leave the reference // with no index at all, and a case about narrowing would then be reading a @@ -330,6 +334,16 @@ function* rootObservation(): Operation { ); } +function observedDocumentation( + reference: SyntaxReference, + names: readonly string[], +): Operation { + return scoped(function* () { + yield* useRootSyntaxReference(reference); + return yield* Syntax.operations.documentation(names); + }); +} + /** A working directory of this case's own, torn down on the way out. */ function useWorkingDirectory(body: (dir: string) => Operation): Operation { return scoped(function* () { @@ -601,11 +615,10 @@ describe("Tier SYN — the named form", () => { // And the snapshot renders that way through the reference an execution // builds from it, rather than only reading that way as a value. - const rendered = yield* syntaxReference( - symbolsOf("Marker"), - symbolsOf("Marker"), - captured, - ).documentation(["Marker"]); + const rendered = yield* observedDocumentation( + syntaxReference(symbolsOf("Marker"), symbolsOf("Marker"), captured), + ["Marker"], + ); expect(rendered).toContain("THE CAPTURED PROSE."); expect(rendered).not.toContain("SUBSTITUTED AFTER COLLECTION"); }); @@ -1314,9 +1327,11 @@ describe("Tier SYN — the named form", () => { const calls = { count: 0 }; const moving: ExecutionInstallation = { // deno-lint-ignore require-yield - *symbols(): Operation { - calls.count += 1; - return symbolsOf(`Marker${calls.count}`); + *install(): Operation { + yield* useSyntax(function* (): Operation { + calls.count += 1; + return symbolsOf(`Marker${calls.count}`); + }); }, }; @@ -1942,8 +1957,10 @@ describe("Tier SYN — the record one occurrence keeps", () => { // contribution refuses to answer at all. const moved: ExecutionInstallation = { // deno-lint-ignore require-yield - *symbols(): Operation { - throw new Error("the continuation rebuilt the symbols"); + *install(): Operation { + yield* useSyntax(function* (): Operation { + throw new Error("the continuation rebuilt the symbols"); + }); }, }; const continued = String(yield* run("\n", [moved], yield* continuing(first))); @@ -1982,12 +1999,14 @@ describe("Tier SYN — the record one occurrence keeps", () => { const teardown: string[] = []; const stream = new InMemoryStream(); const hanging: ExecutionInstallation = { - *symbols(): Operation { - yield* ensure(function* () { - teardown.push("released"); + *install(): Operation { + yield* useSyntax(function* (): Operation { + yield* ensure(function* () { + teardown.push("released"); + }); + yield* suspend(); + throw new Error("unreachable"); }); - yield* suspend(); - throw new Error("unreachable"); }, }; @@ -2085,10 +2104,10 @@ describe("Tier SYN — reference never permits", () => { it("SYN25b: a narrowed reference answers with exactly the symbols it was given", function* () { const narrowed = symbolsOf("Admitted"); const reference = syntaxReference(narrowed); - expect(yield* reference.symbols()).toBe(renderSyntaxMarkdown(narrowed)); + expect(renderSyntaxMarkdown(yield* reference.symbols())).toBe(renderSyntaxMarkdown(narrowed)); // Nothing of the enclosing site leaks into it: a name the wider profile has // is absent, because the symbols it was handed do not hold one. - expect(yield* reference.symbols()).not.toContain("### ``"); + expect(renderSyntaxMarkdown(yield* reference.symbols())).not.toContain("### ``"); }); /** @@ -2105,26 +2124,26 @@ describe("Tier SYN — reference never permits", () => { // What may execute here is the narrowed set, and the bare form reports // exactly that. - const available = yield* reference.symbols(); + const available = renderSyntaxMarkdown(yield* reference.symbols()); expect(available).toContain("### ``"); expect(available).not.toContain("### ``"); // Reference material comes from the enclosing symbols, so a component this // subtree may not run can still be explained — and the entry says so // rather than leaving a reader to assume they have both. - const documented = yield* reference.documentation(["Withheld"]); + const documented = yield* observedDocumentation(reference, ["Withheld"]); expect(documented).toContain("### ``"); expect(documented).toContain("**Available in this evaluation:** no"); // And one that is admitted reports the other answer, so the field is // discriminating rather than a constant. - const admitted = yield* reference.documentation(["Admitted"]); + const admitted = yield* observedDocumentation(reference, ["Admitted"]); expect(admitted).toContain("**Available in this evaluation:** yes"); // A boundary that narrows nothing has one set, and everything in it is // available — the ordinary case. const open = syntaxReference(enclosing); - expect(yield* open.documentation(["Withheld"])).toContain( + expect(yield* observedDocumentation(open, ["Withheld"])).toContain( "**Available in this evaluation:** yes", ); }); @@ -2192,7 +2211,7 @@ describe("Tier SYN — reference never permits", () => { for (const [what, origin] of Object.entries(impostors)) { const nested = syntaxReference(holding(origin), holding(authored)); - const rendered = yield* nested.documentation(["Elicit"]); + const rendered = yield* observedDocumentation(nested, ["Elicit"]); expect([what, rendered.includes("**Available in this evaluation:** no")]).toEqual([ what, true, @@ -2209,12 +2228,14 @@ describe("Tier SYN — reference never permits", () => { }; const moved: NamedOrigin = { ...bundled, sourceHash: "c".repeat(40) }; expect( - yield* syntaxReference(holding(moved), holding(bundled)).documentation(["Elicit"]), + yield* observedDocumentation(syntaxReference(holding(moved), holding(bundled)), ["Elicit"]), ).toContain("**Available in this evaluation:** no"); // The positive control: one exact identity, admitted. expect( - yield* syntaxReference(holding(authored), holding(authored)).documentation(["Elicit"]), + yield* observedDocumentation(syntaxReference(holding(authored), holding(authored)), [ + "Elicit", + ]), ).toContain("**Available in this evaluation:** yes"); }); @@ -2229,20 +2250,20 @@ describe("Tier SYN — reference never permits", () => { const narrowed = enclosing.available(admitted); // What may run is the admission. - const executable = yield* narrowed.symbols(); + const executable = renderSyntaxMarkdown(yield* narrowed.symbols()); expect(executable).toContain("### ``"); expect(executable).not.toContain("### ``"); // What may be read about is still the enclosing site's, with the enclosing // index behind it — so a real component's real documentation survives. - const documented = yield* narrowed.documentation(["Elicit"]); + const documented = yield* observedDocumentation(narrowed, ["Elicit"]); expect(documented).toContain("### ``"); expect(documented).toContain("Asks a person a structured question"); expect(documented).toContain("**Available in this evaluation:** no"); // And the enclosing reference is unchanged by having been narrowed. - expect(yield* enclosing.symbols()).toContain("### ``"); - expect(yield* enclosing.documentation(["Elicit"])).toContain( + expect(renderSyntaxMarkdown(yield* enclosing.symbols())).toContain("### ``"); + expect(yield* observedDocumentation(enclosing, ["Elicit"])).toContain( "**Available in this evaluation:** yes", ); }); @@ -2271,7 +2292,7 @@ describe("Tier SYN — reference never permits", () => { supplies.add("Beta"); supplies.delete("Alpha"); - const rendered = yield* reference.documentation(["Alpha"]); + const rendered = yield* observedDocumentation(reference, ["Alpha"]); expect(rendered).toContain("The captured documentation."); expect(rendered).not.toContain("SUBSTITUTED AFTER CAPTURE"); }); diff --git a/specs/executable-mdx-spec.md b/specs/executable-mdx-spec.md index ec23c5450..09dd1bb9a 100644 --- a/specs/executable-mdx-spec.md +++ b/specs/executable-mdx-spec.md @@ -2783,7 +2783,11 @@ the bare form continues to list structural syntax alongside components. The documentation is read from the owning package through the direct filesystem, not through `API.Fs` or the document-facing `Files` provider. Both of those are middleware a running document can compose around, and a document that could -answer the read would decide what the product says about itself. +answer the read would decide what the product says about itself. What that read +produces is the *captured default* catalog, and canonical collection owns it. +The effective answer an occurrence receives is a separate thing: live `Syntax` +middleware composes in front of that catalog and may transform or replace what +it produced. **Documentation composes with the components it describes.** A package's bootstrap installs its registrations and its documentation together — the @@ -2852,12 +2856,16 @@ host-maintained list of every package's registrations, is two lists that drift; this is one. **Reference and availability are separate.** The reference carries two inputs. -Bare `` reports what may **execute** at this site. The named form -selects from the **enclosing authoring symbols** and states, per entry, -`**Available in this evaluation:** yes` or `no`. At a root the two are one set -of symbols and everything selected is available; a trusted evaluation boundary that -narrows execution keeps the enclosing reference, so a nested author can be told -how a component works where they may not run one — and is told which it is. +Default bare `` reports the effective contextual symbols at this +site. The named form selects from the **enclosing authoring symbols** and +states, per entry, +`**Available in this evaluation:** yes` or `no`. Uncustomized at a root the two +are one set of symbols and everything selected is available; a trusted +evaluation boundary that narrows execution keeps the enclosing reference, so a +nested author can be told how a component works where they may not run one — +and is told which it is. Descriptive middleware that filters the effective +symbols separates the two at a root as well, and the per-entry label is what +says so. Neither input carries definitions, import witnesses, invocation capabilities, providers, registrations or any other execution capability. @@ -2885,14 +2893,41 @@ the import, delegate it and refuse it by throwing, but cannot answer it, replace what came back, mutate it, or hand back a definition kept from another import or built by another loaded copy. -**The symbols are the execution's own.** Canonical core builds them at the root from -the selection inputs that execution captured before any installation, middleware -or document code ran — its includes, the registry it started with, the identity -components and exact Markdown its host declared, and the component bundle it is -closed over — and carries them lexically on canonical core's own expansion -environment. A trusted host may state the symbols its profile describes instead, -captured on the same terms; one execution accepts one, and two are refused rather -than ordered. Nothing is built until an occurrence asks. +**Live descriptions use the contextual Syntax API.** Core exports this API and +its types from `@executablemd/core/api`, alongside the same Execution descriptor +exported from the package root: + +```ts +interface SyntaxApi { + readonly symbols: Operation; + readonly text: Operation; + documentation(names: readonly string[]): Operation; +} +``` + +Callers yield `Syntax.operations.symbols` and `.text` without parentheses; +property middleware and its continuation return Operations. A host binds a +complete descriptive profile of its own with +`useSyntax(source: () => Operation): Operation` from +`@executablemd/core/host`, which installs ordinary symbols and documentation +providers without reading the source. `ExecutionInstallation` carries no symbols +provider and no descriptive provider count rule: ordinary middleware +composition decides the live answer. How the defaults, the execution's own +fallback and a narrowing boundary rank against each other is +[the syntax symbols boundary](../architecture.md#the-syntax-symbols-boundary). + +Default text renders the effective contextual symbols, and text middleware may +transform or replace that answer. Default named lookup retains its enclosing +catalog for metadata and long-form text and reads effective symbols once for +availability, so filtering symbols affects bare text and named availability +while an omitted enclosing component remains documentable; text-only +customization changes neither symbols nor named lookup, and a synthetic answer +that does not delegate skips the default work entirely. Generated producer and +body scopes narrow symbols only, retaining the enclosing named-documentation +owner, and their descriptive bindings end with those scopes. Standalone Plan +and CLI inspection bind the target run's declarative profile through +`useSyntax`; embedded Plan inherits its enclosing profile. CLI inspection +starts no execution initialization or operational provider merely to print. Seeing a component in the symbols grants nothing. It neither registers, resolves nor authorizes that component: what a name means is still this section's @@ -9862,6 +9897,33 @@ workflow and returns a `DocumentExecution` handle. Options: - `secretDetection?` — detect credentials before durable events persist (default: enabled) +#### Execution initialization + +`Execution.init(): Operation` runs once in the existing invocation-owned +scope. Its default is no-op and supplies no Files provider. A host registers +ordinary middleware, installs its live dependencies there and delegates with +`yield* next()`. Where that sits in the order core already guarantees — after +the trusted host contributions are copied and their revocation armed, before +installation callbacks, documentation capture and public `Execution.execute` +policy — is +[capability-backed execution](../architecture.md#capability-backed-execution). +The generated profile's provider-bound definitions are resolved during document +assembly. + +Failure or cancellation during init prevents execution policy, root import and +body and waits for all acquired cleanup before settlement. Concurrent +executions have independent initialized state; ordinary descendants inherit it. +A later sibling inherits no completed execution's state. Observing a completed +handle starts no new init. A fresh continuation establishes fresh live state +while completed durable operations and Syntax answers replay passively. + +`runDocument`, REPL `runEntry` and isolated ordinary `runProfileChild` all reach +this core initialization point. Isolated children explicitly reinstall selected +host/Plugin middleware after cwd assembly and inherit no arbitrary parent +handlers. REPL state is per entry below the existing Agent attachment, which +init does not replace, and so are durable preparation and the journal owner. +The REPL expansion controller does not pause at initialization. + #### The host's repository provider A document execution reaches repository operations only through the provider @@ -12176,6 +12238,31 @@ composition underneath it. | FX5–FX5d | Directory ownership | A body that fails and a body cancelled inside the region both leave the created directory, and the enclosing directory is read back **in the scope that installed it** rather than in a fresh run that would install its own; a refused ensure expands no content and writes nothing; an existing directory keeps what is in it | | ORC3 | The shipped command | A real `xmd run` performs the whole journey with candidates for all three names beside the document, ignores them, and restores the working directory for the sibling after the region | + +### Tier EI — Execution initialization + +| ID | Assertion | +| --- | --- | +| EI1 | Root and `/api` export the same descriptor; init defaults to no-op, dispatches once per fresh execution, and does not rerun for completed handle observation. | +| EI2 | Init-installed state is visible to assembly, documentation capture and execute/document policy in that order. | +| EI3 | Init cannot replace already captured admissions, declarations or generated profile tables. | +| EI4–EI5 | Initializer failure and deterministic cancellation reach no policy/root and settle only after held cleanup finishes. | +| EI6–EI7 | Concurrent state is independent and inherited by ordinary descendants; sibling state restores; no-op init invents no Files provider while explicit init supplies one. | +| EI-REPL | Two entries beneath the same session attachment own distinct initialized state and completed observation does not reinitialize. | +| EI-child | An isolated run child reinstalls chosen Plugin initialization after cwd assembly and does not inherit unrelated parent middleware. | + +### Tier SA — Contextual Syntax + +| ID | Assertion | +| --- | --- | +| SA1 | Exact public Operation-valued properties use reusable lazy defaults and a suspending source/transform. | +| SA2–SA3 | Symbols affect default text and named availability, text-only middleware changes neither structured symbols nor named lookup, synthetic answers skip default work, and scoped providers restore. | +| SA4 | Moving named occurrences each resolve one authoring basis and one effective symbols answer and retain the final transformed text. | +| SA5 | Complete and partial replay restore final bare/named text while every live symbols, text, catalog and source provider in scope is one that throws if consulted. | +| SA6–SA7 | Generated scopes retain host-only metadata while reporting availability; adding descriptions admits nothing and hiding an admitted component disables nothing. | +| SA8 | Canonical unknown names retain safe refusal values while provider-thrown UnknownComponentError remains an operation error. | +| SA9 | Independently constructed same-name API descriptors compose ordinary middleware. | + ### Tier SYN — The public `` component Named `SYN` rather than `SY` or `SL`, which already name the syntax symbols and diff --git a/specs/plan-command-spec.md b/specs/plan-command-spec.md index a1af74e09..b1ebc1f71 100644 --- a/specs/plan-command-spec.md +++ b/specs/plan-command-spec.md @@ -342,9 +342,11 @@ The host supplies two fixed internal inputs as that root's props: They are the adapter's own, and nothing a Plan declares is bound here: the properties a Plan's root declares are resolved by whoever runs it. The symbols -are not among them either. The command states the vocabulary its profile -describes at the execution boundary, captured before any installed code runs, -and the packaged Component reaches them by writing the public `` any +are not among them either. The command enters the target run's declarative +bootstraps and binds a lazy source through core's `useSyntax` helper before +execution. The source inspects captured target registry/declaration inputs, and +setup snapshots the same assembly's documentation contributions. The packaged +Component reaches the contextual answer by writing the public `` any document may write — so an authored phase can say that preparation is starting before the read happens, and the symbols the Agent is shown are the ones an operator can print.