From d42287abe0241e85732617fe29f79b21d31c6b56 Mon Sep 17 00:00:00 2001 From: Taras Mankovski Date: Wed, 14 Oct 2026 08:28:22 -0400 Subject: [PATCH 1/2] =?UTF-8?q?=E2=9C=A8=20Add=20execution=20initializatio?= =?UTF-8?q?n=20and=20contextual=20Syntax?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Ordinary setup gets one execution-owned entry point, and live `` descriptions become ordinary composed state. `Execution.init(): Operation` joins the existing owner. Canonical core invokes it in `invoke()` after the trusted host contributions are copied and their revocation armed, and before the installation callbacks, documentation capture and public `execute` policy — so a host installs its live dependencies once per execution, in the scope the invocation already owns, and nothing it installs can reach back and change what was already captured. The public default is no-op rather than a refusal: an execution with no host middleware has no live dependencies to establish and invents none. `Syntax` is a contextual Api with `Operation`-valued `symbols` and `text` properties and an Operation-valued `documentation(names)` method, exported from `/api` and the root. Its defaults are reusable lazy operations. The reference an execution derives is bound as the Api's *fallback*, not installed as middleware, so a host that stated its own profile with the new `useSyntax(source)` helper on `/host` still answers; a canonical evaluation boundary narrows symbols only, and keeps the enclosing named-documentation owner. `ExecutionInstallation.symbols` and its one-provider count rule are gone, and `xmd plan`, `xmd syntax` and the Plan fixtures bind their descriptions through the helper instead. A description now registers, resolves and authorizes nothing: it may name a component this host will refuse, or hide one it will run, and preflight decides either way. The protected component keeps its authenticated invocation, site and form checks, safe named refusal, publication and secret gate, and records the final middleware answer inside its existing durable live callback — completed replay restores that text without consulting any live provider. Intentionally unchanged: U4's metadata/identity work, U5's generated-selection and input removal, durable preparation's position after retained-history admission, Plan's structural validation and trusted reads, REPL's Agent attachment and journal lifetimes, and every other installation member. --- architecture.md | 112 ++++--- packages/cli/src/cli.ts | 14 +- packages/cli/src/plan-writer-profile.ts | 167 +++++----- packages/cli/src/plan.ts | 26 +- packages/cli/src/repl/expansion.ts | 12 +- packages/cli/src/syntax.ts | 62 +++- packages/cli/tests/plan-cli.test.ts | 24 +- .../cli/tests/plan-command-document.test.ts | 5 +- packages/cli/tests/plan-component.test.ts | 3 +- packages/cli/tests/repl-execution.test.ts | 42 ++- packages/cli/tests/support/plan-harness.ts | 16 +- .../cli/tests/testing-execution-host.test.ts | 49 ++- packages/core/api.ts | 27 ++ packages/core/host.ts | 21 +- packages/core/mod.ts | 10 +- packages/core/src/components/Syntax.ts | 54 +-- packages/core/src/documentation-index.ts | 3 - packages/core/src/execute.ts | 128 ++++---- packages/core/src/expand.ts | 21 ++ packages/core/src/generated-xmd.ts | 7 + packages/core/src/syntax-api.ts | 123 +++++++ packages/core/src/syntax-markdown.ts | 13 +- packages/core/src/syntax-reference.ts | 154 +++++---- packages/core/src/use-syntax.ts | 42 +++ .../core/tests/evaluate-component.test.ts | 11 +- packages/core/tests/execution-init.test.ts | 258 +++++++++++++++ packages/core/tests/protected-content.test.ts | 12 +- packages/core/tests/syntax-api.test.ts | 310 ++++++++++++++++++ packages/core/tests/syntax-component.test.ts | 87 +++-- specs/executable-mdx-spec.md | 109 +++++- specs/plan-command-spec.md | 8 +- 31 files changed, 1538 insertions(+), 392 deletions(-) create mode 100644 packages/core/src/syntax-api.ts create mode 100644 packages/core/src/use-syntax.ts create mode 100644 packages/core/tests/execution-init.test.ts create mode 100644 packages/core/tests/syntax-api.test.ts diff --git a/architecture.md b/architecture.md index 734a1f10e..b795663d0 100644 --- a/architecture.md +++ b/architecture.md @@ -3746,6 +3746,22 @@ by registration order. They may refuse; they may not complete. ### Capability-backed execution +`Execution.init(): Operation` establishes live dependencies once in each +existing execution-owned scope. Its public default is no-op; middleware installs +ordinary package behavior and delegates. Core invokes it after copying trusted +host contributions and arming their revocation, before provider/installation +callbacks, documentation capture and public execution policy. Setup receives no +installation array, request, entry identifier or receipt and creates no second +execution owner. + +Initialization failure prevents policy, root import and body. Failure and +cancellation join partially acquired teardown before the invocation settles. +Ordinary descendants inherit initialized state; concurrent executions and REPL +entries own distinct state. Completed handle observation initializes nothing. +An isolated run child reinstalls the selected host/Plugin middleware after cwd +assembly; arbitrary parent handlers do not cross that isolation. Existing REPL +Agent attachment and durable preparation keep their owners and ordering. + Canonical core is authoritative for document execution. It invokes the stable `Execution` middleware through a private, per-invocation same-name Api whose instance-owned default handler is the authoritative terminal. A stable name @@ -5067,11 +5083,11 @@ cannot repair a missing or multiply selected domain. Paired producers retain their lexical site route and operational permission. A trusted provider may therefore delegate and register canonical self-closing -`` as a component answer. Its bare form reports the admitted vocabulary; -its named form renders the enclosing authoring documentation with narrowed -availability. Verified protected documentation provenance survives sealing even -when the delegating provider has a different origin. That provenance is -descriptive data and never a callable lookup key. +`` as a component answer. Its default bare form reports the admitted +vocabulary; its named form renders the enclosing authoring documentation with +narrowed availability. Verified protected documentation provenance survives +sealing even when the delegating provider has a different origin. That +provenance is descriptive data and never a callable lookup key. Routes, projection edges, bodies, domains, callable sets and site operations belong to one live execution. Child closure and execution teardown invalidate @@ -5233,40 +5249,58 @@ declared origin and digest as its origin. Its private closure contributes nothing: those names are not syntax a document may write, so a set of symbols that listed them would describe an environment that does not exist. -**An execution carries one of its own, lexically.** The symbols a running -document is shown travel by value on canonical core's private expansion -environment, beside component resolution and the identity domains — not through a -Context, because a context resolves by name and a name is not a secret, so a -document could build one and answer for the vocabulary it is shown. - -Canonical core builds it at the execution root from the selection inputs that -execution captured before any installation, middleware or document code ran: the -includes it resolves against, the registry it started with, the identity -components and exact Markdown its host declared, and the component bundle it is -closed over when it has one. A bundled name is described from the pinned bytes -already in hand — nothing is imported, executed, or read from a file to describe -one — and reported at the canonical repository-relative path that blob has in the -commit, which is where a reader of a workflow run looks for it. - -A trusted host may state the symbols its profile describes instead, on the terms -every other trusted-host value travels: captured by value with the rest of the -installation, before any installed code exists. One execution accepts one, and -two are refused rather than ordered, because ordering them would make which -profile a document is shown depend on assembly order. `xmd plan` states one — a -Plan is a program a later `xmd run` executes, so the vocabulary the agent must be -shown is that profile's rather than the authorship execution's, which searches no -repository and refuses almost every capability. An ordinary run states none and -describes itself. - -Nothing is built until an occurrence asks, so a run whose document never writes -`` enumerates no includes, parses no component and reads no -frontmatter. Each ask builds afresh, which is what makes an occurrence's retained -symbols its own rather than a copy of whichever one ran first. - -A trusted canonical evaluation boundary may install a narrower reference for -the subtree it evaluates, and leaving that subtree restores the enclosing one. It -adds nothing: the symbols it installs are the ones its own admission already -selected, so an entry absent from that admission is absent from the reference. +**Live descriptions compose through Syntax.** Core exports `Syntax` and +`SyntaxApi` from `@executablemd/core/api`. Its `symbols` and `text` properties +are `Operation` and `Operation`; callers yield the +properties without parentheses. `documentation(names)` is an Operation-valued +method. Defaults are reusable lazy Operations, so each live read starts fresh +work. Default text renders the effective contextual symbols through the shared +Markdown renderer. Middleware may answer, delegate or transform these views. + +The execution-derived private reference supplies the fallback. It is built from +the selection inputs that execution captured before any installation, +middleware or document code ran: the includes it resolves against, the registry +it started with, the identity components and exact Markdown its host declared, +and the component bundle it is closed over when it has one. A bundled name is +described from the pinned bytes already in hand — nothing is imported, executed, +or read from a file to describe one — and reported at the canonical +repository-relative path that blob has in the commit, which is where a reader of +a workflow run looks for it. It is bound as the Api's fallback rather than as +inner middleware, so it does not shadow an enclosing host profile. + +`useSyntax(source)` from `@executablemd/core/host` binds a host's lazy complete +descriptive profile and snapshots the documentation contributions already in its +calling scope. It installs ordinary symbols and documentation +providers at `min` without reading the source. There is no installation-member +count rule for descriptive providers. + +Standalone Plan enters the ordinary run profile's declarative bootstraps and +installs this helper before its constrained authorship execution. Its source +inspects captured target registry/declaration inputs lazily, so named metadata +also describes the program it is preparing. An embedded Plan keeps its enclosing +profile. CLI inspection uses the same bounded declarative assembly and helper, +without running execution initialization or operational providers. + +Named documentation retains the enclosing catalog for metadata and long-form +text and reads effective contextual symbols once for availability. Each +occurrence resolves its authoring basis once; its default symbols provider +reuses that basis rather than inspecting a moving root twice. Later occurrences +resolve independently. A symbols filter can hide an entry from the bare view +while named documentation still explains it as unavailable. Text-only +middleware changes neither structured symbols nor named lookup. + +Generated producer and body scopes narrow symbols through ordinary middleware +while keeping the enclosing named-documentation owner. Leaving those scopes +restores the enclosing view. Synthetic answers that do not delegate perform no +default inspection. Descriptions register, resolve and authorize nothing, so a +custom view may describe a forbidden name without admitting it, or hide an +admitted name without disabling it. + +The protected Syntax component records the final middleware answer inside its +existing durable live callback. Completed replay restores that final text +without consulting any live symbols, text or catalog provider. Its authenticated +invocation, safe canonical named refusal, publication and secret gate remain +separate from descriptive composition. **Inspection is observation, never execution.** Producing symbols installs only the declarative registration layer selection needs. It does not enter diff --git a/packages/cli/src/cli.ts b/packages/cli/src/cli.ts index 0cea3443a..8b90a99de 100755 --- a/packages/cli/src/cli.ts +++ b/packages/cli/src/cli.ts @@ -142,8 +142,9 @@ import { componentSearchPath, resolveTestTarget } from "./test-target.ts"; import { renderSyntaxDocumentation, renderSyntaxJson, - renderSyntaxMarkdown, + syntaxText, syntaxSymbols, + useCommandSyntax, } from "./syntax.ts"; import { assembleRunProfile } from "./run-profile.ts"; import type { PluginModuleLoader } from "./plugin-loader.ts"; @@ -2858,7 +2859,11 @@ function* dispatch( { ...(sessions === undefined ? {} : { sessions }), plugins, - symbols: (includes) => syntaxSymbols(includes, plugins), + // What the Plan being written will be run under. The same declarative + // assembly `xmd syntax` prints from, entered inside the authorship + // scope so the documentation this host's packages contribute is + // captured with it. + installSyntax: (includes) => useCommandSyntax(includes, plugins), // The two facts about this process's own stderr that nothing further // in may go and read: whether it is a terminal, and whether it took // what it was handed. The approved Plan's sinks are stdout and @@ -3083,10 +3088,9 @@ function* dispatch( // The compact list of symbols, unchanged: routine discovery output and // every default Plan prompt read it, and long documentation would make // both unnecessarily large. - const catalog = yield* syntaxSymbols(command.config.include, plugins); rendered = command.config.json - ? renderSyntaxJson(catalog) - : renderSyntaxMarkdown(catalog); + ? renderSyntaxJson(yield* syntaxSymbols(command.config.include, plugins)) + : yield* syntaxText(command.config.include, plugins); } else { // The same selection, index and renderer `` uses, so // the command and the component cannot describe one component two diff --git a/packages/cli/src/plan-writer-profile.ts b/packages/cli/src/plan-writer-profile.ts index d8f2b8b27..de5648f81 100644 --- a/packages/cli/src/plan-writer-profile.ts +++ b/packages/cli/src/plan-writer-profile.ts @@ -45,7 +45,7 @@ import { useTerminalOutput, } from "@executablemd/core"; import type { AgentProviderOptions, Json } from "@executablemd/core"; -import type { SyntaxSymbolsProvider, MarkdownComponent } from "@executablemd/core/host"; +import type { MarkdownComponent } from "@executablemd/core/host"; import { executeInstalled, installInvocationAgentProvider } from "@executablemd/core/host"; import { createAcpxProvider } from "@executablemd/acp"; import type { AcpxProviderDependencies } from "@executablemd/acp"; @@ -170,15 +170,16 @@ export interface PlanWriterProfile { */ declaration: MarkdownComponent; /** - * The symbols this authorship describes. + * Install the description this authorship shows. * - * The `run` profile's, because a Plan is a program a later `xmd run` executes: - * deriving them from this execution — which searches no repository and - * refuses almost every capability — would describe a vocabulary the approved - * program would not have. Captured with the rest of the installation, before - * any installed code, middleware or document code runs. + * Invoked inside the bounded scope below, before the execution exists, which + * is the only place a target-run profile can be both bootstrapped and + * captured: the helper core exports snapshots the documentation contributions + * that are in scope when it is called, so a host that bootstrapped elsewhere + * and handed over a bare symbols callback would lose them, and named metadata + * would fall back to this restricted authorship execution's own catalog. */ - symbols: SyntaxSymbolsProvider; + installSyntax(): Operation; } /** What building the constrained provider needs, and nothing more. */ @@ -428,84 +429,88 @@ export function* runPlanCommandDocument(profile: PlanWriterProfile): Operation` the Component writes reads the profile the Plan + // is being written for. Nothing further in may state another. + yield* profile.installSyntax(); + const source = yield* readPackagedDocument(PLAN_COMMAND_DOCUMENT); + try { + // The presentation every phase this command renders passes through. A raw + // capture would show an operator whitespace nobody wrote, and terminal + // formatting belongs to a terminal — which is the host's fact about its + // own stream, not something this module may go and detect. + yield* useNormalizedOutput(); + if (profile.progress.terminal) { + yield* useTerminalOutput(); + } - // The command root is a thin adapter: it projects the request into - // `` and returns what comes back. Everything that used to be - // installed around it — the directory, the provider, the Elicitation, the - // refusals — is installed by the Component itself, inside the invocation that - // owns it, so the command and an ordinary document run one workflow rather - // than two arrangements of one. - const execution = yield* executeInstalled( - { - ...retainedSource(PLAN_COMMAND_IDENTITY, source), - stream: profile.stream, - // No repository component search. What the adapter may name is - // what this command declares, so a file in the caller's tree - // cannot answer for `` or anything else. - includes: [], - // The one secret boundary this command has. Every live durable event - // crosses the serialized pre-append gate, so a draft or a diagnostic - // becomes readable — to the journal and to the phase that presents it - // — only after the event supplying it has cleared and committed. - secretDetection: true, - props: { - request: profile.request, - session: profile.session, - }, - }, - [ + // The command root is a thin adapter: it projects the request into + // `` and returns what comes back. Everything that used to be + // installed around it — the directory, the provider, the Elicitation, the + // refusals — is installed by the Component itself, inside the invocation that + // owns it, so the command and an ordinary document run one workflow rather + // than two arrangements of one. + const execution = yield* executeInstalled( { - components: agentIdentityComponents(), - declarations: [profile.declaration], - // What a Plan may write is the `run` profile's vocabulary, not this - // authorship execution's — the adapter searches no repository, the - // Component installs a ceiling that refuses almost everything, and - // symbols derived from *that* would describe a program nobody can - // run. Captured before any installation runs, so no prop, binding, - // middleware answer or projected content reaches it. - symbols: profile.symbols, - // The ordinary run's evaluation ceiling, so an information request - // this authorship evaluates reads through exactly what `xmd run` - // states — File, Glob and canonical Syntax, with the write table - // present and never selected. The host installs it, not ``: - // a component that installed its own would be choosing its - // ceiling instead of being given one. - evaluation: ordinaryEvaluationProfile(), + ...retainedSource(PLAN_COMMAND_IDENTITY, source), + stream: profile.stream, + // No repository component search. What the adapter may name is + // what this command declares, so a file in the caller's tree + // cannot answer for `` or anything else. + includes: [], + // The one secret boundary this command has. Every live durable event + // crosses the serialized pre-append gate, so a draft or a diagnostic + // becomes readable — to the journal and to the phase that presents it + // — only after the event supplying it has cleared and committed. + secretDetection: true, + props: { + request: profile.request, + session: profile.session, + }, }, - ], - ); - - // Drains while the document is still producing. A destination that fails - // raises here, and leaving this scope cancels the execution and waits for - // its teardown before anything is reported. - yield* forEach(function* (chunk: string) { - const written = yield* profile.progress.write(chunk); - if (!written.ok) { - throw new ProgressDeliveryError(written.error); + [ + { + components: agentIdentityComponents(), + declarations: [profile.declaration], + // The ordinary run's evaluation ceiling, so an information request + // this authorship evaluates reads through exactly what `xmd run` + // states — File, Glob and canonical Syntax, with the write table + // present and never selected. The host installs it, not ``: + // a component that installed its own would be choosing its + // ceiling instead of being given one. + evaluation: ordinaryEvaluationProfile(), + }, + ], + ); + + // Drains while the document is still producing. A destination that fails + // raises here, and leaving this scope cancels the execution and waits for + // its teardown before anything is reported. + yield* forEach(function* (chunk: string) { + const written = yield* profile.progress.write(chunk); + if (!written.ok) { + throw new ProgressDeliveryError(written.error); + } + }, execution.output); + + const completed = yield* execution; + if (!completed.ok) { + return completed; } - }, execution.output); - - const completed = yield* execution; - if (!completed.ok) { - return completed; - } - if (typeof completed.value !== "string") { - return Err(new Error("the plan command document returned something that is not a Plan")); + if (typeof completed.value !== "string") { + return Err(new Error("the plan command document returned something that is not a Plan")); + } + return Ok(completed.value); + } catch (error) { + return Err(error instanceof Error ? error : new Error(String(error))); } - return Ok(completed.value); - } catch (error) { - return Err(error instanceof Error ? error : new Error(String(error))); - } + }); }); } diff --git a/packages/cli/src/plan.ts b/packages/cli/src/plan.ts index 4c0d82aa1..8659b8112 100644 --- a/packages/cli/src/plan.ts +++ b/packages/cli/src/plan.ts @@ -40,7 +40,6 @@ import { randomUUID } from "node:crypto"; import { resolve } from "node:path"; import process from "node:process"; -import type { SyntaxSymbols } from "@executablemd/core"; import type { AcpxProviderDependencies } from "@executablemd/acp"; import { InMemoryStream } from "@executablemd/durable-streams"; import type { DurableStream } from "@executablemd/durable-streams"; @@ -89,8 +88,21 @@ export interface PlanDependencies { sessions?: MachineSessionAssembly; /** What the profile's provider is built on, beyond the host's own assembly. */ acp?: AcpxProviderDependencies; - /** The run profile's complete structured vocabulary. */ - symbols(includes: readonly string[]): Operation; + /** + * Install the description this authorship shows, in the scope it is given. + * + * A setup operation rather than a symbols callback, because binding the + * profile and building it are two different moments: the host enters the + * target run's declarative bootstraps, so the documentation its packages + * contribute is in scope when `useSyntax()` snapshots it, and only the lazy + * source it binds there runs later, when an occurrence renders. + * + * Production installs the `run` profile's: a Plan is a program a later + * `xmd run` executes, so the vocabulary the agent must be shown is that + * profile's rather than the authorship execution's, which searches no + * repository and refuses almost every capability. + */ + installSyntax(includes: readonly string[]): Operation; /** Who answers the review question. */ installElicitation(): Operation; /** @@ -232,13 +244,7 @@ export function* runPlan(command: PlanCommand, deps: PlanDependencies): Operatio declaration, stream, progress: deps.progress, - // The vocabulary this authorship describes is the `run` profile's, not - // this execution's: a Plan is a program a later `xmd run` executes, so the - // symbols the Agent must be shown are the ones that run will have. Stated - // at the execution boundary and captured before any installed code — no - // prop on the thin adapter, and nothing the Component projects, could - // supply another. - symbols: () => deps.symbols(command.include), + installSyntax: () => deps.installSyntax(command.include), }); } catch (error) { console.error(describeError(error)); diff --git a/packages/cli/src/repl/expansion.ts b/packages/cli/src/repl/expansion.ts index 5725f7f30..c44d4944a 100644 --- a/packages/cli/src/repl/expansion.ts +++ b/packages/cli/src/repl/expansion.ts @@ -112,15 +112,17 @@ export const COMPONENT_BOUNDARIES: BoundaryPartition = { * exits until the run is over, which makes it the right anchor for *which* * expansion is being paused and a useless place to rest. * - * `execute` is not an expansion boundary. It is the request-policy seam, asked - * once before a document exists, and canonical core dispatches it through a - * private instance — so holding it would delay building the run rather than - * stopping its expansion, and would not reliably be reached at all. + * Neither `init` nor `execute` is an expansion boundary. `execute` is the + * request-policy seam, asked once before a document exists, and canonical core + * dispatches it through a private instance — so holding it would delay building + * the run rather than stopping its expansion, and would not reliably be reached + * at all. `init` is earlier still: it establishes the run's live dependencies + * before there is any expansion to step. */ export const EXECUTION_BOUNDARIES: BoundaryPartition = { walks: ["document"], gates: [], - unheld: ["execute"], + unheld: ["execute", "init"], }; /** diff --git a/packages/cli/src/syntax.ts b/packages/cli/src/syntax.ts index 1a860891b..45de9fdc4 100644 --- a/packages/cli/src/syntax.ts +++ b/packages/cli/src/syntax.ts @@ -29,15 +29,13 @@ import { planComponentDescription } from "./plan-component.ts"; import { scoped } from "effection"; +import { Component, Syntax } from "@executablemd/core/api"; +import { useSyntax } from "@executablemd/core/host"; import type { Operation } from "effection"; import { agentIdentityComponents, - capturedDocumentation, - documentationIndexFor, inspectSyntax, - renderSelectedDocumentation, renderSyntaxMarkdown, - selectDocumented, useAgentComponents, } from "@executablemd/core"; import type { SyntaxSymbols } from "@executablemd/core"; @@ -74,24 +72,58 @@ export function* syntaxSymbols( plugins: CommandPlugins = NO_PLUGINS, ): Operation { return yield* scoped(function* () { - yield* useCommandComponents(); - return yield* profileSymbols(includes, plugins); + yield* useCommandSyntax(includes, plugins); + return yield* Syntax.operations.symbols; }); } -/** The profile's symbols, inside a scope that has already bootstrapped it. */ -function* profileSymbols( +/** + * Describe this scope with the command's profile. + * + * The one descriptive setup every `xmd syntax` projection and `xmd plan` + * authorship goes through, so an operator printing a profile and an agent being + * told to write against it are shown one construction. + * + * Capture now, inspect later. The bootstraps are entered and their registry and + * declarations are read here, into one object the source closes over, because + * those are the *target* inputs: what this command installed, before anything + * else registers. Inspection itself is deferred to the source, so a command + * that never renders a description enumerates no includes and parses no + * component. Calling the helper in this same scope is also what captures the + * documentation those bootstraps contributed. + */ +export function* useCommandSyntax( includes: readonly string[], - plugins: CommandPlugins, -): Operation { - return yield* inspectSyntax({ - includes, + plugins: CommandPlugins = NO_PLUGINS, +): Operation { + yield* useCommandComponents(); + const inputs = { + includes: [...includes], + registry: new Map(yield* Component.operations.registry), components: agentIdentityComponents(), // `` is part of the run profile, so symbols that left it out would // describe a vocabulary no run has. Described from the packaged bytes: // inspection mints nothing, so it reports the Component's identity and // contract without building the capabilities only a run can build. declarations: commandDeclarations(plugins, yield* planComponentDescription()), + }; + yield* useSyntax(() => inspectSyntax(inputs)); +} + +/** + * The profile's symbols as Markdown, through the contextual construction. + * + * Read from the Api rather than rendered from the value beside it, so this + * command prints what a document writing `` under the same profile + * would be shown — including whatever middleware the profile installed. + */ +export function syntaxText( + includes: readonly string[], + plugins: CommandPlugins = NO_PLUGINS, +): Operation { + return scoped(function* () { + yield* useCommandSyntax(includes, plugins); + return yield* Syntax.operations.text; }); } @@ -185,9 +217,7 @@ export function* renderSyntaxDocumentation( plugins: CommandPlugins = NO_PLUGINS, ): Operation { return yield* scoped(function* () { - yield* useCommandComponents(); - const catalog = yield* profileSymbols(includes, plugins); - const index = documentationIndexFor(yield* capturedDocumentation()); - return renderSelectedDocumentation(selectDocumented(catalog, catalog, names, index)); + yield* useCommandSyntax(includes, plugins); + return yield* Syntax.operations.documentation(names); }); } diff --git a/packages/cli/tests/plan-cli.test.ts b/packages/cli/tests/plan-cli.test.ts index ad759b24f..705baa31d 100644 --- a/packages/cli/tests/plan-cli.test.ts +++ b/packages/cli/tests/plan-cli.test.ts @@ -36,6 +36,7 @@ import { fileURLToPath } from "node:url"; import process from "node:process"; import * as cliModule from "../src/cli.ts"; +import { syntaxSymbols } from "../src/syntax.ts"; import { runPlan } from "../src/plan.ts"; import type { PlanCommand } from "../src/plan.ts"; import type { PlanWriterStack } from "../src/agent-stack.ts"; @@ -1805,19 +1806,22 @@ describe( it("PO15: the catalog is built once, from inside the command document", function* () { yield* useWorkingDirectory(function* (dir, planWriterRoot) { - const harness = createPlanHarness({ planWriterRoot }); - harness.fake.script({ reply: PLAIN }); - harness.script({ decision: "Approve" }); - // Recorded against the progress the drain had at the time, so what this // observes is that Preparing reached the operator before the catalog - // was read rather than merely that both happened. + // was read rather than merely that both happened. Observed at the lazy + // source rather than at the setup that binds it: binding happens when + // the authorship scope opens, and what this case is about is when the + // description is *built*. const before: string[][] = []; - const catalog = harness.deps.symbols; - harness.deps.symbols = function* (includes) { - before.push(phasesOf(harness.progress.join(""))); - return yield* catalog(includes); - }; + const harness = createPlanHarness({ + planWriterRoot, + *symbols(includes) { + before.push(phasesOf(harness.progress.join(""))); + return yield* syntaxSymbols(includes); + }, + }); + harness.fake.script({ reply: PLAIN }); + harness.script({ decision: "Approve" }); const { value } = yield* delivered(harness, () => runPlan(planning(dir), harness.deps)); diff --git a/packages/cli/tests/plan-command-document.test.ts b/packages/cli/tests/plan-command-document.test.ts index a084d27eb..0f182934a 100644 --- a/packages/cli/tests/plan-command-document.test.ts +++ b/packages/cli/tests/plan-command-document.test.ts @@ -53,6 +53,7 @@ import type { } from "@executablemd/core/host"; import { ordinaryEvaluationProfile } from "../src/evaluation-profile.ts"; import { elicitWriteEntry } from "@executablemd/core/host"; +import { useSyntax } from "@executablemd/core/host"; import { Elicitation } from "@executablemd/core"; import { recordedFiles, useRecordedFiles } from "../../core/tests/support/fragment-files.ts"; import type { RecordedFiles } from "../../core/tests/support/fragment-files.ts"; @@ -229,6 +230,7 @@ function* runDocument(options: RunOptions = {}): Operation { // would show an operator whitespace nobody wrote. yield* useNormalizedOutput(); try { + yield* useSyntax(harness.symbols); const execution = yield* executeInstalled( { ...retainedSource(PLAN_COMMAND_IDENTITY, source), @@ -244,7 +246,6 @@ function* runDocument(options: RunOptions = {}): Operation { { components: agentIdentityComponents(), declarations: [harness.declaration], - symbols: harness.symbols, // The ceiling the command installs, so an information request this // document evaluates reads through exactly what `xmd run` states. // A case that supplies its own recorder gets that instead, which is @@ -569,6 +570,7 @@ describe("the packaged plan command document", () => { yield* scoped(function* () { yield* installAgentComponents({ defaultAgent: AGENT, permissionMode: "deny-all" }); yield* useNormalizedOutput(); + yield* useSyntax(harness.symbols); const execution = yield* executeInstalled( { ...retainedSource(PLAN_COMMAND_IDENTITY, source), @@ -581,7 +583,6 @@ describe("the packaged plan command document", () => { { components: agentIdentityComponents(), declarations: [installed], - symbols: harness.symbols, }, ], ); diff --git a/packages/cli/tests/plan-component.test.ts b/packages/cli/tests/plan-component.test.ts index 545ba54eb..fcc9a1daa 100644 --- a/packages/cli/tests/plan-component.test.ts +++ b/packages/cli/tests/plan-component.test.ts @@ -40,6 +40,7 @@ import { syntaxReadEntry, } from "@executablemd/core/host"; import type { FragmentEvaluationInput } from "@executablemd/core/host"; +import { useSyntax } from "@executablemd/core/host"; import { API, Permissions } from "@executablemd/runtime"; import { recordedFiles, useRecordedFiles } from "../../core/tests/support/fragment-files.ts"; import type { RecordedFiles } from "../../core/tests/support/fragment-files.ts"; @@ -192,6 +193,7 @@ function* runDocument(options: { yield* useNormalizedOutput(); } try { + yield* useSyntax(harness.symbols); const execution = yield* executeInstalled( { ...retainedSource(ROOT, options.source), @@ -206,7 +208,6 @@ function* runDocument(options: { // Where the profile a document observes is settled now: the // `` the packaged Plan writes is canonical core's public // component, and what it answers with is this execution's. - symbols: harness.symbols, ...(options.evaluation === undefined ? {} : { evaluation: options.evaluation }), ...(options.files === undefined && options.installing === undefined ? {} diff --git a/packages/cli/tests/repl-execution.test.ts b/packages/cli/tests/repl-execution.test.ts index 7fd51da3b..fe9a1369e 100644 --- a/packages/cli/tests/repl-execution.test.ts +++ b/packages/cli/tests/repl-execution.test.ts @@ -18,7 +18,7 @@ import { expect } from "@executablemd/test-support/expect"; import { useTempFileCompiler } from "@executablemd/core"; import { InMemoryStream, serializeDurableEvent } from "@executablemd/durable-streams"; import type { DurableEvent } from "@executablemd/durable-streams"; -import { ensure, race, scoped, sleep, spawn, until } from "effection"; +import { createContext, ensure, race, scoped, sleep, spawn, until } from "effection"; import type { Operation, Result } from "effection"; import { ensureDir, rm, writeTextFile } from "@effectionx/fs"; import { API } from "@executablemd/runtime"; @@ -919,3 +919,43 @@ describe("REPL execution: one execution's file", () => { expect(yield* opened.stream.readAll()).toEqual([]); }); }); + +describe("execution initialization in REPL entries", () => { + it("EI-REPL: two entries own fresh state below the same session attachment", function* () { + const State = createContext("executablemd.test.repl.init", undefined); + const initialized: object[] = []; + const observed: object[] = []; + yield* Execution.around({ + *init(_args, next) { + const state = {}; + initialized.push(state); + yield* State.set(state); + yield* next(); + }, + *document([request], next) { + const state = yield* State.expect(); + if (state === undefined) { + throw new Error("entry has no initialized state"); + } + observed.push(state); + yield* next(request); + }, + }); + const session = opened( + yield* submitReplEntry({ ...options(execution()), source: "first entry\n" }), + ); + yield* session.join(); + const submitted = yield* session.submit("second entry\n"); + if (!submitted.ok) { + throw submitted.error; + } + yield* session.join(); + expect(initialized).toHaveLength(2); + expect(observed).toEqual(initialized); + expect(initialized[0]).not.toBe(initialized[1]); + expect(yield* State.get()).toBe(undefined); + expect(session.model.entries).toHaveLength(2); + yield* session.join(); + expect(initialized).toHaveLength(2); + }); +}); diff --git a/packages/cli/tests/support/plan-harness.ts b/packages/cli/tests/support/plan-harness.ts index 943e5320c..5725ce662 100644 --- a/packages/cli/tests/support/plan-harness.ts +++ b/packages/cli/tests/support/plan-harness.ts @@ -29,7 +29,8 @@ import { planComponentDeclaration } from "../../src/plan-component.ts"; import type { PlanSurface, StructuralValidation } from "../../src/plan-component.ts"; import { planAgentContext } from "../../src/plan-writer-profile.ts"; import type { PlanWriterStack } from "../../src/agent-stack.ts"; -import type { SyntaxSymbolsProvider, MarkdownComponent } from "@executablemd/core/host"; +import { useSyntax } from "@executablemd/core/host"; +import type { MarkdownComponent } from "@executablemd/core/host"; import type { PlanDependencies } from "../../src/plan.ts"; import { createFakeAcp, makeRegistry, makeStore } from "./fake-acp.ts"; import type { FakeAcp, FakeStore } from "./fake-acp.ts"; @@ -184,9 +185,14 @@ export function createPlanHarness(options: { sessionStore: options.store ?? makeStore(), agentRegistry: makeRegistry({ [AGENT]: `${AGENT}-cmd` }), }, - *symbols(includes) { - symbolCalls.push([...includes]); - return yield* (options.symbols ?? syntaxSymbols)(includes); + *installSyntax(includes) { + // The harness's own lazy source, bound through the helper production + // uses — so what a case replaces is the description, not the way it + // reaches the execution. + yield* useSyntax(function* () { + symbolCalls.push([...includes]); + return yield* (options.symbols ?? syntaxSymbols)(includes); + }); }, planWriterRoot: options.planWriterRoot, *installElicitation() { @@ -287,7 +293,7 @@ export interface PlanDeclarationHarness { * public, canonical core owns it, and what it answers with is the execution's * own. */ - symbols: SyntaxSymbolsProvider; + symbols: () => Operation; /** How many times that provider was asked. */ symbolCalls: number; /** Review answers, taken in order. Running out is a test defect, not a case. */ diff --git a/packages/cli/tests/testing-execution-host.test.ts b/packages/cli/tests/testing-execution-host.test.ts index c07b20c12..43e909ae7 100644 --- a/packages/cli/tests/testing-execution-host.test.ts +++ b/packages/cli/tests/testing-execution-host.test.ts @@ -19,7 +19,7 @@ import { expect } from "@executablemd/test-support/expect"; import { Err, Ok, ensure, resource, scoped, spawn, withResolvers } from "effection"; import type { Operation } from "effection"; import { ensureDir, readdir, rm, writeTextFile } from "@effectionx/fs"; -import { mkdtemp } from "node:fs/promises"; +import { mkdtemp, realpath } from "node:fs/promises"; import { until } from "effection"; import { join } from "node:path"; import { tmpdir } from "node:os"; @@ -1097,3 +1097,50 @@ describe("deterministic dependencies declared for a nested run", () => { expect(result.code).toBe(0); }); }); + +describe("execution initialization in isolated run children", () => { + it("EI-child: chosen Plugin setup is reinstalled after cwd and parent middleware stays outside", function* () { + const project = yield* useProject({}); + const api = new URL("../../core/api.ts", import.meta.url).href; + const runtime = new URL("../../runtime/mod.ts", import.meta.url).href; + yield* writeTextFile( + join(project, "init-plugin.ts"), + [ + `import { Execution, Plugin, Syntax } from ${JSON.stringify(api)};`, + `import { cwd } from ${JSON.stringify(runtime)};`, + "const calls: string[] = [];", + "export default Plugin({ name: 'init-evidence', *install(request) {", + " yield* Execution.around({ *init(_args, next) {", + " calls.push(`${request.command}@${yield* cwd()}`);", + " yield* Syntax.around({ *text() { return calls.join(','); } }, { at: 'min' });", + " yield* next();", + " } });", + " return undefined;", + "} });", + ].join("\n"), + ); + const canonical = yield* until(realpath(project)); + const expected = `test@${canonical},run@${canonical}`; + yield* writeTextFile( + join(project, "README.md"), + doc( + '', + '\\n"} as="child">', + '', + '', + "", + ``, + "", + "", + ), + ); + const result = yield* runCli(["test", "--plugin", "./init-plugin.ts", "README.md"], { + cwd: project, + }).join(); + if (result.code !== 0) { + throw new Error(result.stdout + result.stderr); + } + expect(result.code).toBe(0); + expect(result.stdout + result.stderr).not.toContain("❌"); + }); +}); diff --git a/packages/core/api.ts b/packages/core/api.ts index fee42b6e1..6e679974a 100644 --- a/packages/core/api.ts +++ b/packages/core/api.ts @@ -128,3 +128,30 @@ export type { ComponentExpansionRequest, } from "./src/component-api.ts"; export type { Expansion } from "./src/expansion.ts"; + +/** + * The execution a host composes around, and the requests its layers are handed. + * + * `init` is the member a command host installs on: it establishes the live + * dependencies this execution runs on, once, inside the scope the invocation + * already owns. `execute` and `document` stay what they were — the + * request-policy seam and the whole-document seam — and their public defaults + * still refuse, because only canonical core completes a document. + */ +export { Execution } from "./src/execute.ts"; +export type { ExecutionApi } from "./src/execute.ts"; +export type { ExecutionRequest } from "./src/execution-request.ts"; +export type { DocumentRequest } from "./src/document-request.ts"; + +/** + * The description in effect, as the thing a consumer composes on. + * + * Here rather than only at the root because answering *what may be written + * here* is a consumer boundary: a host binds its own profile, and middleware + * may transform the symbols, the text or a named lookup. A description grants + * nothing — `./host`'s `useSyntax` is how a host supplies one, and admission + * remains the execution's. + */ +export { Syntax } from "./src/syntax-api.ts"; +export type { SyntaxApi } from "./src/syntax-api.ts"; +export type { SyntaxSymbols } from "./src/inspect.ts"; diff --git a/packages/core/host.ts b/packages/core/host.ts index f9d3e8ac4..422701678 100644 --- a/packages/core/host.ts +++ b/packages/core/host.ts @@ -189,12 +189,6 @@ export { generatedRequestRefusal } from "./src/generated-request-refusal.ts"; */ export { classifyPlanResponse } from "./src/plan-response.ts"; export type { PlanResponseKind } from "./src/plan-response.ts"; -/** - * The symbols a host's profile describes, when they are not the ones the - * execution would derive from its own captured inputs — see - * `src/syntax-reference.ts`. - */ -export type { SyntaxSymbolsProvider } from "./src/syntax-reference.ts"; export type { DurablePreparation } from "./src/document-request.ts"; /** @@ -407,3 +401,18 @@ export type { AnswerConfiguration, AnswerMatcher } from "./src/answers.ts"; */ export { DeclarationScan } from "./src/declaration-scan.ts"; export type { AnswersPlacement, DeclarationScanner } from "./src/declaration-scan.ts"; + +/** + * Where a trusted host states the description its profile shows. + * + * A setup operation rather than a value on the installation, because what it + * binds is two things: the lazy source, and the documentation contributions + * that are in scope where it is called. `xmd plan` needs both — a Plan is a + * program a later `xmd run` executes, so the vocabulary the agent must be shown + * is that profile's, and so is the long-form prose a named lookup renders. + * + * It reads nothing when called and installs ordinary providers, so nearer + * middleware still answers, delegates or transforms. Stating a description + * confers no execution authority. + */ +export { useSyntax } from "./src/use-syntax.ts"; diff --git a/packages/core/mod.ts b/packages/core/mod.ts index be32a01a2..ebfa353a4 100644 --- a/packages/core/mod.ts +++ b/packages/core/mod.ts @@ -256,7 +256,8 @@ export { } from "./src/documentation-api.ts"; export type { DocumentationApi } from "./src/documentation-api.ts"; export { select as selectDocumented } from "./src/syntax-reference.ts"; -export { NO_DOCUMENTATION, UnknownComponentError } from "./src/documentation-index.ts"; +export { UnknownComponentError } from "./src/documentation-index.ts"; +export { NO_DOCUMENTATION } from "./src/syntax-markdown.ts"; export type { DocumentationIndex } from "./src/documentation-index.ts"; export { PROTECTED_COMPONENT_NAMES, ProtectedComponentError } from "./src/components/protected.ts"; export { SYNTAX_COMPONENT } from "./src/components/Syntax.ts"; @@ -450,3 +451,10 @@ export type { FileSecretFinding } from "./src/secrets/files.ts"; // here can disable, replace, or outlive the detection an execution runs under. export { scanSecrets, secretPolicy } from "./src/secrets/policy.ts"; export type { SecretPolicy } from "./src/secrets/policy.ts"; + +// What a document may write here. The same descriptor `./api` exports — one +// stable name, so middleware composes across separately loaded copies — kept at +// the root as well because reading a description is ordinary work, not a +// consumer-boundary concern. +export { Syntax } from "./src/syntax-api.ts"; +export type { SyntaxApi } from "./src/syntax-api.ts"; diff --git a/packages/core/src/components/Syntax.ts b/packages/core/src/components/Syntax.ts index 6982b8e55..6b4e47c07 100644 --- a/packages/core/src/components/Syntax.ts +++ b/packages/core/src/components/Syntax.ts @@ -3,39 +3,41 @@ * * An author asking "which components do I have here?" and an agent being told * what to write are the same question, and `xmd syntax` already answers it from - * outside. This is the same answer from inside: the symbols for the site the - * element was written at, as the Markdown that command prints. + * outside. This is the same answer from inside: the description in effect at + * the site the element was written at, as the Markdown that command prints. * * ## Why canonical core owns it * - * The symbols describe the vocabulary an execution actually has. A repository - * `Syntax.md`, a bundled `Syntax`, a registration, an import handler or a - * second loaded copy answering for the name would each describe a vocabulary - * the run does not have — to whoever is reading, and to whichever agent is - * being told what to write next. So the name is reserved by the canonical - * protected tier, ahead of every host and author tier, and the definition - * canonical core selected is what runs. + * The description is what an agent is told to write next, so whoever answers + * for it decides what gets written. A repository `Syntax.md`, a bundled + * `Syntax`, a registration, an import handler or a second loaded copy answering + * for the name would each do that from outside the composition the run + * assembled. So the name is reserved by the canonical protected tier, ahead of + * every host and author tier, and the definition canonical core selected is + * what runs. * - * Protection is about the *answer*, not about power. The component receives one - * reference that renders symbol text and nothing else: no definitions, no - * import witness, no invocation capability, no policy table, no provider and no - * registration handle. Naming a component in the symbols is not permission to - * run it. + * Protection is about *who answers*, not about power. The description itself is + * ordinary composed state: a host binds its own profile, a trusted boundary + * narrows it, and middleware may transform the text — none of which registers, + * resolves or admits anything. Naming a component in a description is not + * permission to run it, and hiding one does not disable it. * * ## What one occurrence does * - * It authenticates the occurrence identity this execution minted, renders - * once, and retains exactly what it rendered. A continuation reads that record - * and hands the same text back without consulting the filesystem, the - * registry, the bundle, the host or the lexical reference again — so an agent - * resuming authorship is shown the vocabulary the run actually showed it, not - * one rebuilt from a tree that has moved since. + * It authenticates the occurrence identity this execution minted, asks the + * contextual `Syntax` Api inside its own durable live callback, and retains + * exactly the final answer that came back. A continuation reads that record and + * hands the same text back without consulting the filesystem, the registry, the + * bundle, the host, or any live symbols, text or catalog provider again — so an + * agent resuming authorship is shown the vocabulary the run actually showed it, + * not one rebuilt from a tree, or a composition, that has moved since. */ import { createDurableOperation, StaleInputError } from "@executablemd/durable-streams"; import type { Json as DurableJson, Workflow } from "@executablemd/durable-streams"; import type { Operation } from "effection"; +import { Syntax } from "../syntax-api.ts"; import { getExpansion } from "../expansion.ts"; import { markGeneratedRequestRefusal } from "../generated-request-refusal.ts"; import { SyntaxSelectionRefusal } from "../syntax-refusal.ts"; @@ -132,7 +134,6 @@ function syntax(claim: IdentityClaimant): ProtectedBody { invocation: ComponentInvocation, site: ProtectedSite, ): Operation { - const reference = site.syntax; // Read off the issuance the engine holds rather than off a method the // caller could have written, and answered before anything is claimed or // rendered: a paired spelling is a document asking for something this @@ -155,7 +156,12 @@ function syntax(claim: IdentityClaimant): ProtectedBody { // that somebody validated them. const names = requestedNames(props.names); const id = yield* claim(invocation); - if (reference === undefined) { + // Site validation, kept where the element is: an expansion driven outside + // an execution established no description at all, so there is nothing for + // the contextual Api to answer from. Answering that here rather than + // leaving it to the Api's own default is what makes the refusal name the + // element a reader is looking at. + if (site.syntax === undefined) { throw new Error(NO_REFERENCE_REFUSAL); } const expansion = yield* getExpansion(); @@ -169,8 +175,8 @@ function syntax(claim: IdentityClaimant): ProtectedBody { return { symbols: names === undefined - ? yield* reference.symbols() - : yield* reference.documentation(names), + ? yield* Syntax.operations.text + : yield* Syntax.operations.documentation(names), }; } catch (error) { if (error instanceof SyntaxSelectionRefusal) { diff --git a/packages/core/src/documentation-index.ts b/packages/core/src/documentation-index.ts index 63a1cd07a..819bf7020 100644 --- a/packages/core/src/documentation-index.ts +++ b/packages/core/src/documentation-index.ts @@ -113,9 +113,6 @@ export interface DocumentationIndex { bundleDocumentation(origin: ComponentOrigin): string | undefined; } -/** What a component with no authored documentation renders instead of prose. */ -export const NO_DOCUMENTATION = "No long-form documentation is available for this component."; - /** A level-two heading, captured without its marker. */ const HEADING = /^##\s+(.+?)\s*$/; /** Any ATX heading, so a deeper one can be told from a section boundary. */ diff --git a/packages/core/src/execute.ts b/packages/core/src/execute.ts index 978be5411..3ef92ce62 100644 --- a/packages/core/src/execute.ts +++ b/packages/core/src/execute.ts @@ -158,6 +158,7 @@ import { import { coreComponentDefinition, CORE_ORIGIN } from "./components/registry.ts"; import { CORE_REVISION } from "./generated-xmd.ts"; import { rootSyntaxReference } from "./syntax-reference.ts"; +import { useRootSyntaxReference } from "./syntax-api.ts"; import { capturedDocumentation } from "./documentation-api.ts"; import { CANONICAL_PROFILE_ANSWERS, @@ -176,7 +177,6 @@ import { componentAnswerRegistrar } from "./component-answers.ts"; import type { ComponentAnswerInstallation } from "./component-answers.ts"; import { packagedAssetReader } from "./component-documentation.ts"; import type { DocumentationContribution, DocumentationReader } from "./component-documentation.ts"; -import type { SyntaxSymbolsProvider } from "./syntax-reference.ts"; import type { WorkflowComponentBundle, WorkflowComponentCatalog } from "./components/bundle.ts"; import type { CodeBlockContext, CodeBlockResult, EvalEnv } from "./types.ts"; import { readRootSource, rootSourcePath } from "./root-source.ts"; @@ -2417,13 +2417,12 @@ function* executeDocument( bundles: readonly WorkflowComponentBundle[] = [], identityComponents: readonly IdentityComponent[] = [], installed: readonly RetainedInstallation[] = [], - providers: readonly SyntaxSymbolsProvider[] = [], /** * The documentation each bootstrapped package contributed. * - * Carried by value from the collection boundary, like the symbols provider - * beside it, so the index a document's own `` reads is the - * index the profile actually assembled. + * Carried by value from the collection boundary, so the index a document's + * own `` reads is the index the profile actually + * assembled. */ documentation: readonly DocumentationContribution[] = [], /** This execution's packaged-asset reader, carried by value from the caller. */ @@ -2442,8 +2441,8 @@ function* executeDocument( * The one evaluation profile this host stated, copied and bound, when it * stated one. * - * Carried by value like the symbols provider beside it. It is sealed below, - * once the provider-backed names it admits have been resolved, and the sealed + * Carried by value from the collection boundary. It is sealed below, once + * the provider-backed names it admits have been resolved, and the sealed * result is handed to core's own expansion on the private environment rather * than through any context: it is the ceiling `` narrows from, and a * document that could reach it could raise it. @@ -2702,6 +2701,25 @@ function* executeDocument( return imports === undefined ? definition : imports.issue(name, definition); } + // Built from what this execution captured before any installation, + // middleware or document code ran, and asked only when an occurrence + // renders: a run whose document never writes `` enumerates + // nothing. + const derivedSyntax = rootSyntaxReference( + { + includes, + registry: startingRegistry, + components: identityComponents, + declarations: installed.flatMap((one) => [...one.declarations]), + ...(bundle === undefined ? {} : { workflow: bundle }), + }, + documentation, + ); + // Bound as the contextual Api's fallback, not installed in front of + // anything: a host that stated its own profile with `useSyntax()` still + // answers, and this is what answers when none did. + yield* useRootSyntaxReference(derivedSyntax); + const environment: ExecutionEnvironment = { componentResolution: imports, // Everything this host declared, admitted above. Expansion asks it @@ -2716,21 +2734,7 @@ function* executeDocument( // document, a component, middleware or a separately loaded copy can // name reaches this object. sourceSegments: createExactSource(), - // Built from what this execution captured before any installation, - // middleware or document code ran, and asked only when an occurrence - // renders: a run whose document never writes `` enumerates - // nothing. - syntax: rootSyntaxReference( - { - includes, - registry: startingRegistry, - components: identityComponents, - declarations: installed.flatMap((one) => [...one.declarations]), - ...(bundle === undefined ? {} : { workflow: bundle }), - }, - providers[0], - documentation, - ), + syntax: derivedSyntax, // The ceiling a generated fragment is evaluated under, when this host // offers evaluation at all. Absent is a host that offers none, and // `` refuses on that rather than inventing one. @@ -3008,24 +3012,6 @@ export interface ExecutionInstallation { * operation after its invocation. */ readonly components?: readonly IdentityComponent[]; - /** - * The symbols this host's profile describes, when its profile is not the one - * the execution itself would derive. - * - * Omitted is the ordinary case — canonical core derives the symbols from the - * selection inputs this execution captured, which is what makes an ordinary - * run's reference the run's own. - * - * `xmd plan` states one, because the Plan being written is a program a later - * `xmd run` executes: the vocabulary the agent must be shown is that profile's - * rather than the authorship execution's. One execution accepts one. - * - * Documentation does not travel here. It is additive — a profile installing - * four packages has four boundaries — so each package's bootstrap contributes - * its own through the `Documentation` Api, and this execution collects them - * once at the same boundary. - */ - readonly symbols?: SyntaxSymbolsProvider; /** * The maximum permission a generated fragment may be evaluated under here. * @@ -3062,6 +3048,27 @@ export interface ExecutionInstallation { * it returns is ignored: only canonical execution completes a document. */ export interface ExecutionApi { + /** + * Establish the live dependencies this execution runs on. + * + * The seam a command host uses to install what it provides — a Files + * provider, a recorder, a service — once per execution, inside the scope the + * invocation already owns. So a resource acquired here belongs to this + * execution, ordinary descendants inherit it, two concurrent executions get + * their own, and teardown is joined before the invocation settles. + * + * It receives nothing. There is no request to brand, no installation array + * to read and no receipt to return, because initialization is not a decision + * about this run: it is the run's dependencies being put in place. A layer + * establishes its state and delegates with `yield* next()`. + * + * Invoked after the trusted host contributions are copied and their + * revocation armed, and before installation callbacks, documentation capture + * and `execute`'s policy — so what a host installs here is visible to all + * three, and nothing a host installs here can reach back and change what was + * already captured. + */ + init(): Operation; execute(request: ExecutionRequest): Operation; /** * The document's expansion, as `durableRun` runs it. @@ -3082,6 +3089,11 @@ export interface ExecutionApi { * handed to it. Canonical core dispatches through a private instance instead. */ export const Execution: Api = createApi("Execution", { + // No-op rather than a refusal, unlike the two below. Initialization is not a + // protocol a caller can complete on core's behalf — there is nothing to + // settle and nothing to record — so an execution with no host middleware + // simply has no live dependencies to establish, and invents none. + *init() {}, // deno-lint-ignore require-yield *execute(_request: ExecutionRequest): Operation { throw new ExecutionProtocolError("invoked execution outside canonical core"); @@ -3098,10 +3110,11 @@ export const Execution: Api = createApi("Execution", * * The order is the contract. Admissions are copied and frozen first, so what * ends up authoritative is fixed before any installation, any middleware and - * any document code exists. Installations then run, then the chain is invoked - * with one opaque request, then the request must have reached the terminal - * exactly once, and only then does canonical core execute the document with the - * options the terminal recorded. + * any document code exists. Initialization then establishes the live + * dependencies, installations run, and the chain is invoked with one opaque + * request; then the request must have reached the terminal exactly once, and + * only then does canonical core execute the document with the options the + * terminal recorded. * * One structured owner task holds the invocation scope. Everything an * installation established, and every child the document spawned, lives inside @@ -3528,23 +3541,6 @@ function* invoke( }), ); - // Read once and frozen with the rest, and before any installation runs: which - // profile a document is shown is settled before anything can replace it. Two - // are refused rather than ordered — symbols chosen by installation order - // would make what an agent is told to write depend on assembly order. - const providers = Object.freeze( - installations.flatMap((installation) => { - const provider = installation.symbols; - return provider === undefined ? [] : [provider]; - }), - ); - if (providers.length > 1) { - throw new Error( - "two installations stated the symbols this execution describes. One execution describes " + - "one vocabulary, so which profile a document is shown is never a question of order.", - ); - } - // What each installation will actually be asked to do, read once and in // order, before the first of them runs. Both members are read here for the // reason everything else on the installation is: a property that answered @@ -3604,6 +3600,15 @@ function* invoke( }); canonicalImports.activate(); + // The host's live dependencies, established here: after everything trusted + // has been copied and its revocation armed, so initialization cannot reach + // back and change what is already authoritative, and before the installation + // callbacks and documentation capture below, so what it establishes is there + // for all of them. In this scope, which the owner task holds — so what a + // layer acquires is torn down with this execution and joined before it + // settles, and a failure here reaches no policy, no root import and no body. + yield* Execution.operations.init(); + // Each provider installer first, then that installation's ordinary `install`, // in the order the installations were captured. A provider registers // middleware through its registrar, and each invocation states what it is @@ -3659,7 +3664,6 @@ function* invoke( bundles, identityComponents, installed, - providers, documentation, readAsset, canonicalImports, diff --git a/packages/core/src/expand.ts b/packages/core/src/expand.ts index 648f57e48..aefab6143 100644 --- a/packages/core/src/expand.ts +++ b/packages/core/src/expand.ts @@ -122,6 +122,7 @@ import { isPrivateImplementation, issueInvocation } from "./invocation-identity. import type { IdentityDomain } from "./invocation-identity.ts"; import { protectedContentLease } from "./protected-content.ts"; import type { SyntaxReference } from "./syntax-reference.ts"; +import { useNarrowedSyntax } from "./syntax-api.ts"; import { withInvocation } from "./invocation.ts"; import { observeExpansion } from "./component-expansion.ts"; import type { ObservedExpansion } from "./component-expansion.ts"; @@ -369,6 +370,17 @@ interface ProjectionState { */ checkedFailures: CheckedFailures | undefined; environment: ExecutionEnvironment | undefined; + /** + * The admission a protected producer's content is narrowed to, when one + * narrowed it. + * + * Set only where a boundary replaced the reference, and reported to the + * contextual Api in the content scope below. The ordinary handle leaves it + * absent on purpose: the execution's own reference is already the Api's + * fallback, and reinstalling it here as middleware would shadow a host that + * stated its own profile. + */ + narrowedSyntax: SyntaxReference | undefined; imports: ComponentImportTerminal | undefined; } @@ -493,6 +505,9 @@ function createProjectionHandle(state: ProjectionState): ProjectionHandle { const task = contentScope.scope.run(function* () { try { yield* ErrorMode.set(options.mode); + if (state.narrowedSyntax !== undefined) { + yield* useNarrowedSyntax(state.narrowedSyntax); + } if (options.segments.length === 0) { outcome.resolve({ segments: [] }); return; @@ -592,6 +607,9 @@ function createProjectionHandle(state: ProjectionState): ProjectionHandle { const task = contentScope.scope.run(function* () { try { yield* ErrorMode.set(mode); + if (state.narrowedSyntax !== undefined) { + yield* useNarrowedSyntax(state.narrowedSyntax); + } if (!slotErrorsEmitted && slots.errors.length > 0) { slotErrorsEmitted = true; for (const slotError of slots.errors) { @@ -3074,6 +3092,7 @@ function* expandComponent( invocation, enclosing, children, + narrowedSyntax: undefined, caller: { env: capturedCallerEnv, meta: callerMeta, @@ -3587,6 +3606,7 @@ function* expandFunctionComponent( projecting: issued.projecting, enclosing, children, + narrowedSyntax: undefined, caller: { env: undefined, meta: callerMeta, @@ -3741,6 +3761,7 @@ function* expandFunctionComponent( // content scope, not a second expansion. const narrowed = createProjectionHandle({ ...projectionState, + narrowedSyntax: syntax, // Only the reference is replaced, and only where there is an // environment to replace it on: an expansion driven without // one narrows nothing because it admitted nothing. diff --git a/packages/core/src/generated-xmd.ts b/packages/core/src/generated-xmd.ts index 0b4101818..818e7ffa3 100644 --- a/packages/core/src/generated-xmd.ts +++ b/packages/core/src/generated-xmd.ts @@ -157,6 +157,7 @@ import { installFormSyntax, invocationForm } from "./invocation-identity.ts"; import type { FormSyntax, ComponentRouting } from "./invocation-identity.ts"; import type { ComponentInvocation } from "./invocation-identity.ts"; import type { SyntaxReference } from "./syntax-reference.ts"; +import { useNarrowedSyntax } from "./syntax-api.ts"; import type { ComponentElement, FunctionComponentDefinition, @@ -2443,6 +2444,12 @@ function expand( ): Operation { return scoped(function* () { yield* ErrorMode.set("throw"); + // The admission this fragment was evaluated under, reported for the length + // of the fragment. Named documentation stays with the enclosing owner, so a + // generated body can still be told about a component it may not run. + if (syntax !== undefined) { + yield* useNarrowedSyntax(syntax); + } // Every expression this expansion resolves — an ordinary prop and a // declared capture alike — reads the declarative grammar rather than the // trusted-document evaluator. Set on this scope, so it ends with the diff --git a/packages/core/src/syntax-api.ts b/packages/core/src/syntax-api.ts new file mode 100644 index 000000000..fa39f6593 --- /dev/null +++ b/packages/core/src/syntax-api.ts @@ -0,0 +1,123 @@ +/** + * The description in effect here, as a contextual Api. + * + * ``, `xmd syntax` and a Plan's authorship all ask the same three + * questions — what are the symbols, what is the text, what do these names + * mean — and before this they asked three different things in three places. So + * they are one descriptor, dispatched by name, and a host, a trusted boundary + * or ordinary middleware can answer, delegate or transform any of them. + * + * ## Why the properties are Operations + * + * `symbols` and `text` are values a caller reads, not calls it makes, and they + * are also work: a description is inspected lazily, from a filesystem, on + * demand. `Operation`-valued properties are both at once. A caller writes + * `yield* Syntax.operations.text` with no parentheses, and a provider's body + * may suspend, read a file and delegate — with no cast hiding an operation + * behind a string. + * + * The defaults are *reusable* operations. `call(factory)` starts fresh + * generator work on each read, so two reads of one property are two answers + * rather than one consumed generator and then nothing. + * + * ## The fallback is not middleware + * + * An execution derives a description from what it captured + * (`syntax-reference.ts`), and that has to be what answers when nothing else + * does — while still losing to a host that stated its own profile. Those are + * different slots, and conflating them is the bug this module exists to avoid: + * + * - {@link useRootSyntaxReference} binds the execution's own reference as the + * value the *default* reads. A default is the furthest thing from the + * caller, so an enclosing `useSyntax()` profile still wins. + * - {@link useNarrowedSyntax} installs real middleware at `min` for a + * boundary that has admitted less than its enclosing site. That one is meant + * to shadow: it is reporting an admission, not offering a fallback, and it + * ends with the scope that installed it. + * + * Neither grants anything. A description that names a component does not admit + * it, and one that hides a component does not disable it. + */ + +import { type Api, createApi } from "@effectionx/context-api"; +import { call, createContext, type Operation } from "effection"; +import type { SyntaxReference } from "./syntax-reference.ts"; +import type { SyntaxSymbols } from "./inspect.ts"; +import { renderSyntaxMarkdown } from "./syntax-markdown.ts"; + +/** The three questions anything describing this site answers. */ +export interface SyntaxApi { + /** What may be written here, structured. */ + readonly symbols: Operation; + /** The same answer as Markdown, which is what `xmd syntax` prints. */ + readonly text: Operation; + /** These components' metadata and long-form documentation. */ + documentation(names: readonly string[]): Operation; +} + +/** + * The execution's own reference, read by the defaults below. + * + * A context rather than installed middleware, because this is the answer of + * last resort: middleware would sit in front of an enclosing host profile and + * shadow it, which is precisely backwards. + */ +const DefaultReference = createContext( + "executablemd.core.syntax.defaultReference", + undefined, +); + +function* reference(): Operation { + const value = yield* DefaultReference.get(); + if (value === undefined) { + throw new Error("Syntax has no descriptive reference in this scope."); + } + return value; +} + +export const Syntax: Api = createApi("executablemd.core.syntax", { + symbols: call(function* (): Operation { + return yield* (yield* reference()).symbols(); + }), + // Through the Api, not through the reference: a nearer provider that replaced + // the symbols is describing this site, so the text has to render what that + // provider said rather than what the fallback would have. + text: call(function* (): Operation { + return renderSyntaxMarkdown(yield* Syntax.operations.symbols); + }), + documentation(names): Operation { + return call(function* () { + return yield* (yield* reference()).documentation(names); + }); + }, +}); + +/** + * The authoring basis one named lookup is resolving, while it resolves it. + * + * Set by the reference that owns the lookup and read by its own default symbols + * provider, so an occurrence that needs both its metadata and its availability + * inspects a moving root once rather than twice. Nothing outside that one + * lookup's scope sees it. + */ +export const NamedBasis = createContext( + "executablemd.core.syntax.namedBasis", + undefined, +); + +/** Bind the reference this execution derived as the Api's fallback answer. */ +export function* useRootSyntaxReference(value: SyntaxReference): Operation { + // A fresh execution resolves its own basis. Started inside a named lookup — + // an isolated child, or a fragment evaluated while one is open — it would + // otherwise inherit that occurrence's basis as its root symbols. + yield* NamedBasis.set(undefined); + yield* DefaultReference.set(value); +} + +/** Report a narrowed admission for the scope a boundary is evaluating. */ +export function useNarrowedSyntax(value: SyntaxReference): Operation { + // Symbols only. Named documentation stays with the enclosing owner, because a + // subtree that may run less is still entitled to read about what it may not + // run — and the entry it reads says so. + return Syntax.around({ symbols: () => value.symbols() }, { at: "min" }); +} diff --git a/packages/core/src/syntax-markdown.ts b/packages/core/src/syntax-markdown.ts index 730fad365..eb1396c2e 100644 --- a/packages/core/src/syntax-markdown.ts +++ b/packages/core/src/syntax-markdown.ts @@ -24,9 +24,20 @@ import type { StructuralSyntaxEntry, SyntaxSymbols, } from "./inspect.ts"; -import { NO_DOCUMENTATION } from "./documentation-index.ts"; import type { ComponentOrigin, Json, PropsSchema } from "./types.ts"; +/** + * What a component with no authored documentation renders instead of prose. + * + * Here, with the renderer that emits it, rather than with the index that + * returns nothing for such a component: an absent entry and the sentence + * standing in for one are different facts, and only the renderer needs the + * second. Keeping it here is also what lets this module stay a leaf of the + * documentation index, so the contextual `Syntax` Api can reach the renderer + * without the component registry reaching back through it. + */ +export const NO_DOCUMENTATION = "No long-form documentation is available for this component."; + /** The three category kinds, taken from the symbols rather than restated. */ type CategoryKind = SyntaxSymbols["categories"][number]["kind"]; diff --git a/packages/core/src/syntax-reference.ts b/packages/core/src/syntax-reference.ts index 0169f2adf..77e448114 100644 --- a/packages/core/src/syntax-reference.ts +++ b/packages/core/src/syntax-reference.ts @@ -1,38 +1,40 @@ /** * What a document may write here, as one thing an execution carries. * - * `xmd syntax` answers that question for an environment nobody is running. - * Canonical `` answers it for the site an element was actually written - * at, and the two have to be the same answer — symbols an agent is shown and - * symbols an operator prints describe one vocabulary or they describe none. + * `xmd syntax` answers that question for an environment nobody is running, and + * canonical `` answers it for the site an element was actually written + * at. Both reach the answer through the contextual `Syntax` Api, so there is + * one construction and one renderer, and this module owns what that Api falls + * back to when nothing nearer answers. * - * So there is one construction and one renderer, and this module is where an - * execution keeps its own use of them. The reference is built from the - * selection inputs the execution captured before any installation, middleware or - * document code ran: the includes it resolves against, the registry it started - * with, the identity components and exact Markdown its host declared, and the - * component bundle it is closed over when it has one. Nothing is read from a - * context, a registry answer, or anything a document can reach. + * Two references are built here. The execution's own is built from the + * selection inputs it captured before any installation, middleware or document + * code ran: the includes it resolves against, the registry it started with, the + * identity components and exact Markdown its host declared, and the component + * bundle it is closed over when it has one. A host's own is built from the lazy + * source `useSyntax()` was handed, closed over the documentation contributions + * that were in scope when it was called. `xmd plan` uses that one, because a + * Plan is a program a later `xmd run` executes: the vocabulary the agent must + * be shown is that profile's rather than the authorship execution's. * - * A trusted host may state the symbols for its own profile instead. `xmd plan` - * does: a Plan is written to be run by `xmd run`, so the vocabulary the agent - * must be shown is the run profile's rather than the authorship execution's. - * That provider is captured with the rest of the installation, before any - * installed code exists, and one execution accepts one — two are refused rather - * than ordered, because ordering them would make which profile a document is - * shown depend on installation order. + * Either way the reference holds the *default* answer, not the effective one. + * It is reached through ordinary contextual dispatch, so nearer middleware can + * answer instead, delegate to it, or transform what it returns — which is why + * `documentation()` asks the Api for the symbols availability is decided + * against rather than deciding that for itself. * - * The reference decides nothing at all. It answers with text. Seeing a - * component named in the symbols neither registers it, resolves it, nor + * The reference admits nothing at all. It answers with symbols and with text. + * Seeing a component named in either neither registers it, resolves it, nor * admits it: what a name means is still `selectComponent()`'s decision, and * what may run is still the execution's. */ -import type { Operation } from "effection"; +import { type Operation, scoped } from "effection"; +import { NamedBasis, Syntax } from "./syntax-api.ts"; import { inspectSyntax } from "./inspect.ts"; import type { SyntaxSymbols } from "./inspect.ts"; -import { renderSelectedDocumentation, renderSyntaxMarkdown } from "./syntax-markdown.ts"; +import { renderSelectedDocumentation } from "./syntax-markdown.ts"; import type { SelectedEntry } from "./syntax-markdown.ts"; import { documentationIndexFor } from "./component-documentation.ts"; import type { DocumentationContribution } from "./component-documentation.ts"; @@ -45,21 +47,23 @@ import { SyntaxSelectionRefusal } from "./syntax-refusal.ts"; import type { ComponentOrigin, ComponentRegistry } from "./types.ts"; /** - * The symbols in scope for the segments being expanded. + * The default description for the segments being expanded. * * Held by the execution and handed to core's own expansion by value, beside the - * component resolution and the identity domains. It is not a Context: a context - * resolves by name, and a name is not a secret, so a document could build one - * and answer for the vocabulary it is shown. + * component resolution and the identity domains. It is not itself a Context: a + * context resolves by name, and a name is not a secret, so a document could + * build one and answer for the vocabulary it is shown. What a document *can* + * reach is the public `Syntax` Api in front of it, which answers with text and + * grants nothing. */ export interface SyntaxReference { - /** The symbols this site describes, rendered as Markdown. */ - symbols(): Operation; + /** The symbols this site describes, structured. */ + symbols(): Operation; /** * The selected components' metadata and long-form documentation. * * Two inputs, not one, and this is the reason the reference is an object - * rather than a string. *What may I write here* and *what may I read about* + * rather than a value. *What may I write here* and *what may I read about* * are different questions, and a narrowing evaluation boundary answers them * differently on purpose: the vocabulary it admits is smaller than the * vocabulary an author is entitled to understand. @@ -69,6 +73,10 @@ export interface SyntaxReference { * and each rendered entry states whether it is available in the current * evaluation. Collapsing the two would either hide reference material an * author needs or imply a permission they do not have. + * + * Availability is decided against the *effective* symbols, read once through + * the Api, so a host or boundary that customized what may run here is the + * same thing that decides what this annotation says. */ documentation(names: readonly string[]): Operation; /** @@ -89,16 +97,6 @@ export interface SyntaxReference { available(symbols: SyntaxSymbols): SyntaxReference; } -/** - * A trusted host's statement of the symbols its profile describes. - * - * Captured by value with the rest of the installation, before any installed - * code, middleware or document code runs. It returns the symbols and core - * renders them, so a host cannot make its profile print differently from the - * way `xmd syntax` prints the same symbols. - */ -export type SyntaxSymbolsProvider = () => Operation; - /** The selection inputs an execution captured, as symbol construction reads them. */ export interface CapturedSymbolInputs { readonly includes: readonly string[]; @@ -124,7 +122,6 @@ export interface CapturedSymbolInputs { */ export function rootSyntaxReference( inputs: CapturedSymbolInputs, - provider: SyntaxSymbolsProvider | undefined, /** * The documentation the installed packages contribute. * @@ -135,9 +132,6 @@ export function rootSyntaxReference( */ contributions: readonly DocumentationContribution[] = [], ): SyntaxReference { - function* current(): Operation { - return provider === undefined ? yield* derived(inputs) : yield* provider(); - } // Snapshotted once, here, so the contributions a reference reads are the // ones the collection boundary captured rather than whatever the caller's // objects hold by the time a document asks. @@ -145,7 +139,7 @@ export function rootSyntaxReference( // No admission at a root: nothing has narrowed what may execute, so the one // set of symbols this resolves is both what a document may write and what it // may read about. - return referencing(current, undefined, captured); + return referencing(() => derived(inputs), undefined, captured); } /** @@ -171,31 +165,40 @@ function referencing( contributions: readonly DocumentationContribution[], ): SyntaxReference { return { - *symbols(): Operation { + *symbols(): Operation { // A narrowed reference reports its admission and asks the enclosing - // symbols for nothing — the bare form is about what runs. - return renderSyntaxMarkdown(admitted ?? (yield* authoring())); + // symbols for nothing — the bare form is about what runs. Otherwise this + // is the default answer, and the basis below is consulted first so that + // the one occurrence already resolving a basis gets that same value back + // instead of resolving a moving root a second time. + return admitted ?? (yield* NamedBasis.get()) ?? (yield* authoring()); }, - *documentation(names: readonly string[]): Operation { - // One resolution, both decisions. The provider is asked first and its - // failures propagate untouched: a symbols provider that throws is this - // run's infrastructure, whatever it happens to call its error. - const readable = yield* authoring(); - const runnable = admitted ?? readable; - const index = documentationIndexFor(contributions); - // Only the selection this module performs itself is restated under core's - // own namespaced identity. The provider has already returned, so nothing - // it raised can reach this. - let selected; - try { - selected = select(readable, runnable, names, index); - } catch (error) { - if (error instanceof UnknownComponentError) { - throw new SyntaxSelectionRefusal(error.message); + documentation(names: readonly string[]): Operation { + return scoped(function* () { + // One basis per occurrence, resolved here and published for the length + // of this lookup only. Availability is then read through the Api, so a + // host or boundary that customized what runs decides the annotation — + // and when that read reaches *this* reference's default, it comes back + // with the basis above rather than inspecting the root again. Two + // occurrences enter two scopes and resolve independently. + const readable = yield* authoring(); + yield* NamedBasis.set(readable); + const runnable = yield* Syntax.operations.symbols; + const index = documentationIndexFor(contributions); + // Only the selection this module performs itself is restated under + // core's own namespaced identity. Both symbol reads have already + // returned, so nothing either of them raised can reach this. + let selected; + try { + selected = select(readable, runnable, names, index); + } catch (error) { + if (error instanceof UnknownComponentError) { + throw new SyntaxSelectionRefusal(error.message); + } + throw error; } - throw error; - } - return renderSelectedDocumentation(selected); + return renderSelectedDocumentation(selected); + }); }, available(next: SyntaxSymbols): SyntaxReference { // The enclosing authoring symbols and the enclosing contributions, @@ -384,3 +387,22 @@ export function syntaxReference( captured, ); } + +/** + * The reference a host's own profile carries. + * + * What `useSyntax()` binds. The source is lazy for the reason the derived one + * is — a command that never renders a description inspects nothing — and the + * contributions are snapshotted here, from the scope that called the helper, + * because a host that bootstrapped a profile and then handed over a bare + * symbols callback would lose the index its own packages contributed. + * + * No admission: a host states one profile, and what narrows it afterwards is a + * canonical evaluation boundary rather than this. + */ +export function profileSyntaxReference( + source: () => Operation, + contributions: readonly DocumentationContribution[], +): SyntaxReference { + return referencing(source, undefined, snapshotContributions(contributions)); +} diff --git a/packages/core/src/use-syntax.ts b/packages/core/src/use-syntax.ts new file mode 100644 index 000000000..fb9688160 --- /dev/null +++ b/packages/core/src/use-syntax.ts @@ -0,0 +1,42 @@ +/** + * Binding a host's own description, as one setup operation. + * + * The thing a host actually has is not a set of symbols — it is a profile it + * can inspect, plus the documentation its packages contributed. Both have to be + * taken here, in the scope the host assembled, because only one of them is + * lazy: the source runs later, when an occurrence renders, but the + * contributions are gone as soon as that assembly's scope ends. A host that + * bootstrapped its profile and then handed over a bare symbols callback would + * keep the first and lose the second, and a named lookup would fall back to + * whichever catalog the execution happened to have. + */ + +import type { Operation } from "effection"; +import type { SyntaxSymbols } from "./inspect.ts"; +import { capturedDocumentation } from "./documentation-api.ts"; +import { Syntax } from "./syntax-api.ts"; +import { profileSyntaxReference } from "./syntax-reference.ts"; + +/** + * Describe this scope with the host's own profile. + * + * The source is not read here. Installing is binding, not inspecting, so a + * command that prints nothing and a run whose document never writes + * `` both enumerate nothing. + * + * Installed at `min`, which is the implementation slot: this is a host stating + * what it has, so ordinary middleware nearer the caller still answers, + * delegates to it or transforms what it returns. Symbols and named + * documentation both, because a host supplying one and not the other would + * describe its own vocabulary with somebody else's prose. + */ +export function* useSyntax(source: () => Operation): Operation { + const value = profileSyntaxReference(source, yield* capturedDocumentation()); + yield* Syntax.around( + { + symbols: () => value.symbols(), + documentation: ([names]) => value.documentation(names), + }, + { at: "min" }, + ); +} diff --git a/packages/core/tests/evaluate-component.test.ts b/packages/core/tests/evaluate-component.test.ts index d3a8aaa74..f16d00f28 100644 --- a/packages/core/tests/evaluate-component.test.ts +++ b/packages/core/tests/evaluate-component.test.ts @@ -34,6 +34,7 @@ import { import type { ExecutionInstallation, FragmentEvaluationInput } from "../host.ts"; import { registerComponents } from "../src/components/registration.ts"; import { elicitWriteEntry } from "../host.ts"; +import { useSyntax } from "../host.ts"; import { Elicitation } from "../src/elicitation-api.ts"; import type { ElicitationRequest } from "../src/elicitation-api.ts"; import { retainedSource } from "../src/root-source.ts"; @@ -1922,10 +1923,12 @@ describe("Tier FE34 — the shared read profile", () => { shared(files), { // deno-lint-ignore require-yield - *symbols(): Operation { - const error = new Error("the provider could not build the symbols"); - error.name = "executablemd.core.syntax-selection-refusal"; - throw error; + *install(): Operation { + yield* useSyntax(function* (): Operation { + const error = new Error("the provider could not build the symbols"); + error.name = "executablemd.core.syntax-selection-refusal"; + throw error; + }); }, }, ]); diff --git a/packages/core/tests/execution-init.test.ts b/packages/core/tests/execution-init.test.ts new file mode 100644 index 000000000..6b0696834 --- /dev/null +++ b/packages/core/tests/execution-init.test.ts @@ -0,0 +1,258 @@ +import { describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import { createContext, ensure, scoped, spawn, suspend, withResolvers } from "effection"; +import type { Operation } from "effection"; +import { InMemoryStream } from "@executablemd/durable-streams"; +import { Execution } from "../api.ts"; +import { collect, execute, Execution as RootExecution, inlineSource } from "../mod.ts"; +import { executeInstalled, fileReadEntry, Markdown, sourceDigest } from "../host.ts"; +import type { JournalAdmission } from "../host.ts"; +import { Documentation } from "../src/documentation-api.ts"; +import { recordedFiles, useRecordedFiles } from "./support/fragment-files.ts"; + +const State = createContext<{ id: number } | undefined>("executablemd.test.init.state", undefined); + +function* run(text = "hello\n"): Operation { + return String( + yield* collect( + yield* execute({ ...inlineSource(text), stream: new InMemoryStream(), includes: [] }), + ), + ); +} + +function* failure(operation: Operation): Operation { + try { + yield* operation; + return undefined; + } catch (error) { + return error; + } +} + +describe("Tier EI — execution-owned initialization", () => { + it("EI1: the public descriptors agree, default init is no-op and each fresh owner dispatches once", function* () { + expect(Execution).toBe(RootExecution); + yield* Execution.operations.init(); + expect(yield* State.get()).toBe(undefined); + let calls = 0; + yield* Execution.around({ + *init(_args, next) { + calls += 1; + yield* State.set({ id: calls }); + yield* next(); + }, + }); + const execution = yield* execute({ + ...inlineSource("hello\n"), + stream: new InMemoryStream(), + includes: [], + }); + expect(String(yield* collect(execution))).toContain("hello"); + const completed = yield* execution; + expect(yield* execution).toBe(completed); + expect(calls).toBe(1); + yield* run(); + expect(calls).toBe(2); + expect(yield* State.get()).toBe(undefined); + }); + + it("EI2: initialization precedes assembly, captured documentation and execution/document policy", function* () { + const order: string[] = []; + yield* Execution.around({ + *init(_args, next) { + order.push("init"); + yield* State.set({ id: 7 }); + yield* next(); + }, + *execute([request], next) { + expect((yield* State.expect())?.id).toBe(7); + order.push("execute"); + yield* next(request); + }, + *document([request], next) { + expect((yield* State.expect())?.id).toBe(7); + order.push("document"); + yield* next(request); + }, + }); + yield* Documentation.around({ + *contributions([read], next) { + expect((yield* State.expect())?.id).toBe(7); + order.push("documentation"); + return yield* next(read); + }, + }); + const execution = yield* executeInstalled( + { ...inlineSource("\n"), includes: [], stream: new InMemoryStream() }, + [ + { + *install() { + expect((yield* State.expect())?.id).toBe(7); + order.push("install"); + }, + }, + ], + ); + expect(String(yield* collect(execution))).toContain("### ``"); + expect(order).toEqual(["init", "install", "documentation", "execute", "document"]); + }); + + it("EI3: init cannot rewrite captured admissions, declarations or generated profile tables", function* () { + const admitted: string[] = []; + const admissions: JournalAdmission[] = [ + function* () { + admitted.push("original"); + }, + ]; + const source = "CAPTURED DECLARATION\n"; + const declarations = [ + Markdown({ name: "Kept", origin: "test://init/kept", source, digest: sourceDigest(source) }), + ]; + const read = [fileReadEntry()]; + const files = recordedFiles({ "note.md": "CAPTURED PROFILE" }); + yield* Execution.around({ + *init(_args, next) { + admissions.length = 0; + declarations.length = 0; + read.length = 0; + yield* useRecordedFiles(files); + yield* next(); + }, + }); + const text = "\n'} />\n"; + const execution = yield* executeInstalled( + { ...inlineSource(text), includes: [], stream: new InMemoryStream() }, + [{ admissions, declarations, evaluation: { read } }], + ); + const output = String(yield* collect(execution)); + expect(admitted).toEqual(["original"]); + expect(output).toContain("CAPTURED DECLARATION"); + expect(output).toContain("CAPTURED PROFILE"); + expect(files.performed).toEqual(["read note.md"]); + }); + + it("EI4: initializer failure settles only after acquired teardown and reaches no policy or root", function* () { + const entered = withResolvers(); + const release = withResolvers(); + const cleanup = withResolvers(); + const phases: string[] = []; + const stream = new InMemoryStream(); + yield* Execution.around({ + *init() { + yield* ensure(function* () { + cleanup.resolve(); + yield* release.operation; + phases.push("released"); + }); + entered.resolve(); + throw new Error("init refused"); + }, + *execute([request], next) { + phases.push("policy"); + yield* next(request); + }, + }); + let settled = false; + const running = yield* spawn(function* () { + const error = yield* failure( + execute({ ...inlineSource("must not run"), stream, includes: [] }), + ); + settled = true; + return error; + }); + yield* entered.operation; + yield* cleanup.operation; + expect(settled).toBe(false); + expect(stream.snapshot()).toEqual([]); + release.resolve(); + expect(String(yield* running)).toContain("init refused"); + expect(phases).toEqual(["released"]); + }); + + it("EI5: deterministic cancellation during init joins cleanup before halt returns", function* () { + const entered = withResolvers(); + const cleaning = withResolvers(); + const release = withResolvers(); + const phases: string[] = []; + const stream = new InMemoryStream(); + yield* Execution.around({ + *init() { + yield* ensure(function* () { + cleaning.resolve(); + yield* release.operation; + phases.push("released"); + }); + entered.resolve(); + yield* suspend(); + }, + *execute([request], next) { + phases.push("policy"); + yield* next(request); + }, + }); + const running = yield* spawn(() => + execute({ ...inlineSource("must not run"), stream, includes: [] }), + ); + yield* entered.operation; + let halted = false; + const halting = yield* spawn(function* () { + yield* running.halt(); + halted = true; + }); + yield* cleaning.operation; + expect(halted).toBe(false); + expect(stream.snapshot()).toEqual([]); + release.resolve(); + yield* halting; + expect(phases).toEqual(["released"]); + }); + + it("EI6: concurrent owners get independent state inherited by ordinary descendants", function* () { + let counter = 0; + const both = withResolvers(); + const release = withResolvers(); + const seen: number[] = []; + yield* Execution.around({ + *init(_args, next) { + yield* State.set({ id: ++counter }); + yield* next(); + }, + *execute([request], next) { + seen.push((yield* State.expect())?.id ?? 0); + if (seen.length === 2) { + both.resolve(); + } + yield* release.operation; + const parent = yield* State.get(); + expect(yield* scoped(() => State.get())).toBe(parent); + yield* next(request); + }, + }); + const first = yield* spawn(() => run()); + const second = yield* spawn(() => run()); + yield* both.operation; + expect([...seen].sort((a, b) => a - b)).toEqual([1, 2]); + expect(yield* State.get()).toBe(undefined); + release.resolve(); + yield* first; + yield* second; + expect(yield* scoped(() => State.get())).toBe(undefined); + }); + + it("EI7: a no-op initializer invents no Files provider; explicit initialization supplies one", function* () { + expect(String(yield* failure(run('')))).toContain("Files"); + const files = recordedFiles({ "note.md": "INSTALLED BY INIT" }); + const output = yield* scoped(function* () { + yield* Execution.around({ + *init(_args, next) { + yield* useRecordedFiles(files); + yield* next(); + }, + }); + return yield* run(''); + }); + expect(output).toContain("INSTALLED BY INIT"); + expect(files.performed).toEqual(["read note.md"]); + expect(String(yield* failure(run('')))).toContain("Files"); + }); +}); diff --git a/packages/core/tests/protected-content.test.ts b/packages/core/tests/protected-content.test.ts index ffed8de1a..1a66c48c1 100644 --- a/packages/core/tests/protected-content.test.ts +++ b/packages/core/tests/protected-content.test.ts @@ -24,6 +24,7 @@ import type { Operation } from "effection"; import { ComponentInvocationError } from "../src/invocation-identity.ts"; import { protectedContentLease } from "../src/protected-content.ts"; +import type { SyntaxSymbols } from "../api.ts"; import type { SyntaxReference } from "../src/syntax-reference.ts"; /** @@ -36,8 +37,15 @@ import type { SyntaxReference } from "../src/syntax-reference.ts"; */ const REFERENCE: SyntaxReference = { // deno-lint-ignore require-yield - *symbols(): Operation { - return ""; + *symbols(): Operation { + return { + version: 2, + categories: [ + { kind: "structural", entries: [] }, + { kind: "built-in", entries: [] }, + { kind: "user-provided", entries: [] }, + ], + }; }, // deno-lint-ignore require-yield *documentation(): Operation { diff --git a/packages/core/tests/syntax-api.test.ts b/packages/core/tests/syntax-api.test.ts new file mode 100644 index 000000000..0c6e4f982 --- /dev/null +++ b/packages/core/tests/syntax-api.test.ts @@ -0,0 +1,310 @@ +import { describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import { type Api, createApi } from "@effectionx/context-api"; +import { call, ensure, scoped, until } from "effection"; +import type { Operation } from "effection"; +import { readTextFile, rm, writeTextFile } from "@effectionx/fs"; +import { mkdtemp } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { InMemoryStream } from "@executablemd/durable-streams"; +import { Syntax } from "../api.ts"; +import type { SyntaxApi, SyntaxSymbols } from "../api.ts"; +import { + Syntax as RootSyntax, + collect, + inlineSource, + renderSyntaxMarkdown, + UnknownComponentError, +} from "../mod.ts"; +import { executeInstalled, syntaxReadEntry, useSyntax } from "../host.ts"; + +function symbolsOf(...names: string[]): SyntaxSymbols { + return { + version: 2, + categories: [ + { kind: "structural", entries: [] }, + { + kind: "built-in", + entries: names.map((name) => ({ + kind: "component", + name, + origin: { kind: "registered", origin: "test://syntax-api", reserved: false }, + sourceKind: "registered", + inspectability: "complete", + forms: ["self-closing"], + props: { type: "object", properties: {}, additionalProperties: false }, + captures: [], + returnMode: "text", + returns: { type: "string" }, + })), + }, + { kind: "user-provided", entries: [] }, + ], + }; +} + +function without(symbols: SyntaxSymbols, name: string): SyntaxSymbols { + return { + ...symbols, + categories: [ + symbols.categories[0], + { + ...symbols.categories[1], + entries: symbols.categories[1].entries.filter((entry) => entry.name !== name), + }, + symbols.categories[2], + ], + }; +} + +function* run(text: string, stream = new InMemoryStream()): Operation { + return String( + yield* collect( + yield* executeInstalled({ ...inlineSource(text), stream, includes: [] }, [ + { evaluation: { read: [syntaxReadEntry()] } }, + ]), + ), + ); +} + +describe("Tier SA — contextual Syntax", () => { + it("SA1: exact public property types, reusable lazy defaults and a suspending transform", function* () { + const descriptor: Api = Syntax; + expect(descriptor).toBe(RootSyntax); + const directory = yield* until(mkdtemp(join(tmpdir(), "syntax-api-"))); + yield* ensure(() => rm(directory, { recursive: true, force: true })); + const path = join(directory, "marker"); + yield* writeTextFile(path, "First"); + let reads = 0; + yield* useSyntax(function* () { + reads += 1; + return symbolsOf(yield* readTextFile(path), "Hidden"); + }); + expect(reads).toBe(0); + yield* Syntax.around({ + *symbols(_args, next) { + return without(yield* next(), "Hidden"); + }, + }); + const first = yield* descriptor.operations.text; + expect(first).toBe(renderSyntaxMarkdown(without(symbolsOf("First", "Hidden"), "Hidden"))); + yield* writeTextFile(path, "Second"); + expect(yield* descriptor.operations.text).toContain("### ``"); + expect(reads).toBe(2); + }); + + it("SA2: symbols affect default text and named availability; text-only middleware changes neither", function* () { + const all = symbolsOf("Kept", "Hidden"); + let source = 0; + let effective = 0; + yield* useSyntax(function* () { + source += 1; + return all; + }); + yield* Syntax.around({ + *symbols(_args, next) { + effective += 1; + return without(yield* next(), "Hidden"); + }, + }); + const text = yield* Syntax.operations.text; + expect(text).toContain("### ``"); + expect(text).not.toContain("### ``"); + source = effective = 0; + const named = yield* Syntax.operations.documentation(["Hidden", "Kept"]); + expect(named).toContain("### ``"); + expect(named).toContain("**Available in this evaluation:** no"); + expect(named).toContain("**Available in this evaluation:** yes"); + expect(source).toBe(1); + expect(effective).toBe(1); + yield* scoped(function* () { + yield* Syntax.around({ + *text() { + return "custom text"; + }, + }); + expect(yield* Syntax.operations.text).toBe("custom text"); + expect(yield* Syntax.operations.symbols).toEqual(without(all, "Hidden")); + expect(yield* Syntax.operations.documentation(["Hidden", "Kept"])).toBe(named); + }); + expect(yield* Syntax.operations.text).toBe(text); + }); + + it("SA3: synthetic answers skip default source work and leaving the child restores its providers", function* () { + let reads = 0; + yield* useSyntax(function* () { + reads += 1; + return symbolsOf("Outer"); + }); + yield* scoped(function* () { + yield* Syntax.around({ + *symbols() { + return symbolsOf("Synthetic"); + }, + *documentation() { + return "synthetic named answer"; + }, + }); + expect(yield* Syntax.operations.text).toContain("Synthetic"); + expect(yield* Syntax.operations.documentation(["Unknown"])).toBe("synthetic named answer"); + expect(reads).toBe(0); + }); + expect(yield* Syntax.operations.text).toContain("Outer"); + expect(reads).toBe(1); + }); + + it("SA4: moving named occurrences resolve one basis each and record the final transformed answer", function* () { + let reads = 0; + let symbols = 0; + yield* useSyntax(function* () { + return symbolsOf(`Marker${++reads}`); + }); + yield* Syntax.around({ + *symbols(_args, next) { + symbols += 1; + return yield* next(); + }, + *documentation([names], next) { + return `${yield* next(names)}\nFINAL NAMED`; + }, + }); + const stream = new InMemoryStream(); + const source = '\n\n'; + const output = yield* run(source, stream); + expect(output).toContain("Marker1"); + expect(output).toContain("Marker2"); + expect(output.match(/FINAL NAMED/g)).toHaveLength(2); + expect(reads).toBe(2); + expect(symbols).toBe(2); + const records = stream + .snapshot() + .filter((event) => event.type === "yield" && event.description.type === "syntax_symbols"); + expect(records).toHaveLength(2); + for (const record of records) { + if (record.type !== "yield" || record.result.status !== "ok") { + throw new Error("no retained Syntax answer"); + } + expect(JSON.stringify(record.result.value)).toContain("FINAL NAMED"); + } + }); + + it("SA5: complete and partial replay restore bare/named final text without live providers", function* () { + const source = '\n\n'; + const stream = new InMemoryStream(); + const first = yield* scoped(function* () { + yield* useSyntax(function* () { + return symbolsOf("HostOnly"); + }); + yield* Syntax.around({ + *text(_args, next) { + return `${yield* next()}\nFINAL BARE`; + }, + *documentation([names], next) { + return `${yield* next(names)}\nFINAL NAMED`; + }, + }); + return yield* run(source, stream); + }); + expect(first).toContain("FINAL BARE"); + expect(first).toContain("FINAL NAMED"); + const partial = new InMemoryStream(stream.snapshot().filter((event) => event.type !== "close")); + yield* useSyntax(function* (): Operation { + throw new Error("live source reached on replay"); + }); + yield* Syntax.around({ + *symbols(): Operation { + throw new Error("live symbols reached on replay"); + }, + *text(): Operation { + throw new Error("live text reached on replay"); + }, + *documentation(): Operation { + throw new Error("live catalog reached on replay"); + }, + }); + expect(yield* run(source, stream)).toBe(first); + expect(yield* run(source, partial)).toBe(first); + }); + + it("SA6: generated descriptions retain host-only metadata and never grant execution", function* () { + let reads = 0; + yield* useSyntax(function* () { + reads += 1; + return symbolsOf("HostOnly"); + }); + const output = yield* run( + "'} />\n\n", + ); + expect(output).toContain("### ``"); + expect(output).toContain("**Available in this evaluation:** no"); + expect(reads).toBe(2); + yield* Syntax.around({ + *symbols() { + return symbolsOf("HostOnly"); + }, + }); + const refused = yield* run("'} />\n"); + expect(refused).toContain("a component this host did not admit"); + expect(refused).not.toContain("No long-form documentation"); + }); + + it("SA7: a descriptive filter cannot disable an admitted generated component", function* () { + yield* Syntax.around({ + *symbols() { + return symbolsOf(); + }, + }); + yield* Syntax.around({ + *text() { + return "ADMITTED SYNTAX RAN"; + }, + }); + expect(yield* run("'} />\n")).toContain("ADMITTED SYNTAX RAN"); + }); + + it("SA8: canonical unknown names are safe records; provider-thrown UnknownComponentError is terminal", function* () { + const stream = new InMemoryStream(); + yield* useSyntax(function* () { + return symbolsOf("HostOnly"); + }); + yield* run('\n', stream); + const record = stream + .snapshot() + .find((event) => event.type === "yield" && event.description.type === "syntax_symbols"); + expect(record?.type === "yield" ? record.result.status : "absent").toBe("ok"); + expect(JSON.stringify(record)).toContain('"refused"'); + yield* scoped(function* () { + yield* useSyntax(function* (): Operation { + throw new UnknownComponentError("provider failure"); + }); + const failed = new InMemoryStream(); + yield* run('\n', failed); + const retained = failed + .snapshot() + .find((event) => event.type === "yield" && event.description.type === "syntax_symbols"); + expect(retained?.type === "yield" ? retained.result.status : "absent").toBe("err"); + expect(JSON.stringify(retained)).toContain("provider failure"); + }); + }); + + it("SA9: independently constructed same-name descriptors share ordinary middleware", function* () { + const other: Api = createApi("executablemd.core.syntax", { + symbols: call(function* (): Operation { + throw new Error("foreign default must not run"); + }), + text: call(function* (): Operation { + throw new Error("foreign default must not run"); + }), + *documentation() { + throw new Error("foreign default must not run"); + }, + }); + yield* other.around({ + *text() { + return "OTHER DESCRIPTOR"; + }, + }); + expect(yield* run("\n")).toContain("OTHER DESCRIPTOR"); + }); +}); diff --git a/packages/core/tests/syntax-component.test.ts b/packages/core/tests/syntax-component.test.ts index d4ef56982..80b9f7c7c 100644 --- a/packages/core/tests/syntax-component.test.ts +++ b/packages/core/tests/syntax-component.test.ts @@ -65,6 +65,9 @@ import { retainedSource } from "../src/root-source.ts"; import { renderSyntaxMarkdown } from "../src/syntax-markdown.ts"; import { syntaxReference, rootSyntaxReference } from "../src/syntax-reference.ts"; import type { SyntaxReference } from "../src/syntax-reference.ts"; +import { useRootSyntaxReference } from "../src/syntax-api.ts"; +import { useSyntax } from "../host.ts"; +import { Syntax } from "../api.ts"; import { capturedDocumentation, contributeDocumentation, @@ -123,9 +126,11 @@ function symbolsOf(...names: readonly string[]): SyntaxSymbols { function stating(symbols: SyntaxSymbols, calls: { count: number } = { count: 0 }) { const installation: ExecutionInstallation = { // deno-lint-ignore require-yield - *symbols(): Operation { - calls.count += 1; - return symbols; + *install(): Operation { + yield* useSyntax(function* (): Operation { + calls.count += 1; + return symbols; + }); }, }; return { installation, calls }; @@ -321,7 +326,6 @@ function symbolsNamed(name: string, origin: NamedOrigin): SyntaxSymbols { function* rootObservation(): Operation { return rootSyntaxReference( { includes: [], registry: new Map(), components: [], declarations: [] }, - undefined, // What canonical execution hands it: whatever the scope bootstrapped, // terminating in core's own. Passing nothing here would leave the reference // with no index at all, and a case about narrowing would then be reading a @@ -330,6 +334,16 @@ function* rootObservation(): Operation { ); } +function observedDocumentation( + reference: SyntaxReference, + names: readonly string[], +): Operation { + return scoped(function* () { + yield* useRootSyntaxReference(reference); + return yield* Syntax.operations.documentation(names); + }); +} + /** A working directory of this case's own, torn down on the way out. */ function useWorkingDirectory(body: (dir: string) => Operation): Operation { return scoped(function* () { @@ -601,11 +615,10 @@ describe("Tier SYN — the named form", () => { // And the snapshot renders that way through the reference an execution // builds from it, rather than only reading that way as a value. - const rendered = yield* syntaxReference( - symbolsOf("Marker"), - symbolsOf("Marker"), - captured, - ).documentation(["Marker"]); + const rendered = yield* observedDocumentation( + syntaxReference(symbolsOf("Marker"), symbolsOf("Marker"), captured), + ["Marker"], + ); expect(rendered).toContain("THE CAPTURED PROSE."); expect(rendered).not.toContain("SUBSTITUTED AFTER COLLECTION"); }); @@ -1314,9 +1327,11 @@ describe("Tier SYN — the named form", () => { const calls = { count: 0 }; const moving: ExecutionInstallation = { // deno-lint-ignore require-yield - *symbols(): Operation { - calls.count += 1; - return symbolsOf(`Marker${calls.count}`); + *install(): Operation { + yield* useSyntax(function* (): Operation { + calls.count += 1; + return symbolsOf(`Marker${calls.count}`); + }); }, }; @@ -1942,8 +1957,10 @@ describe("Tier SYN — the record one occurrence keeps", () => { // contribution refuses to answer at all. const moved: ExecutionInstallation = { // deno-lint-ignore require-yield - *symbols(): Operation { - throw new Error("the continuation rebuilt the symbols"); + *install(): Operation { + yield* useSyntax(function* (): Operation { + throw new Error("the continuation rebuilt the symbols"); + }); }, }; const continued = String(yield* run("\n", [moved], yield* continuing(first))); @@ -1982,12 +1999,14 @@ describe("Tier SYN — the record one occurrence keeps", () => { const teardown: string[] = []; const stream = new InMemoryStream(); const hanging: ExecutionInstallation = { - *symbols(): Operation { - yield* ensure(function* () { - teardown.push("released"); + *install(): Operation { + yield* useSyntax(function* (): Operation { + yield* ensure(function* () { + teardown.push("released"); + }); + yield* suspend(); + throw new Error("unreachable"); }); - yield* suspend(); - throw new Error("unreachable"); }, }; @@ -2085,10 +2104,10 @@ describe("Tier SYN — reference never permits", () => { it("SYN25b: a narrowed reference answers with exactly the symbols it was given", function* () { const narrowed = symbolsOf("Admitted"); const reference = syntaxReference(narrowed); - expect(yield* reference.symbols()).toBe(renderSyntaxMarkdown(narrowed)); + expect(renderSyntaxMarkdown(yield* reference.symbols())).toBe(renderSyntaxMarkdown(narrowed)); // Nothing of the enclosing site leaks into it: a name the wider profile has // is absent, because the symbols it was handed do not hold one. - expect(yield* reference.symbols()).not.toContain("### ``"); + expect(renderSyntaxMarkdown(yield* reference.symbols())).not.toContain("### ``"); }); /** @@ -2105,26 +2124,26 @@ describe("Tier SYN — reference never permits", () => { // What may execute here is the narrowed set, and the bare form reports // exactly that. - const available = yield* reference.symbols(); + const available = renderSyntaxMarkdown(yield* reference.symbols()); expect(available).toContain("### ``"); expect(available).not.toContain("### ``"); // Reference material comes from the enclosing symbols, so a component this // subtree may not run can still be explained — and the entry says so // rather than leaving a reader to assume they have both. - const documented = yield* reference.documentation(["Withheld"]); + const documented = yield* observedDocumentation(reference, ["Withheld"]); expect(documented).toContain("### ``"); expect(documented).toContain("**Available in this evaluation:** no"); // And one that is admitted reports the other answer, so the field is // discriminating rather than a constant. - const admitted = yield* reference.documentation(["Admitted"]); + const admitted = yield* observedDocumentation(reference, ["Admitted"]); expect(admitted).toContain("**Available in this evaluation:** yes"); // A boundary that narrows nothing has one set, and everything in it is // available — the ordinary case. const open = syntaxReference(enclosing); - expect(yield* open.documentation(["Withheld"])).toContain( + expect(yield* observedDocumentation(open, ["Withheld"])).toContain( "**Available in this evaluation:** yes", ); }); @@ -2192,7 +2211,7 @@ describe("Tier SYN — reference never permits", () => { for (const [what, origin] of Object.entries(impostors)) { const nested = syntaxReference(holding(origin), holding(authored)); - const rendered = yield* nested.documentation(["Elicit"]); + const rendered = yield* observedDocumentation(nested, ["Elicit"]); expect([what, rendered.includes("**Available in this evaluation:** no")]).toEqual([ what, true, @@ -2209,12 +2228,14 @@ describe("Tier SYN — reference never permits", () => { }; const moved: NamedOrigin = { ...bundled, sourceHash: "c".repeat(40) }; expect( - yield* syntaxReference(holding(moved), holding(bundled)).documentation(["Elicit"]), + yield* observedDocumentation(syntaxReference(holding(moved), holding(bundled)), ["Elicit"]), ).toContain("**Available in this evaluation:** no"); // The positive control: one exact identity, admitted. expect( - yield* syntaxReference(holding(authored), holding(authored)).documentation(["Elicit"]), + yield* observedDocumentation(syntaxReference(holding(authored), holding(authored)), [ + "Elicit", + ]), ).toContain("**Available in this evaluation:** yes"); }); @@ -2229,20 +2250,20 @@ describe("Tier SYN — reference never permits", () => { const narrowed = enclosing.available(admitted); // What may run is the admission. - const executable = yield* narrowed.symbols(); + const executable = renderSyntaxMarkdown(yield* narrowed.symbols()); expect(executable).toContain("### ``"); expect(executable).not.toContain("### ``"); // What may be read about is still the enclosing site's, with the enclosing // index behind it — so a real component's real documentation survives. - const documented = yield* narrowed.documentation(["Elicit"]); + const documented = yield* observedDocumentation(narrowed, ["Elicit"]); expect(documented).toContain("### ``"); expect(documented).toContain("Asks a person a structured question"); expect(documented).toContain("**Available in this evaluation:** no"); // And the enclosing reference is unchanged by having been narrowed. - expect(yield* enclosing.symbols()).toContain("### ``"); - expect(yield* enclosing.documentation(["Elicit"])).toContain( + expect(renderSyntaxMarkdown(yield* enclosing.symbols())).toContain("### ``"); + expect(yield* observedDocumentation(enclosing, ["Elicit"])).toContain( "**Available in this evaluation:** yes", ); }); @@ -2271,7 +2292,7 @@ describe("Tier SYN — reference never permits", () => { supplies.add("Beta"); supplies.delete("Alpha"); - const rendered = yield* reference.documentation(["Alpha"]); + const rendered = yield* observedDocumentation(reference, ["Alpha"]); expect(rendered).toContain("The captured documentation."); expect(rendered).not.toContain("SUBSTITUTED AFTER CAPTURE"); }); diff --git a/specs/executable-mdx-spec.md b/specs/executable-mdx-spec.md index ec23c5450..ff6c5a4ae 100644 --- a/specs/executable-mdx-spec.md +++ b/specs/executable-mdx-spec.md @@ -2783,7 +2783,10 @@ the bare form continues to list structural syntax alongside components. The documentation is read from the owning package through the direct filesystem, not through `API.Fs` or the document-facing `Files` provider. Both of those are middleware a running document can compose around, and a document that could -answer the read would decide what the product says about itself. +answer the read would decide what the product says about itself. What that read +produces is the *captured default* catalog. Live `Syntax` middleware composes in +front of it and may transform or replace the answer an occurrence receives — +which is a composition the host assembled, not a read a document answered. **Documentation composes with the components it describes.** A package's bootstrap installs its registrations and its documentation together — the @@ -2852,8 +2855,9 @@ host-maintained list of every package's registrations, is two lists that drift; this is one. **Reference and availability are separate.** The reference carries two inputs. -Bare `` reports what may **execute** at this site. The named form -selects from the **enclosing authoring symbols** and states, per entry, +Default bare `` reports the effective contextual symbols at this +site. The named form selects from the **enclosing authoring symbols** and +states, per entry, `**Available in this evaluation:** yes` or `no`. At a root the two are one set of symbols and everything selected is available; a trusted evaluation boundary that narrows execution keeps the enclosing reference, so a nested author can be told @@ -2885,14 +2889,47 @@ the import, delegate it and refuse it by throwing, but cannot answer it, replace what came back, mutate it, or hand back a definition kept from another import or built by another loaded copy. -**The symbols are the execution's own.** Canonical core builds them at the root from -the selection inputs that execution captured before any installation, middleware -or document code ran — its includes, the registry it started with, the identity -components and exact Markdown its host declared, and the component bundle it is -closed over — and carries them lexically on canonical core's own expansion -environment. A trusted host may state the symbols its profile describes instead, -captured on the same terms; one execution accepts one, and two are refused rather -than ordered. Nothing is built until an occurrence asks. +**Live descriptions use the contextual Syntax API.** Core exports this API and +its types from `@executablemd/core/api`, alongside the same Execution descriptor +exported from the package root: + +```ts +interface SyntaxApi { + readonly symbols: Operation; + readonly text: Operation; + documentation(names: readonly string[]): Operation; +} +``` + +Callers yield `Syntax.operations.symbols` and `.text` without parentheses. +Property middleware and its continuation return Operations. Defaults are +reusable lazy Operations, not stored consumable generator instances. Default +text renders effective contextual symbols through `renderSyntaxMarkdown()`. +Text middleware may then transform or replace that answer. Synthetic answers +that do not delegate perform no default inspection. + +The execution's private reference supplies its root fallback. A host can call +`useSyntax(source: () => Operation): Operation` from +`@executablemd/core/host` to bind a different complete descriptive profile. The +helper captures and snapshots existing scoped documentation contributions and +installs ordinary symbols/documentation providers at `min` without reading the +source. `ExecutionInstallation` carries no symbols provider or descriptive +provider count rule. Ordinary middleware composition decides the live answer. + +Default named lookup retains its enclosing catalog for metadata and long-form +text and reads effective symbols once for availability. Its authoring basis is +resolved once per occurrence and reused by that occurrence's default symbols +provider. Later occurrences resolve independently. Filtering symbols affects +bare text and named availability, while an omitted enclosing component remains +documentable. Text-only customization changes neither symbols nor named lookup. +A synthetic named answer skips the catalog entirely. + +Generated producer and body scopes narrow symbols only, retaining the enclosing +named-documentation owner. Their descriptive bindings end with those scopes. +Standalone Plan and CLI inspection bind the target run's declarative profile +through `useSyntax`; embedded Plan inherits its enclosing profile. CLI +inspection starts no execution initialization or operational provider merely to +print. Seeing a component in the symbols grants nothing. It neither registers, resolves nor authorizes that component: what a name means is still this section's @@ -9862,6 +9899,31 @@ workflow and returns a `DocumentExecution` handle. Options: - `secretDetection?` — detect credentials before durable events persist (default: enabled) +#### Execution initialization + +`Execution.init(): Operation` runs once in the existing invocation-owned +scope. Its default is no-op and supplies no Files provider. A host registers +ordinary middleware, installs its live dependencies there and delegates with +`yield* next()`. Core copies trusted admissions, declarations, preparations, +provider/install callbacks and the generated profile's host inputs and arms +revocation before calling init. Init finishes before installation callbacks, +documentation capture and public `Execution.execute` policy. The generated +profile's provider-bound definitions are resolved during document assembly. + +Failure or cancellation during init prevents execution policy, root import and +body and waits for all acquired cleanup before settlement. Concurrent +executions have independent initialized state; ordinary descendants inherit it. +A later sibling inherits no completed execution's state. Observing a completed +handle starts no new init. A fresh continuation establishes fresh live state +while completed durable operations and Syntax answers replay passively. + +`runDocument`, REPL `runEntry` and isolated ordinary `runProfileChild` all reach +this core initialization point. Isolated children explicitly reinstall selected +host/Plugin middleware after cwd assembly and inherit no arbitrary parent +handlers. REPL state is per entry below the existing Agent attachment; init does +not replace that attachment, durable preparation or the journal owner. The +REPL expansion controller does not pause at initialization. + #### The host's repository provider A document execution reaches repository operations only through the provider @@ -12176,6 +12238,31 @@ composition underneath it. | FX5–FX5d | Directory ownership | A body that fails and a body cancelled inside the region both leave the created directory, and the enclosing directory is read back **in the scope that installed it** rather than in a fresh run that would install its own; a refused ensure expands no content and writes nothing; an existing directory keeps what is in it | | ORC3 | The shipped command | A real `xmd run` performs the whole journey with candidates for all three names beside the document, ignores them, and restores the working directory for the sibling after the region | + +### Tier EI — Execution initialization + +| ID | Assertion | +| --- | --- | +| EI1 | Root and `/api` export the same descriptor; init defaults to no-op, dispatches once per fresh execution, and does not rerun for completed handle observation. | +| EI2 | Init-installed state is visible to assembly, documentation capture and execute/document policy in that order. | +| EI3 | Init cannot replace already captured admissions, declarations or generated profile tables. | +| EI4–EI5 | Initializer failure and deterministic cancellation reach no policy/root and settle only after held cleanup finishes. | +| EI6–EI7 | Concurrent state is independent and inherited by ordinary descendants; sibling state restores; no-op init invents no Files provider while explicit init supplies one. | +| EI-REPL | Two entries beneath the same session attachment own distinct initialized state and completed observation does not reinitialize. | +| EI-child | An isolated run child reinstalls chosen Plugin initialization after cwd assembly and does not inherit unrelated parent middleware. | + +### Tier SA — Contextual Syntax + +| ID | Assertion | +| --- | --- | +| SA1 | Exact public Operation-valued properties use reusable lazy defaults and a suspending source/transform. | +| SA2–SA3 | Symbols affect default text and named availability, text-only middleware changes neither structured symbols nor named lookup, synthetic answers skip default work, and scoped providers restore. | +| SA4 | Moving named occurrences each resolve one authoring basis and one effective symbols answer and retain the final transformed text. | +| SA5 | Complete and partial replay restore final bare/named text while every live symbols, text, catalog and source provider in scope is one that throws if consulted. | +| SA6–SA7 | Generated scopes retain host-only metadata while reporting availability; adding descriptions admits nothing and hiding an admitted component disables nothing. | +| SA8 | Canonical unknown names retain safe refusal values while provider-thrown UnknownComponentError remains an operation error. | +| SA9 | Independently constructed same-name API descriptors compose ordinary middleware. | + ### Tier SYN — The public `` component Named `SYN` rather than `SY` or `SL`, which already name the syntax symbols and diff --git a/specs/plan-command-spec.md b/specs/plan-command-spec.md index a1af74e09..b1ebc1f71 100644 --- a/specs/plan-command-spec.md +++ b/specs/plan-command-spec.md @@ -342,9 +342,11 @@ The host supplies two fixed internal inputs as that root's props: They are the adapter's own, and nothing a Plan declares is bound here: the properties a Plan's root declares are resolved by whoever runs it. The symbols -are not among them either. The command states the vocabulary its profile -describes at the execution boundary, captured before any installed code runs, -and the packaged Component reaches them by writing the public `` any +are not among them either. The command enters the target run's declarative +bootstraps and binds a lazy source through core's `useSyntax` helper before +execution. The source inspects captured target registry/declaration inputs, and +setup snapshots the same assembly's documentation contributions. The packaged +Component reaches the contextual answer by writing the public `` any document may write — so an authored phase can say that preparation is starting before the read happens, and the symbols the Agent is shown are the ones an operator can print. From bc2c59c8b6f7a5854dda7b6a8bc70970003987ba Mon Sep 17 00:00:00 2001 From: Taras Mankovski Date: Wed, 14 Oct 2026 14:18:37 -0400 Subject: [PATCH 2/2] =?UTF-8?q?=F0=9F=93=9D=20Separate=20the=20Syntax=20de?= =?UTF-8?q?scription=20from=20the=20admission=20it=20never=20makes?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Review feedback on #912. Prose and one test fixture; no behavior changes. Descriptions and execution permission are described as the separate things the SA rows prove. The spec's root sentence holds for the uncustomized default: descriptive middleware that filters the effective symbols separates the authoring and effective sets at a root as well, and the per-entry availability label is what says so. `SyntaxApi.symbols` is the effective structured description and `text` the effective text whose *default* renders it, rather than "what may be written" and "the same answer as Markdown". The captured catalog and the effective answer are separated where they were conflated: canonical collection owns the catalog a package ships, and live middleware composes in front of it — which SA9 shows is not necessarily a composition the host assembled. `SyntaxReference`'s rationale states the boundary that holds instead of the one that no longer explains it: component selection, invocation identity and durable recording stay canonical, and a description reaches none of them. The detailed contract stays in its owning chapter. `architecture.md`'s *Capability-backed execution* and *The syntax symbols boundary* keep the ordering, precedence, per-occurrence basis and scope-restoration invariants; the language spec keeps signatures, representative usage, observable behavior and the EI/SA rows, and links to those owners. Module headers keep the surprising constraints and lose the repeated introductions. `ExecutionApi`'s interface comment no longer promises every handler an `ExecutionRequest` — `init` is handed nothing, `document` a `DocumentRequest` — and the `Syntax` re-exports identify one descriptor and its owner instead of disagreeing about whether describing a site is a consumer boundary. The EI-child plugin fixture is written as `.mjs`. Its project is a bare directory with no `package.json`, so a `.ts` plugin there loads as CommonJS under Node, and the dependencies core reaches through publish an import condition only. The suite now passes under Deno, Node and Bun. --- architecture.md | 5 +- .../cli/tests/testing-execution-host.test.ts | 9 +- packages/core/api.ts | 19 ++-- packages/core/mod.ts | 6 +- packages/core/src/execute.ts | 17 ++-- packages/core/src/syntax-api.ts | 20 +++-- packages/core/src/syntax-reference.ts | 14 +-- specs/executable-mdx-spec.md | 86 +++++++++---------- 8 files changed, 97 insertions(+), 79 deletions(-) diff --git a/architecture.md b/architecture.md index b795663d0..f2a72a543 100644 --- a/architecture.md +++ b/architecture.md @@ -3759,8 +3759,9 @@ cancellation join partially acquired teardown before the invocation settles. Ordinary descendants inherit initialized state; concurrent executions and REPL entries own distinct state. Completed handle observation initializes nothing. An isolated run child reinstalls the selected host/Plugin middleware after cwd -assembly; arbitrary parent handlers do not cross that isolation. Existing REPL -Agent attachment and durable preparation keep their owners and ordering. +assembly; arbitrary parent handlers do not cross that isolation. REPL Agent +attachment and durable preparation are owned and ordered elsewhere, and +initialization neither replaces nor reorders them. Canonical core is authoritative for document execution. It invokes the stable `Execution` middleware through a private, per-invocation same-name Api whose diff --git a/packages/cli/tests/testing-execution-host.test.ts b/packages/cli/tests/testing-execution-host.test.ts index 43e909ae7..9359a7139 100644 --- a/packages/cli/tests/testing-execution-host.test.ts +++ b/packages/cli/tests/testing-execution-host.test.ts @@ -1103,12 +1103,15 @@ describe("execution initialization in isolated run children", () => { const project = yield* useProject({}); const api = new URL("../../core/api.ts", import.meta.url).href; const runtime = new URL("../../runtime/mod.ts", import.meta.url).href; + // ESM by extension, because the project is a bare directory with no + // `package.json`: a `.ts` plugin there loads as CommonJS under Node, and + // the dependencies core reaches through publish an import condition only. yield* writeTextFile( - join(project, "init-plugin.ts"), + join(project, "init-plugin.mjs"), [ `import { Execution, Plugin, Syntax } from ${JSON.stringify(api)};`, `import { cwd } from ${JSON.stringify(runtime)};`, - "const calls: string[] = [];", + "const calls = [];", "export default Plugin({ name: 'init-evidence', *install(request) {", " yield* Execution.around({ *init(_args, next) {", " calls.push(`${request.command}@${yield* cwd()}`);", @@ -1134,7 +1137,7 @@ describe("execution initialization in isolated run children", () => { "", ), ); - const result = yield* runCli(["test", "--plugin", "./init-plugin.ts", "README.md"], { + const result = yield* runCli(["test", "--plugin", "./init-plugin.mjs", "README.md"], { cwd: project, }).join(); if (result.code !== 0) { diff --git a/packages/core/api.ts b/packages/core/api.ts index 6e679974a..531aaec45 100644 --- a/packages/core/api.ts +++ b/packages/core/api.ts @@ -132,11 +132,11 @@ export type { Expansion } from "./src/expansion.ts"; /** * The execution a host composes around, and the requests its layers are handed. * - * `init` is the member a command host installs on: it establishes the live - * dependencies this execution runs on, once, inside the scope the invocation - * already owns. `execute` and `document` stay what they were — the - * request-policy seam and the whole-document seam — and their public defaults - * still refuse, because only canonical core completes a document. + * Three seams. `init` is the one a command host installs on: it establishes + * the live dependencies this execution runs on, once, inside the scope the + * invocation already owns. `execute` is the request-policy seam and `document` + * wraps the whole expansion; both of those defaults refuse, because only + * canonical core completes a document. */ export { Execution } from "./src/execute.ts"; export type { ExecutionApi } from "./src/execute.ts"; @@ -146,11 +146,10 @@ export type { DocumentRequest } from "./src/document-request.ts"; /** * The description in effect, as the thing a consumer composes on. * - * Here rather than only at the root because answering *what may be written - * here* is a consumer boundary: a host binds its own profile, and middleware - * may transform the symbols, the text or a named lookup. A description grants - * nothing — `./host`'s `useSyntax` is how a host supplies one, and admission - * remains the execution's. + * The same descriptor the root exports, reachable from here as well because a + * consumer installing middleware imports from `./api`. What the three + * properties answer, and what composing on them does and does not change, is + * in `src/syntax-api.ts`; `./host`'s `useSyntax` is how a host supplies one. */ export { Syntax } from "./src/syntax-api.ts"; export type { SyntaxApi } from "./src/syntax-api.ts"; diff --git a/packages/core/mod.ts b/packages/core/mod.ts index ebfa353a4..8e898209f 100644 --- a/packages/core/mod.ts +++ b/packages/core/mod.ts @@ -452,9 +452,9 @@ export type { FileSecretFinding } from "./src/secrets/files.ts"; export { scanSecrets, secretPolicy } from "./src/secrets/policy.ts"; export type { SecretPolicy } from "./src/secrets/policy.ts"; -// What a document may write here. The same descriptor `./api` exports — one +// The description in effect here. The same descriptor `./api` exports — one // stable name, so middleware composes across separately loaded copies — kept at -// the root as well because reading a description is ordinary work, not a -// consumer-boundary concern. +// the root as well because reading a description is ordinary work. What the +// properties answer is in `src/syntax-api.ts`. export { Syntax } from "./src/syntax-api.ts"; export type { SyntaxApi } from "./src/syntax-api.ts"; diff --git a/packages/core/src/execute.ts b/packages/core/src/execute.ts index 3ef92ce62..37785e884 100644 --- a/packages/core/src/execute.ts +++ b/packages/core/src/execute.ts @@ -3041,11 +3041,12 @@ export interface ExecutionInstallation { /** * Execution Api — a policy surface around document execution. * - * A handler is given an `ExecutionRequest`, not the execution. It may inspect - * the options, narrow or replace them with `withOptions()`, register an - * additive completion failure, install contextual behavior the document will - * inherit, refuse by throwing, and delegate. It returns nothing, and whatever - * it returns is ignored: only canonical execution completes a document. + * No handler is given the execution, and whatever a handler returns is + * ignored: only canonical execution completes a document. What a handler *is* + * given differs by member — `init` nothing, `execute` an `ExecutionRequest`, + * `document` a `DocumentRequest` — and each one below says what it may decide + * with what it was handed. Any of them may install contextual behavior the + * document will inherit, refuse by throwing, and delegate. */ export interface ExecutionApi { /** @@ -3069,6 +3070,12 @@ export interface ExecutionApi { * already captured. */ init(): Operation; + /** + * The policy seam for one execution request. + * + * A handler may inspect the options, narrow or replace them with + * `withOptions()`, register an additive completion failure, and delegate. + */ execute(request: ExecutionRequest): Operation; /** * The document's expansion, as `durableRun` runs it. diff --git a/packages/core/src/syntax-api.ts b/packages/core/src/syntax-api.ts index fa39f6593..84b4299ff 100644 --- a/packages/core/src/syntax-api.ts +++ b/packages/core/src/syntax-api.ts @@ -3,8 +3,7 @@ * * ``, `xmd syntax` and a Plan's authorship all ask the same three * questions — what are the symbols, what is the text, what do these names - * mean — and before this they asked three different things in three places. So - * they are one descriptor, dispatched by name, and a host, a trusted boundary + * mean — of one descriptor, dispatched by name, so a host, a trusted boundary * or ordinary middleware can answer, delegate or transform any of them. * * ## Why the properties are Operations @@ -35,8 +34,12 @@ * to shadow: it is reporting an admission, not offering a fallback, and it * ends with the scope that installed it. * - * Neither grants anything. A description that names a component does not admit - * it, and one that hides a component does not disable it. + * Neither grants anything, and neither takes anything away: a description that + * names a component does not admit it, and one that hides a component does not + * disable it. Component selection, invocation identity and durable recording + * stay canonical whatever the description says, which is why composing on this + * Api is ordinary rather than privileged work. The boundary itself is + * architecture.md's *The syntax symbols boundary*. */ import { type Api, createApi } from "@effectionx/context-api"; @@ -47,9 +50,14 @@ import { renderSyntaxMarkdown } from "./syntax-markdown.ts"; /** The three questions anything describing this site answers. */ export interface SyntaxApi { - /** What may be written here, structured. */ + /** The effective structured description of this site. */ readonly symbols: Operation; - /** The same answer as Markdown, which is what `xmd syntax` prints. */ + /** + * The effective text, which is what `xmd syntax` prints. + * + * Its *default* renders {@link SyntaxApi.symbols}. Middleware may answer with + * text of its own, so the two are not two views of one value by construction. + */ readonly text: Operation; /** These components' metadata and long-form documentation. */ documentation(names: readonly string[]): Operation; diff --git a/packages/core/src/syntax-reference.ts b/packages/core/src/syntax-reference.ts index 77e448114..73a9c33ed 100644 --- a/packages/core/src/syntax-reference.ts +++ b/packages/core/src/syntax-reference.ts @@ -50,11 +50,11 @@ import type { ComponentOrigin, ComponentRegistry } from "./types.ts"; * The default description for the segments being expanded. * * Held by the execution and handed to core's own expansion by value, beside the - * component resolution and the identity domains. It is not itself a Context: a - * context resolves by name, and a name is not a secret, so a document could - * build one and answer for the vocabulary it is shown. What a document *can* - * reach is the public `Syntax` Api in front of it, which answers with text and - * grants nothing. + * component resolution and the identity domains: this is what the public + * `Syntax` Api answers from when nothing nearer does, not a second place where + * anything is decided. Composing in front of it is ordinary and expected. What + * protection keeps canonical is component selection, invocation identity and + * durable recording — none of which a description reaches. */ export interface SyntaxReference { /** The symbols this site describes, structured. */ @@ -75,8 +75,8 @@ export interface SyntaxReference { * author needs or imply a permission they do not have. * * Availability is decided against the *effective* symbols, read once through - * the Api, so a host or boundary that customized what may run here is the - * same thing that decides what this annotation says. + * the Api, so whatever customized the descriptive view here is the same thing + * that decides what this annotation says. */ documentation(names: readonly string[]): Operation; /** diff --git a/specs/executable-mdx-spec.md b/specs/executable-mdx-spec.md index ff6c5a4ae..09dd1bb9a 100644 --- a/specs/executable-mdx-spec.md +++ b/specs/executable-mdx-spec.md @@ -2784,9 +2784,10 @@ The documentation is read from the owning package through the direct filesystem, not through `API.Fs` or the document-facing `Files` provider. Both of those are middleware a running document can compose around, and a document that could answer the read would decide what the product says about itself. What that read -produces is the *captured default* catalog. Live `Syntax` middleware composes in -front of it and may transform or replace the answer an occurrence receives — -which is a composition the host assembled, not a read a document answered. +produces is the *captured default* catalog, and canonical collection owns it. +The effective answer an occurrence receives is a separate thing: live `Syntax` +middleware composes in front of that catalog and may transform or replace what +it produced. **Documentation composes with the components it describes.** A package's bootstrap installs its registrations and its documentation together — the @@ -2858,10 +2859,13 @@ this is one. Default bare `` reports the effective contextual symbols at this site. The named form selects from the **enclosing authoring symbols** and states, per entry, -`**Available in this evaluation:** yes` or `no`. At a root the two are one set -of symbols and everything selected is available; a trusted evaluation boundary that -narrows execution keeps the enclosing reference, so a nested author can be told -how a component works where they may not run one — and is told which it is. +`**Available in this evaluation:** yes` or `no`. Uncustomized at a root the two +are one set of symbols and everything selected is available; a trusted +evaluation boundary that narrows execution keeps the enclosing reference, so a +nested author can be told how a component works where they may not run one — +and is told which it is. Descriptive middleware that filters the effective +symbols separates the two at a root as well, and the per-entry label is what +says so. Neither input carries definitions, import witnesses, invocation capabilities, providers, registrations or any other execution capability. @@ -2901,35 +2905,29 @@ interface SyntaxApi { } ``` -Callers yield `Syntax.operations.symbols` and `.text` without parentheses. -Property middleware and its continuation return Operations. Defaults are -reusable lazy Operations, not stored consumable generator instances. Default -text renders effective contextual symbols through `renderSyntaxMarkdown()`. -Text middleware may then transform or replace that answer. Synthetic answers -that do not delegate perform no default inspection. - -The execution's private reference supplies its root fallback. A host can call +Callers yield `Syntax.operations.symbols` and `.text` without parentheses; +property middleware and its continuation return Operations. A host binds a +complete descriptive profile of its own with `useSyntax(source: () => Operation): Operation` from -`@executablemd/core/host` to bind a different complete descriptive profile. The -helper captures and snapshots existing scoped documentation contributions and -installs ordinary symbols/documentation providers at `min` without reading the -source. `ExecutionInstallation` carries no symbols provider or descriptive -provider count rule. Ordinary middleware composition decides the live answer. - -Default named lookup retains its enclosing catalog for metadata and long-form -text and reads effective symbols once for availability. Its authoring basis is -resolved once per occurrence and reused by that occurrence's default symbols -provider. Later occurrences resolve independently. Filtering symbols affects -bare text and named availability, while an omitted enclosing component remains -documentable. Text-only customization changes neither symbols nor named lookup. -A synthetic named answer skips the catalog entirely. - -Generated producer and body scopes narrow symbols only, retaining the enclosing -named-documentation owner. Their descriptive bindings end with those scopes. -Standalone Plan and CLI inspection bind the target run's declarative profile -through `useSyntax`; embedded Plan inherits its enclosing profile. CLI -inspection starts no execution initialization or operational provider merely to -print. +`@executablemd/core/host`, which installs ordinary symbols and documentation +providers without reading the source. `ExecutionInstallation` carries no symbols +provider and no descriptive provider count rule: ordinary middleware +composition decides the live answer. How the defaults, the execution's own +fallback and a narrowing boundary rank against each other is +[the syntax symbols boundary](../architecture.md#the-syntax-symbols-boundary). + +Default text renders the effective contextual symbols, and text middleware may +transform or replace that answer. Default named lookup retains its enclosing +catalog for metadata and long-form text and reads effective symbols once for +availability, so filtering symbols affects bare text and named availability +while an omitted enclosing component remains documentable; text-only +customization changes neither symbols nor named lookup, and a synthetic answer +that does not delegate skips the default work entirely. Generated producer and +body scopes narrow symbols only, retaining the enclosing named-documentation +owner, and their descriptive bindings end with those scopes. Standalone Plan +and CLI inspection bind the target run's declarative profile through +`useSyntax`; embedded Plan inherits its enclosing profile. CLI inspection +starts no execution initialization or operational provider merely to print. Seeing a component in the symbols grants nothing. It neither registers, resolves nor authorizes that component: what a name means is still this section's @@ -9904,11 +9902,13 @@ workflow and returns a `DocumentExecution` handle. Options: `Execution.init(): Operation` runs once in the existing invocation-owned scope. Its default is no-op and supplies no Files provider. A host registers ordinary middleware, installs its live dependencies there and delegates with -`yield* next()`. Core copies trusted admissions, declarations, preparations, -provider/install callbacks and the generated profile's host inputs and arms -revocation before calling init. Init finishes before installation callbacks, -documentation capture and public `Execution.execute` policy. The generated -profile's provider-bound definitions are resolved during document assembly. +`yield* next()`. Where that sits in the order core already guarantees — after +the trusted host contributions are copied and their revocation armed, before +installation callbacks, documentation capture and public `Execution.execute` +policy — is +[capability-backed execution](../architecture.md#capability-backed-execution). +The generated profile's provider-bound definitions are resolved during document +assembly. Failure or cancellation during init prevents execution policy, root import and body and waits for all acquired cleanup before settlement. Concurrent @@ -9920,9 +9920,9 @@ while completed durable operations and Syntax answers replay passively. `runDocument`, REPL `runEntry` and isolated ordinary `runProfileChild` all reach this core initialization point. Isolated children explicitly reinstall selected host/Plugin middleware after cwd assembly and inherit no arbitrary parent -handlers. REPL state is per entry below the existing Agent attachment; init does -not replace that attachment, durable preparation or the journal owner. The -REPL expansion controller does not pause at initialization. +handlers. REPL state is per entry below the existing Agent attachment, which +init does not replace, and so are durable preparation and the journal owner. +The REPL expansion controller does not pause at initialization. #### The host's repository provider