diff --git a/architecture.md b/architecture.md index c3252021b..29d3c9661 100644 --- a/architecture.md +++ b/architecture.md @@ -6450,13 +6450,40 @@ printed location, those four answer from what they produced, and the write after the question happens for the first time. Four operations are retained and no others: a `` text read, a `` -write, a ``, and the directory a `` ensures, each in the -ambient checkout the entry was started in. A read is retained because a document -binds its text and a continuation has to offer the same binding; a glob, a path -check and a temporary directory bind nothing a continuation must answer again, -and are left alone. A call outside that checkout is ordinary filesystem -behavior, recorded by nothing — a record would place the call in a checkout it -is not in — so it runs again during a partial continuation. +write, a ``, and the directory a `` ensures, each in a +checkout this entry is working in — the ambient one it was started in, or a +managed one it selected. A read is retained because a document binds its text +and a continuation has to offer the same binding; a glob, a path check and a +temporary directory bind nothing a continuation must answer again, and are left +alone. A call outside every such checkout is ordinary filesystem behavior, +recorded by nothing — a record would place the call in a checkout it is not in — +so it runs again during a partial continuation. A managed placement is looked +for first and the most specific one wins; the ambient route answers everything no +placement covers, so an entry that selects nothing works exactly as it did +before. + +Selection is retained on the same terms. `` and `` call +the *ordinary* managed selector — the same creation, compatible-reuse, conflict +and interrupted-adoption policy `xmd run` gets — and what the selector answered +is recorded as a small routing record: the canonical managed root, the checkout +root, the common Git directory, the credential-free repository identity, the +display name and the owner checkout. A root-level `` records the +Repository it belongs to as its own answer, so a continuation learns that +without asking Git. Nothing else is retained: no origin, no publication +evidence, no object source, no Git session, no authentication, no lock and no +opaque selection identifier. + +Answering a completed selection calls no selector, clones nothing, adds no +worktree, reads no sidecar, opens no credential and takes no lock. The parsed +facts are held to the call they stand at with pure arithmetic — the result +describes the request it is named by, and the checkout it names is the one this +request's own slot would hold under the root the result itself records — and the +root is never recanonicalized against today's disk to check a historical answer. +The entry then registers the placement privately and mints a fresh selection for +this execution; rebuilding that membership is passive and acquires nothing, and +an editable Repository context cannot redirect native work because every +operation authenticates the selection it was handed against that private state +first. Only live new work needs the environment. Answering a completed operation performs no path canonicalization, no ambient discovery, no lease acquisition, @@ -6471,11 +6498,15 @@ an entry was working is not permission to work there. What does the retaining is established by core initialization, once per executing entry, inside the scope the invocation already owns — registered by the selected REPL host rather than by the ordinary run Plugin, because an entry -installs no managed root, credential helper or Git host. Everything it acquires -belongs to that execution: the Git session, its isolation directory, the ambient -discovery and the lease all end when the entry does. A passive view — a settled -history, a frozen position, a refused route — initializes nothing, discovers no -checkout and takes no lease. +installs the Repository selection handlers and the file recorder and nothing +else the ordinary provider would bring with it. The managed root and the +credential-helper assembly are copied where that host is installed, before any +entry runs, so what an entry initializes with is what the command selected +rather than a field somebody changed afterwards. Everything it acquires belongs +to that execution: the Git session, its isolation directory, the managed root, +the ambient discovery and every hold all end when the entry does. A passive view +— a settled history, a frozen position, a refused route — initializes nothing, +discovers no checkout, creates no managed root and takes no hold. Recording is ordinary ancestor-max `Files` middleware over the one document filesystem, so what is kept is the result the whole composition answered with, @@ -6533,10 +6564,44 @@ and that refusal is never saved as a completed result: a lease is busy only while somebody else is working, and recording "refused" would make a temporary condition the permanent answer every later continuation replays. -There is no managed checkout here, no `` or `` selection, -no local Git operation and nothing external: those components stay declared -where they already are and refuse through their own unchanged missing-provider -defaults. +A managed slot has its own hold, and the two answer different questions. The +**placement** hold says one entry at a time owns a managed checkout: a live +selection takes it before it inspects or creates anything, and it lasts until +the entry is torn down so a later `` inside the same entry is still +protected. A completed replay takes none, and the first genuinely new file +operation inside a restored placement — a read included — reacquires that slot +then, without rerunning the selector or looking at what the checkout holds. A +fresh selection of a slot another entry holds keeps the ordinary immediate +refusal, and that refusal is never written down: contention is decided before +the recorded selection begins. Where the reacquisition happens inside a file +operation, the condition travels as a fatal stale-state failure rather than as +that operation's recorded outcome. The **mutation** hold above is the other one: +only a new `` selector and a new write, deletion or directory take it, +the Worktree before its selector because a linked checkout writes administration +every checkout of that repository shares, and when both are needed the slot +comes first. + +What the selector itself acquires is shorter-lived than the entry. The +authentication a clone opens belongs to that one clone, and its scope is torn +down and joined before the placement is published — so no record and no body +work precedes that cleanup, and a cleanup that fails takes the ordinary fatal +path rather than leaving an acknowledged selection behind. The entry keeps what +is entry-owned: the Git session, its isolation directory and both holds. + +A selection's durable append completes before its path is bound or its body +expands. Native work may finish without being acknowledged: on cancellation +before the append no completion is manufactured, and the continuation runs that +selection live, where the existing partial-creation policy decides what the slot +now holds — adoption only under the stricter proof, refusal otherwise, and never +deletion or repair. An acknowledged placement never enters that path again. A +failed append fences later work with the first persistence failure, with no +success result and no compensating Close. + +There is no local Git operation here and nothing external: ``, +``, ``, ``, `` and `` stay +declared where they already are and refuse through their own unchanged +missing-provider defaults. Nothing inventories the managed root, snapshots a +checkout's contents or restores one. ### What the screen teaches, and what leaving it costs @@ -6912,7 +6977,8 @@ Status is measured against main. | `GitQuery.resolve()` | verifies and resolves one Git revision expression contextually [Interface and evidence](specs/workflow-spec.md#7-the-git-capability--moved). | built on main | | workflow run storage | creates or compatibly finds one run by public run ID, retains its identity, state, document executions and filtered journal, and validates immutable Workspace roots through one provider-owned connection entry [Invariant](#workflow-run-storage); [Interface and evidence](specs/workflow-spec.md#9-retained-run-storage). | built on the #365 stack; the CLI lifecycle reaches it on the #366 stack | | caller-owned storage transaction | publishes several changes, including journal events, in one transaction nothing else enlists in [Invariant](#one-database-at-a-time-and-a-transaction-a-caller-can-hold). | built on main | -| REPL entry retained file work | one executing REPL entry's own recording, established by core initialization in the scope the invocation owns and ending with it: it retains the `` reads, `` writes, `` deletions and `` ensures that land inside the ambient checkout the entry was started in, authored and admitted-generated alike, because both cross the one document filesystem. A completed operation answers from its saved result without live path canonicalization, ambient discovery, a lease, a permission lookup or a filesystem call — so only *new* work needs the environment, a continuation still binds the text its read answered with while the file says something else, and a new entry reading that path sees what is there now. A record's name is the digest of the record version, the operation kind and the complete received request, so two equal requests in one entry are one name at two ordered positions and the durable engine's positional matching tells them apart; equal work in two entries is the same name, and each entry executing against its own stream segment is what keeps those apart. Recording is ordinary ancestor-max `Files` middleware, so a provider beneath that answers directly or transforms and delegates is kept as provider behavior while the received request stays the operation's identity. A shared Git directory admits one live *mutating* entry at a time, held lazily for as long as the execution lasts and refused rather than queued; no read acquires it, so a new read proceeds while another entry holds it. There is no managed checkout, no `` or `` selection, no local Git operation and nothing external here; work outside the checkout stays ordinary and unrecorded, and a passive reading initializes nothing [Reference](#what-one-entry-retains-about-the-checkout-it-is-standing-in). | built on the #181 stack | +| REPL entry retained file work | one executing REPL entry's own recording, established by core initialization in the scope the invocation owns and ending with it: it retains the `` reads, `` writes, `` deletions and `` ensures that land inside a checkout the entry is working in — the ambient one it was started in, or a managed one it selected — authored and admitted-generated alike, because both cross the one document filesystem. A completed operation answers from its saved result without live path canonicalization, ambient discovery, a lease, a permission lookup or a filesystem call — so only *new* work needs the environment, a continuation still binds the text its read answered with while the file says something else, and a new entry reading that path sees what is there now. A record's name is the digest of the record version, the operation kind and the complete received request, so two equal requests in one entry are one name at two ordered positions and the durable engine's positional matching tells them apart; equal work in two entries is the same name, and each entry executing against its own stream segment is what keeps those apart. Recording is ordinary ancestor-max `Files` middleware, so a provider beneath that answers directly or transforms and delegates is kept as provider behavior while the received request stays the operation's identity. A shared Git directory admits one live *mutating* entry at a time, held lazily for as long as the execution lasts and refused rather than queued; no read acquires it, so a new read proceeds while another entry holds it. There is no local Git operation and nothing external here; work outside every checkout the entry is working in stays ordinary and unrecorded, and a passive reading initializes nothing [Reference](#what-one-entry-retains-about-the-checkout-it-is-standing-in). | built on the #181 stack | +| REPL entry retained selection | one executing REPL entry's own `` and `` selections, recorded beside its file work as small routing records: the canonical managed root, the checkout root, the common Git directory, the credential-free repository identity, the display name and the owner checkout, with a root-level ``'s ambient owner recorded as its own answer. The selector is the ordinary managed one, with its own creation, compatible-reuse, conflict and interrupted-adoption policy; what this adds is that a completed selection answers from its record, calling no selector and reaching no clone, `worktree add`, sidecar, credential or lock — so a continuation works in a checkout somebody has since changed or removed, while only new work meets today's filesystem. Parsed facts are held to the call they stand at with pure slot arithmetic against the root the record itself carries, registered in entry-private state and minted as a fresh selection, so a copied or edited Repository value can misname a target and never reach one. One entry at a time *owns* a managed slot, held until teardown and reacquired lazily by the first new operation inside a restored placement; that is a different hold from the shared Git directory's mutation lease, which only a new `` selector and a new mutation take. A selection is acknowledged before its path is bound, and native work that was never acknowledged runs live again under the ordinary partial-creation policy [Reference](#what-one-entry-retains-about-the-checkout-it-is-standing-in). | built on the #181 stack | | live durable-operation coordinator | explicitly coordinates structured live execution with existing Yield publication while leaving replay and callback effects unchanged [Invariant](#live-effect-coordination). | built on the #365 stack | | Workspace coordination API | fails closed by default; replaceable context routes only a one-use provider selection, while the selected provider directly invokes an execution-owned credentialed capability for execution, publication and failure activation [Invariant](#live-effect-coordination). | built on the #365 stack; the Deno provider installs an adapter-private atomic handler | | shared Git-host reconciliation | Reconciles Git push and pull-request upsert through one request-only boundary whose private terminal alone settles the retained outcome. [Invariant](#external-effects-and-git-host-mutations); [Invariant](#issue-effects). | Built on #297; Push (#370) and PullRequest (#295) consume it. Issue (#296) has its own middleware boundary. | diff --git a/packages/cli/src/compiled-repl.ts b/packages/cli/src/compiled-repl.ts index 8e3fff9d3..03d9d1dfd 100644 --- a/packages/cli/src/compiled-repl.ts +++ b/packages/cli/src/compiled-repl.ts @@ -14,11 +14,12 @@ import { homedir } from "node:os"; import { sep } from "node:path"; import process from "node:process"; import type { Operation } from "effection"; +import type { HelperAssembly } from "@executablemd/git/credential-helper"; import { installReplHost, platformDataRoot } from "./repl-assembly.ts"; import { useCompiledReplTerminal } from "./compiled-repl-terminal.ts"; import { useDenoReplRepositories } from "./deno-repositories.ts"; -export function* useCompiledRepl(): Operation { +export function* useCompiledRepl(helper: HelperAssembly): Operation { yield* installReplHost({ dataRoot: () => platformDataRoot(process.platform, homedir(), process.env, sep), // Sixteen random bytes as hex: URL-safe, opaque, and not derived from a @@ -28,11 +29,12 @@ export function* useCompiledRepl(): Operation { appendRecord: (path, record) => appendFile(path, record), }); // What an entry's execution initializes: the recording of the File and Dir - // work it does in the checkout the person opened the session in. Registered - // here, in the runtime scope the selected REPL host owns, because only a - // runtime-named module may name the native adapter — and registering it - // acquires nothing, so a session that only reads history discovers no - // checkout and takes no lease. - yield* useDenoReplRepositories(); + // work it does, and of the managed Repository and Worktree selections that + // say where it does it. Registered here, in the runtime scope the selected + // REPL host owns, because only a runtime-named module may name the native + // adapter and this runtime's own credential-helper assembly — and registering + // it acquires nothing, so a session that only reads history discovers no + // checkout, creates no managed root and takes no lease. + yield* useDenoReplRepositories(helper); yield* useCompiledReplTerminal(); } diff --git a/packages/cli/src/compiled.ts b/packages/cli/src/compiled.ts index dcccf64b9..85b7bd98e 100644 --- a/packages/cli/src/compiled.ts +++ b/packages/cli/src/compiled.ts @@ -98,7 +98,7 @@ if (isCredentialHelperMode(process.argv.slice(2))) { importPluginModule, () => useDenoWorkflowHost(HELPER), useMachineSessions(), - useCompiledRepl, + () => useCompiledRepl(HELPER), ); }); } diff --git a/packages/cli/src/deno-repl.ts b/packages/cli/src/deno-repl.ts index ad4f6d8a6..e8e266af6 100644 --- a/packages/cli/src/deno-repl.ts +++ b/packages/cli/src/deno-repl.ts @@ -14,11 +14,12 @@ import { homedir } from "node:os"; import { sep } from "node:path"; import process from "node:process"; import type { Operation } from "effection"; +import type { HelperAssembly } from "@executablemd/git/credential-helper"; import { installReplHost, platformDataRoot } from "./repl-assembly.ts"; import { useDenoReplTerminal } from "./deno-repl-terminal.ts"; import { useDenoReplRepositories } from "./deno-repositories.ts"; -export function* useDenoRepl(): Operation { +export function* useDenoRepl(helper: HelperAssembly): Operation { yield* installReplHost({ dataRoot: () => platformDataRoot(process.platform, homedir(), process.env, sep), // Sixteen random bytes as hex: URL-safe, opaque, and not derived from a @@ -28,11 +29,12 @@ export function* useDenoRepl(): Operation { appendRecord: (path, record) => appendFile(path, record), }); // What an entry's execution initializes: the recording of the File and Dir - // work it does in the checkout the person opened the session in. Registered - // here, in the runtime scope the selected REPL host owns, because only a - // runtime-named module may name the native adapter — and registering it - // acquires nothing, so a session that only reads history discovers no - // checkout and takes no lease. - yield* useDenoReplRepositories(); + // work it does, and of the managed Repository and Worktree selections that + // say where it does it. Registered here, in the runtime scope the selected + // REPL host owns, because only a runtime-named module may name the native + // adapter and this runtime's own credential-helper assembly — and registering + // it acquires nothing, so a session that only reads history discovers no + // checkout, creates no managed root and takes no lease. + yield* useDenoReplRepositories(helper); yield* useDenoReplTerminal(); } diff --git a/packages/cli/src/deno-repositories.ts b/packages/cli/src/deno-repositories.ts index cc003863d..078767b88 100644 --- a/packages/cli/src/deno-repositories.ts +++ b/packages/cli/src/deno-repositories.ts @@ -124,19 +124,31 @@ function snapshot(helper: HelperAssembly): HelperAssembly { } /** - * What a REPL entry's execution initializes: recording, and nothing else. + * What a REPL entry's execution initializes: recording and selection, and + * nothing else. * * Registered by the selected REPL host rather than by the bundled Plugin, - * because an entry is not an ordinary run: it retains the file work it does in - * the checkout the person is standing in, and installs no managed root, - * credential helper or Git host. Each executing entry initializes its own, in + * because an entry is not an ordinary run: it retains the file work it does and + * the managed checkouts it selects to do that work in, and installs no Git host, + * publication or external worker. Each executing entry initializes its own, in * the scope below its Agent attachment, so a passive view — a settled history, - * a frozen position — initializes nothing and takes no lease. + * a frozen position — initializes nothing, creates no managed root and takes no + * lease. + * + * The root and the helper are copied here, at registration, rather than held as + * the caller's object. What an entry initializes with has to be what this host + * selected: a caller that kept a reference and changed a field afterwards would + * be configuring every later entry from outside the selection the command + * already made. */ -export function* useDenoReplRepositories(): Operation { +export function* useDenoReplRepositories( + helper: HelperAssembly, + root: string = DEFAULT_REPOSITORY_ROOT, +): Operation { + const configured = snapshot(helper); yield* Execution.around({ *init(_args, next): Operation { - yield* useReplRepositoryRecording(); + yield* useReplRepositoryRecording({ helper: configured, root }); yield* next(); }, }); diff --git a/packages/cli/src/deno.ts b/packages/cli/src/deno.ts index 742781099..9f8c476a7 100644 --- a/packages/cli/src/deno.ts +++ b/packages/cli/src/deno.ts @@ -117,7 +117,7 @@ if (isCredentialHelperMode(process.argv.slice(2))) { importPluginModule, () => useDenoWorkflowHost(HELPER), useMachineSessions(), - useDenoRepl, + () => useDenoRepl(HELPER), ); }); } diff --git a/packages/cli/tests/fixtures/repl/issue-881/controlled-host.ts b/packages/cli/tests/fixtures/repl/issue-881/controlled-host.ts index 9be9a0719..b18985ae3 100644 --- a/packages/cli/tests/fixtures/repl/issue-881/controlled-host.ts +++ b/packages/cli/tests/fixtures/repl/issue-881/controlled-host.ts @@ -87,7 +87,9 @@ await main(function* (args) { { at: "min" }, ); yield* useHostFiles(); - yield* useDenoRepl(); + // The assembly this host selects, stated here as an entrypoint states it. + // Inert: this fixture selects no repository, so nothing ever reaches it. + yield* useDenoRepl({ runtime: "source", platform: "unix", execPath: process.execPath }); const ran = yield* runReplProgram({ profile: { diff --git a/packages/cli/tests/repl-repositories-pty.test.ts b/packages/cli/tests/repl-repositories-pty.test.ts index 3559805cd..4a923504b 100644 --- a/packages/cli/tests/repl-repositories-pty.test.ts +++ b/packages/cli/tests/repl-repositories-pty.test.ts @@ -24,7 +24,12 @@ import { describe, it } from "@executablemd/test-support/bdd"; import { cliRuntime, cliShellCommand } from "@executablemd/test-support/launch"; import type { Operation } from "effection"; -import { replFilesJourney, replOrdinaryFileWork } from "./support/repl-files-journey.ts"; +import { + replFilesJourney, + replOrdinaryFileWork, + replRepositoryJourney, + replUnsupportedSelection, +} from "./support/repl-files-journey.ts"; const JOURNEY = cliRuntime() === "deno"; @@ -42,4 +47,21 @@ describe("a REPL entry's files, across leaving and coming back", () => { yield* JOURNEY ? replFilesJourney(sourceCommand) : replOrdinaryFileWork(sourceCommand); }, ); + + it( + JOURNEY + ? "E1: a managed Repository is selected once, and answers again on reopening" + : "E8: this runtime refuses a managed Repository before it reaches a clone", + function* (): Operation { + yield* JOURNEY + ? replRepositoryJourney(sourceCommand, "repository") + : replUnsupportedSelection(sourceCommand); + }, + ); + + if (JOURNEY) { + it("E1: a managed Worktree's captured path routes a Dir, and answers again", function* (): Operation { + yield* replRepositoryJourney(sourceCommand, "worktree"); + }); + } }); diff --git a/packages/cli/tests/repl-repositories.test.ts b/packages/cli/tests/repl-repositories.test.ts index 4cff74de6..a5655c590 100644 --- a/packages/cli/tests/repl-repositories.test.ts +++ b/packages/cli/tests/repl-repositories.test.ts @@ -36,10 +36,15 @@ import { REPL_REPOSITORY_LOCAL, REPL_REPOSITORY_SELECTION, } from "../../git/src/retained-repository/records.ts"; +import { + parseReplPlacementRecord, + REPL_REPOSITORY_PLACEMENT, +} from "../../git/src/retained-repository/placement-records.ts"; import { Execution } from "@executablemd/core/api"; import type { ExecutionInstallation } from "@executablemd/core/host"; import { ordinaryGeneratedSelection } from "../src/evaluation-profile.ts"; import { useDenoReplRepositories } from "../src/deno-repositories.ts"; +import type { HelperAssembly } from "../../git/src/deno/composition/credential-helper.ts"; import { gitPlugin } from "../../git/src/plugin.ts"; import { ReplHost } from "../src/repl/host.ts"; import { openReplSession, submitReplEntry } from "../src/repl/session.ts"; @@ -137,14 +142,47 @@ interface Counted { * an entry that executes initializes once, and a view that only reads a history * initializes not at all. */ -function* useCountedRecording(counted: Counted): Operation { +function* useCountedRecording(counted: Counted, root: string): Operation { yield* Execution.around({ *init(_args, next): Operation { counted.initialized += 1; yield* next(); }, }); - yield* useDenoReplRepositories(); + yield* useDenoReplRepositories(SUITE_HELPER, root); +} + +/** + * The credential-helper assembly this suite's host selects. + * + * Inert, and never invoked: every locator these rows select is a local path, so + * the transport carries no identity and no broker is asked for one. + */ +const SUITE_HELPER: HelperAssembly = Object.freeze({ + runtime: "source", + platform: "unix", + execPath: "/nonexistent/xmd", +}); + +/** Where this fixture's managed checkouts live. */ +function managedRoot(fixture: Fixture): string { + return join(fixture.root, "repositories"); +} + +/** + * A source repository with a commit in it, for the rows that select one. + * + * Made inside the row rather than in the shared fixture, so the rows about the + * checkout the entry was opened in keep the exact repository they have had. + */ +function* useOrigin(fixture: Fixture): Operation { + const origin = join(fixture.root, "origin"); + const git = gitSession(denoRepositoryHost(), fixture.root); + yield* git.run(["init", "-b", "main", origin], fixture.root); + yield* until(writeFile(join(origin, "README.md"), "origin\n")); + yield* git.run(["add", "README.md"], origin); + yield* git.run(["commit", "-m", "first"], origin); + return origin; } function execution(events: readonly DurableEvent[] = []): ReplExecution { @@ -205,7 +243,7 @@ function* submit( // Absent for the row about a host that registers none: the entry then runs // on the ordinary Files provider with nothing retained. if (counted !== undefined) { - yield* useCountedRecording(counted); + yield* useCountedRecording(counted, managedRoot(fixture)); } const session = opened( yield* submitReplEntry({ @@ -239,7 +277,7 @@ describe("how a REPL entry gets its repository behavior", () => { yield* useHostFiles(); yield* useWorkingDirectory(fixture.checkout); const installations = yield* replInstallations(); - yield* useCountedRecording(counted); + yield* useCountedRecording(counted, managedRoot(fixture)); const session = opened( yield* submitReplEntry({ execution: run, @@ -293,7 +331,7 @@ describe("how a REPL entry gets its repository behavior", () => { // journal without delegating this far. { at: "min" }, ); - yield* useCountedRecording(again); + yield* useCountedRecording(again, managedRoot(fixture)); const session = opened( yield* openReplSession({ execution: { id: "execution-1", stream: new InMemoryStream([...events]) }, @@ -335,6 +373,49 @@ describe("how a REPL entry gets its repository behavior", () => { expect(recordsOf(events, REPL_REPOSITORY_LOCAL)).toEqual([]); }); + it("E7: shares the recording with a generated File inside a managed Repository", function* () { + const fixture = yield* useFixture(); + const origin = yield* useOrigin(fixture); + const counted: Counted = { initialized: 0 }; + const run = execution(); + + // An admitted generated `` inside a managed Repository body, and two + // generated selections beside it. The fragment's write crosses the same one + // Files route an authored one does; the selections reach the generated + // ceiling before any provider is asked. + const source = [ + ``, + `from a fragment\n')}} allow={["write"]} />`, + `\n`)}} allow={["write"]} />`, + `\n')}} allow={["write"]} />`, + `\n\n')}} allow={["write"]} />`, + "", + ].join("\n"); + + const events = yield* submit(fixture, source, run, counted); + + // One placement: the authored Repository. The generated Repository and + // Worktree were never admitted, so neither reached a selector. + const selected = recordsOf(events, REPL_REPOSITORY_PLACEMENT); + expect(selected.length).toBe(1); + const placement = parseReplPlacementRecord( + selected[0]?.type === "yield" && selected[0].result.status === "ok" + ? selected[0].result.value + : undefined, + ); + const checkout = placement?.outcome.ok === true ? placement.outcome.value.checkoutRoot : ""; + expect(checkout).not.toBe(""); + + // The fragment's write happened in the managed checkout and is recorded + // there, exactly as an authored one would be. + expect(yield* readTextFile(join(checkout, "generated.md"))).toBe("from a fragment"); + expect(recordsOf(events, REPL_REPOSITORY_LOCAL).length).toBe(1); + // And the generated `` made nothing, in the managed checkout or + // anywhere else. + expect(yield* exists(join(checkout, "made"))).toBe(false); + expect(yield* exists(join(fixture.checkout, "made"))).toBe(false); + }); + it("E7: keeps two entries' equal records in their own segments", function* () { const fixture = yield* useFixture(); const counted: Counted = { initialized: 0 }; @@ -380,7 +461,7 @@ describe("how a REPL entry gets its repository behavior", () => { yield* useHostFiles(); yield* useWorkingDirectory(fixture.checkout); const installations = yield* replInstallations(); - yield* useCountedRecording(counted); + yield* useCountedRecording(counted, managedRoot(fixture)); const session = opened( yield* submitReplEntry({ execution: run, @@ -423,7 +504,7 @@ describe("how a REPL entry gets its repository behavior", () => { }, { at: "min" }, ); - yield* useCountedRecording(again); + yield* useCountedRecording(again, managedRoot(fixture)); const session = opened( yield* openReplSession({ execution: continued, @@ -469,7 +550,7 @@ describe("how a REPL entry gets its repository behavior", () => { yield* useHostFiles(); yield* useWorkingDirectory(fixture.checkout); const installations = yield* replInstallations(); - yield* useCountedRecording(reading); + yield* useCountedRecording(reading, managedRoot(fixture)); const session = opened( yield* openReplSession({ execution: { id: "execution-1", stream: new InMemoryStream([...events]) }, @@ -498,7 +579,7 @@ describe("how a REPL entry gets its repository behavior", () => { yield* useHostFiles(); yield* useWorkingDirectory(fixture.checkout); const installations = yield* replInstallations(); - yield* useCountedRecording(counted); + yield* useCountedRecording(counted, managedRoot(fixture)); const session = opened( yield* submitReplEntry({ execution: run, @@ -532,7 +613,7 @@ describe("how a REPL entry gets its repository behavior", () => { yield* useHostFiles(); yield* useWorkingDirectory(fixture.checkout); const installations = yield* replInstallations(); - yield* useCountedRecording(again); + yield* useCountedRecording(again, managedRoot(fixture)); try { const session = opened( yield* openReplSession({ diff --git a/packages/cli/tests/support/repl-files-journey.ts b/packages/cli/tests/support/repl-files-journey.ts index 13611ced8..414c9f0f5 100644 --- a/packages/cli/tests/support/repl-files-journey.ts +++ b/packages/cli/tests/support/repl-files-journey.ts @@ -23,7 +23,7 @@ import { useTempDirectory } from "@executablemd/test-support/temp"; import { type Operation, scoped, sleep, until } from "effection"; import { exists, readTextFile, writeTextFile } from "@effectionx/fs"; import { execFile } from "node:child_process"; -import { mkdir } from "node:fs/promises"; +import { mkdir, readdir } from "node:fs/promises"; import { join } from "node:path"; import process from "node:process"; import { allocate, showing } from "./pty.ts"; @@ -293,6 +293,279 @@ export function replFilesJourney(repl: ReplCommand): Operation { }); } +/** + * A source repository with one commit, for the managed journeys to clone. + * + * Seeded with a plain `git` for the same reason the checkout above is: the + * fixture has to be the same on every runtime this suite starts on. + */ +function* seedOrigin(home: string): Operation { + const origin = join(home, "origin"); + yield* until(mkdir(origin, { recursive: true })); + yield* git(home, ["init", "-b", "main", origin]); + yield* writeTextFile(join(origin, "README.md"), "origin\n"); + yield* git(origin, ["add", "README.md"], home); + yield* git(origin, ["commit", "-m", "first"], home); + return origin; +} + +/** One `git` command, in this journey's own isolated home. */ +function git(cwd: string, args: readonly string[], home = cwd): Operation { + return until( + new Promise((resolve, reject) => { + execFile( + "git", + [...args], + { + cwd, + env: { + ...process.env, + HOME: home, + GIT_CONFIG_GLOBAL: "/dev/null", + GIT_AUTHOR_NAME: "Journey", + GIT_AUTHOR_EMAIL: "journey@example.test", + GIT_COMMITTER_NAME: "Journey", + GIT_COMMITTER_EMAIL: "journey@example.test", + }, + }, + (error) => (error === null ? resolve() : reject(error)), + ); + }), + ); +} + +/** + * The one managed checkout this journey's entry created, once it is there. + * + * Found rather than predicted. A slot is a digest of the whole identity under a + * root this journey never names — the command decides where + * `~/.xmd/repositories` is from the home it was given — so looking for the one + * checkout beneath that layout is what asks the question the person asks: + * where did my Repository end up. + */ +function* managedCheckout(home: string, kind: "repositories" | "worktrees"): Operation { + const base = join(home, ".xmd", "repositories", kind); + const depth = kind === "repositories" ? 1 : 2; + const deadline = Date.now() + ACT_MS; + while (true) { + const found = yield* slotsBeneath(base, depth); + if (found.length === 1 && (yield* exists(join(found[0] ?? "", "checkout")))) { + return join(found[0] ?? "", "checkout"); + } + if (Date.now() > deadline) { + throw new Error( + `no single managed ${kind} slot appeared beneath ${base}; found ${JSON.stringify(found)}`, + ); + } + yield* sleep(200); + } +} + +/** Every slot directory at this depth beneath a managed layout root. */ +function* slotsBeneath(base: string, depth: number): Operation { + if (!(yield* exists(base))) { + return []; + } + let level = [base]; + for (let step = 0; step < depth; step += 1) { + const next: string[] = []; + for (const directory of level) { + for (const entry of yield* until(readdir(directory))) { + next.push(join(directory, entry)); + } + } + level = next; + } + return level.sort(); +} + +/** Which managed journey this is, and what the entry it types looks like. */ +export type ManagedJourney = "repository" | "worktree"; + +const QUESTION = + 'Recorded value: {held}. Continue this experiment?'; + +/** The entry this managed journey types, as the specification shows it. */ +function managedEntry(through: ManagedJourney, origin: string): string { + const body = [ + 'first', + 'remove me', + '', + '', + QUESTION, + 'continued', + ]; + if (through === "repository") { + return [ + ``, + '', + ...body, + "", + "", + ].join("\n"); + } + // The self-closing spelling: it binds the checkout path, and a `` + // sibling renders its descendants at exactly that path. + return [ + ``, + '', + "", + ...body, + "", + "", + ].join("\n"); +} + +/** A new entry that selects the same placement afresh and reads live. */ +function managedLiveRead(through: ManagedJourney, origin: string): string { + if (through === "repository") { + return [ + ``, + '', + "live is {live}", + "", + ].join("\n"); + } + return [ + ``, + '', + "", + '', + "live is {live}", + "", + "", + ].join("\n"); +} + +/** + * The whole journey again, through a managed Repository the entry selected. + * + * Same shape as the ambient one and for the same reasons, with the one addition + * this delivery is about: the checkout the file work happens in is one the entry + * *selected*, under the root the command decides from the home it was given. + * Leaving at the question and coming back must not select it again — and the + * completed read must still answer with what it read the first time, while the + * file on disk says something else. + */ +export function replRepositoryJourney(repl: ReplCommand, through: ManagedJourney): Operation { + return scoped(function* (): Operation { + const home = yield* useTempDirectory("xmd-repl-managed-pty-"); + const origin = yield* seedOrigin(home); + // Typed from a directory inside no checkout, so what the entry works in is + // the placement it selected rather than an ambient repository. + const away = join(home, "away"); + yield* until(mkdir(away, { recursive: true })); + const outside = (args: readonly string[]): string => `cd ${shellQuote(away)} && ${repl(args)}`; + + const first = yield* allocate(outside(["repl", "--deny-all"]), home); + yield* showing(first, "[exit]", REACH_MS); + yield* first.type(managedEntry(through, origin)); + yield* sleep(500); + yield* first.type("\r"); + + // The checkout the selection created, found under the managed layout. + const checkout = yield* managedCheckout( + home, + through === "repository" ? "repositories" : "worktrees", + ); + const working = through === "repository" ? join(checkout, "drafts") : checkout; + + // The prefix ran, inside that checkout, and the write after the question + // has not. + yield* holding(join(working, "note.md"), "first"); + yield* removed(join(working, "obsolete.md")); + expect(yield* exists(join(working, "after.md"))).toBe(false); + yield* showing(first, "waiting for an answer", ACT_MS); + + yield* reach(first, "[exit]"); + yield* first.type("\r"); + expect(yield* ended(first)).toBe(0); + const location = printedLocation((yield* first.shows()).join("\n")); + + // Changed from outside, which this execution model allows. + yield* writeTextFile(join(working, "note.md"), "second"); + yield* writeTextFile(join(working, "obsolete.md"), "back again"); + + const second = yield* allocate(outside(["repl", "--deny-all", location]), home); + yield* showing(second, "[exit]", REACH_MS); + yield* showing(second, "waiting for an answer", ACT_MS); + + // No completed operation ran again — the selection included. A reselected + // placement would have reached a selector; a replayed write would have + // overwritten the shell's edit, and a replayed read would have bound + // "second". + expect(yield* readTextFile(join(working, "note.md"))).toBe("second"); + expect(yield* readTextFile(join(working, "obsolete.md"))).toBe("back again"); + expect(yield* exists(join(working, "after.md"))).toBe(false); + + yield* reach(second, "[answer]"); + yield* second.type("\r"); + yield* showing(second, "Recorded value: first", ACT_MS); + yield* showing(second, "Continue this experiment?", ACT_MS); + yield* click(second, "( ) continue"); + yield* showing(second, "[ok] entry-1", ACT_MS); + + // The suffix write executed, for the first time, on the answer — in the + // checkout the restored placement named. + yield* holding(join(working, "after.md"), "continued"); + + yield* reach(second, "[exit]"); + yield* second.type("\r"); + expect(yield* ended(second)).toBe(0); + + // A separate new entry, selecting the same placement afresh: its selection + // is new work, so it reuses the checkout that is there, and its read is new + // work, so it reads what the file now holds. + const third = yield* allocate(outside(["repl", "--deny-all", location]), home); + yield* showing(third, "[exit]", REACH_MS); + yield* third.type(managedLiveRead(through, origin)); + yield* sleep(500); + yield* third.type("\r"); + yield* showing(third, "live is second", ACT_MS); + + yield* reach(third, "[exit]"); + yield* third.type("\r"); + expect(yield* ended(third)).toBe(0); + }); +} + +/** + * What a managed selection does on a runtime that assembles no native setup. + * + * Node and Bun start a REPL and do ordinary file and directory work; neither + * installs a Repository provider, because the Deno adapter's module graph + * reaches `node:sqlite` and Bun has not got one. So a `` there says + * no provider is installed rather than reaching a clone — and says it before + * anything local or remote is touched. + */ +export function replUnsupportedSelection(repl: ReplCommand): Operation { + return scoped(function* (): Operation { + const home = yield* useTempDirectory("xmd-repl-managed-pty-"); + const origin = yield* seedOrigin(home); + const away = join(home, "away"); + yield* until(mkdir(away, { recursive: true })); + const pty = yield* allocate(`cd ${shellQuote(away)} && ${repl(["repl", "--deny-all"])}`, home); + yield* showing(pty, "[exit]", REACH_MS); + yield* pty.type( + [ + ``, + 'first', + "", + ].join("\n"), + ); + yield* sleep(500); + yield* pty.type("\r"); + + yield* showing(pty, "no Repository composition provider is installed", ACT_MS); + // Nothing was placed: no managed layout was created at all. + expect(yield* exists(join(home, ".xmd", "repositories", "repositories"))).toBe(false); + + yield* reach(pty, "[exit]"); + yield* pty.type("\r"); + expect(yield* ended(pty)).toBe(0); + }); +} + /** * The prefix alone, for a runtime that assembles no entry host. * diff --git a/packages/git/src/deno/composition/host.ts b/packages/git/src/deno/composition/host.ts index 5c55022c6..a78cc2375 100644 --- a/packages/git/src/deno/composition/host.ts +++ b/packages/git/src/deno/composition/host.ts @@ -59,7 +59,7 @@ import type { GitAuthentication, GitAuthenticationSession, } from "./authentication.ts"; -import { runProcess, type ProcessOutcome } from "./subprocess.ts"; +import { runProcess, type ProcessInvocation, type ProcessOutcome } from "./subprocess.ts"; /** What one Git invocation reported. A nonzero exit is an answer, not a throw. */ export type GitOutcome = ProcessOutcome; @@ -106,6 +106,15 @@ export interface GitInvocation { * command that stays inside the materialization. */ readonly attachment?: GitAttachment; + /** + * Handed the child as it is spawned, before anything is attached to it. + * + * Package-private, and forwarded unchanged to the one place a child exists. + * It is the only point at which a cancellation regression can observe the + * process this command really started — a wrapper around this host sees the + * call and its answer, never the child between them. Production passes none. + */ + readonly observe?: ProcessInvocation["observe"]; } export interface RepositoryHost { @@ -210,6 +219,15 @@ function environment( } export interface RepositoryHostOptions { + /** + * How this host creates the isolation directory a Git session works in. + * + * Package-private, for the cancellation regression: the ownership window this + * leaf has is between starting the creation and being handed the path, and + * holding that open is the only way to cut it. Production states none and + * gets `mkdtemp`. + */ + readonly directories?: { create(prefix: string): Promise }; /** * What this host lends a remote-touching invocation. * @@ -223,6 +241,7 @@ export interface RepositoryHostOptions { } export function denoRepositoryHost(options: RepositoryHostOptions = {}): RepositoryHost { + const directories = options.directories ?? { create: (prefix: string) => mkdtemp(prefix) }; const authentication = options.authentication ?? denoGitAuthentication(options.helper === undefined ? {} : { assembly: options.helper }); @@ -235,6 +254,7 @@ export function denoRepositoryHost(options: RepositoryHostOptions = {}): Reposit committedAt, identity, attachment, + observe, }: GitInvocation): Operation { // A command with no attachment reaches no authentication mechanism. That // is what makes a completed replay reach none: replay performs no remote @@ -248,6 +268,7 @@ export function denoRepositoryHost(options: RepositoryHostOptions = {}): Reposit ...(attachment?.environment ?? {}), }, ...(input === undefined ? {} : { input }), + ...(observe === undefined ? {} : { observe }), }); }, @@ -257,10 +278,29 @@ export function denoRepositoryHost(options: RepositoryHostOptions = {}): Reposit useDirectory(): Operation { return resource(function* (provide) { - const created = yield* until(mkdtemp(join(tmpdir(), "xmd-workflow-git-"))); + // Armed before the directory is asked for, and it joins the request + // rather than only removing what the request returned. A halt that + // arrives while `mkdtemp` is still in flight would otherwise find no + // cleanup established, and the directory would be created into a scope + // that had already finished unwinding — a host path left behind by a + // run that was cancelled before it had anything. + let creating: Promise | undefined; yield* ensure(function* () { - yield* until(rm(created, { recursive: true, force: true })); + if (creating === undefined) { + return; + } + const made = yield* until( + creating.then( + (made) => made, + () => undefined, + ), + ); + if (made !== undefined) { + yield* until(rm(made, { recursive: true, force: true })); + } }); + creating = directories.create(join(tmpdir(), "xmd-workflow-git-")); + const created = yield* until(creating); // The resolved path, not the one that was handed back. A temporary // directory reached through a symbolic link — `/var` on macOS is // `/private/var` — is one path to this process and another to Git, and diff --git a/packages/git/src/deno/retained-repository/files.ts b/packages/git/src/deno/retained-repository/files.ts index 03f6b76bd..74a7a033d 100644 --- a/packages/git/src/deno/retained-repository/files.ts +++ b/packages/git/src/deno/retained-repository/files.ts @@ -1,13 +1,18 @@ /** * The file work one REPL entry records, and the work it simply performs. * - * A document reads and writes files in two places: inside the checkout the - * entry was started in, and everywhere else. Only the first is repository work. - * A write beside the document, in a temporary directory or anywhere outside - * that checkout is ordinary filesystem behavior and is journaled by nothing - * here — giving it a record would place it in a checkout it is not in. + * A document reads and writes files in three places: inside a managed checkout + * this entry selected, inside the checkout the entry was started in, and + * everywhere else. Only the first two are repository work. A write beside the + * document, in a temporary directory or anywhere outside both is ordinary + * filesystem behavior and is journaled by nothing here — giving it a record + * would place it in a checkout it is not in. * - * Inside the checkout the call goes through this entry's durable boundary: the + * A managed placement is looked for first, and the most specific one wins. The + * entry's ambient checkout answers for everything no placement covers, which is + * what keeps an entry that selects nothing working exactly as it did before. + * + * Inside either the call goes through this entry's durable boundary: the * composition beneath performs it, and what it produced is recorded once. A * completed read answers with the text it answered with the first time and a * completed write answers from its receipt, each touching no file, however the @@ -18,6 +23,11 @@ * answers is that one entry at a time changes a repository, so a read proceeds * while another entry holds it and never holds it itself. * + * A managed placement adds the other hold, and it answers a different question: + * one entry at a time *owns* a managed slot. A restored placement took none + * when it replayed, so the first new operation inside it — a read included — + * takes the slot before it is performed. + * * ## One route, one policy * * `` reads and writes, `` and `` all arrive as @@ -66,6 +76,24 @@ import type { ReplLocalValue, } from "../../retained-repository/records.ts"; +/** + * One managed checkout this entry selected, as file routing needs it. + * + * The two routing facts and the one thing a restored placement still has to do + * before it is written in: take the slot back. A completed selection recorded + * where the checkout is and took no lease, so the first genuinely new operation + * inside it is what takes it back — without rerunning the selector or looking at + * what the checkout now holds. + */ +export interface ReplManagedPlacement { + /** The canonical root of the selected checkout. */ + readonly checkoutRoot: string; + /** The canonical common Git directory it belongs to. */ + readonly commonDirectory: string; + /** Take ownership of its managed slot for the entry, or refuse. */ + own(): Operation; +} + /** What the entry knows about where it is working and what it may hold. */ export interface ReplLocalRecorder { /** @@ -82,6 +110,15 @@ export interface ReplLocalRecorder { * to the entry rather than to the element that first needed it. */ hold(commonDirectory: string): Operation; + /** + * The managed checkouts this entry has registered, as they stand now. + * + * Not an operation: membership is rebuilt by selections that already + * happened, so reading it acquires nothing. A placement stays registered for + * the rest of the entry, which is what lets a self-closing `` bind + * a path a later `` sibling renders at. + */ + placements(): readonly ReplManagedPlacement[]; } /** The middleware that records this entry's authored mutations. */ @@ -160,43 +197,97 @@ function* routed( return yield* recordedLocalOperation({ operation, request, perform }); } - const checkout = yield* covering(recorder, request); - if (checkout === undefined) { - return yield* perform(); + const target = yield* targetOf(request); + const managed = covering(recorder.placements(), target); + if (managed !== undefined) { + // Placement ownership first, and for a read as much as for a write: what it + // answers is that one entry at a time owns this managed checkout, which an + // observation inside it needs as much as a change does. + yield* managed.own(); + return yield* held(recorder, managed.commonDirectory, { operation, request, perform }); } - // Only a mutation takes the lease. A read is retained like the rest, but the - // lease is about one entry at a time *changing* a checkout, and an - // observation is not a change: a read that took it would refuse while - // somebody else is working, and refuse them while it is only looking. - if (operation !== "file-read") { - yield* recorder.hold(checkout.commonDirectory); + const checkout = yield* recorder.selection(); + if (checkout === undefined || !inside(checkout.checkoutRoot, target)) { + return yield* perform(); } - return yield* recordedLocalOperation({ operation, request, perform }); + return yield* held(recorder, checkout.commonDirectory, { operation, request, perform }); } -/** The checkout this request points inside, when it points inside one. */ -function* covering( +/** + * Record this operation, holding the shared Git directory when it changes one. + * + * Only a mutation takes the lease. A read is retained like the rest, but the + * lease is about one entry at a time *changing* a repository, and an + * observation is not a change: a read that took it would refuse while somebody + * else is working, and refuse them while it is only looking. + */ +function* held( recorder: ReplLocalRecorder, - request: ReplLocalRequest, -): Operation { - const checkout = yield* recorder.selection(); - if (checkout === undefined) { - return undefined; + commonDirectory: string, + operation: { + readonly operation: ReplLocalOperation; + readonly request: ReplLocalRequest; + perform(): Operation>; + }, +): Operation> { + if (operation.operation !== "file-read") { + yield* recorder.hold(commonDirectory); } - const target = yield* targetOf(request); - return inside(checkout.checkoutRoot, target) ? checkout : undefined; + return yield* recordedLocalOperation(operation); +} + +/** + * The managed checkout this path belongs to, most specific first. + * + * The checkout root itself counts, which is where this differs from the ambient + * route below it. A self-closing `` binds its path and a `` + * sibling then ensures exactly that directory, and an entry that recorded + * selecting it has recorded what `` is standing in — so the ensure is + * retained rather than treated as ordinary work beside the document. + * + * A nested selection's checkout is never inside its owner's, so "most specific" + * is about a path addressed through a placement registered more than once and + * about the managed root layout, not about resolving an ambiguity between a + * Repository and its Worktree. + */ +function covering( + placements: readonly ReplManagedPlacement[], + target: string, +): ReplManagedPlacement | undefined { + let found: ReplManagedPlacement | undefined; + for (const placement of placements) { + if (target !== placement.checkoutRoot && !inside(placement.checkoutRoot, target)) { + continue; + } + if (found === undefined || placement.checkoutRoot.length > found.checkoutRoot.length) { + found = placement; + } + } + return found; } /** - * Where this request points, with the directory it is relative to resolved. + * Where this request points. * - * The working directory is resolved through the filesystem rather than used as - * written, because a host's temporary root is commonly reached through a link — - * `/var` for `/private/var` on macOS — and a checkout discovered by its real - * path would not appear to contain a file addressed through the link. The file - * itself need not exist; its directory does. + * An absolute path names a place on its own, and the working directory is no + * part of where it points — which is how a captured checkout path reaches a + * ``. Joining the two would have routed `/managed/…/checkout` as + * `/ambient/managed/…/checkout`, a place inside no placement at all, so a + * completed ensure was never retained and a new one took no ownership of the + * checkout it was making a directory in. The ordinary operation beneath has + * always used the absolute target by itself; this is the route agreeing with it. + * + * A relative path is resolved against the working directory, and that + * directory is resolved through the filesystem rather than used as written: + * a host's temporary root is commonly reached through a link — `/var` for + * `/private/var` on macOS — and a checkout discovered by its real path would + * not appear to contain a file addressed through the link. The file itself need + * not exist; its directory does. */ function* targetOf(request: ReplLocalRequest): Operation { + if (isAbsolute(request.path)) { + return resolve(request.path); + } return resolve(join(yield* canonical(request.cwd), request.path)); } diff --git a/packages/git/src/deno/retained-repository/leases.ts b/packages/git/src/deno/retained-repository/leases.ts index a2bf612e1..a802da3d4 100644 --- a/packages/git/src/deno/retained-repository/leases.ts +++ b/packages/git/src/deno/retained-repository/leases.ts @@ -22,6 +22,8 @@ import type { Operation } from "effection"; import { StaleInputError } from "@executablemd/durable-streams"; import { useAdvisoryLock } from "@executablemd/workflow/deno"; import { commonDirectoryLock } from "../run-composition/placement.ts"; +import { ManagedCheckoutError } from "../run-composition/errors.ts"; +import type { ManagedCheckoutReason } from "../run-composition/errors.ts"; /** * A checkout another live entry or another process is already working in. @@ -62,3 +64,46 @@ export function* useCommonDirectoryLease(commonDirectory: string): Operation` or a ``, whose refusals are ordinary Files outcomes + * the journal keeps. This is not one of those: another entry holding the slot + * is a condition that lasts as long as their work, and recording it as this + * operation's answer would make it permanent. + * + * So the condition travels as a `StaleInputError` carrying the provider's own + * sentence. Core cannot turn it into a recorded File outcome, a document cannot + * print it away, and later siblings do not run as though the file had been + * written. + */ +export class ManagedPlacementHoldError extends StaleInputError { + override name = "ManagedPlacementHoldError"; + + readonly reason: ManagedCheckoutReason; + + constructor(refusal: ManagedCheckoutError) { + super(refusal.message); + this.reason = refusal.reason; + } +} + +/** + * Take a managed slot back for a restored placement, or raise it as fatal. + * + * Only the managed vocabulary's own refusal is translated. Anything else is not + * a condition this boundary has a word for and travels untouched. + */ +export function* holdManagedPlacement(acquire: () => Operation): Operation { + try { + yield* acquire(); + } catch (error) { + if (error instanceof ManagedCheckoutError) { + throw new ManagedPlacementHoldError(error); + } + throw error; + } +} diff --git a/packages/git/src/deno/retained-repository/provider.ts b/packages/git/src/deno/retained-repository/provider.ts index 880f56d7b..7744bba59 100644 --- a/packages/git/src/deno/retained-repository/provider.ts +++ b/packages/git/src/deno/retained-repository/provider.ts @@ -17,36 +17,59 @@ * * ## What it is not * - * This increment records authored and generated file writes, authored deletions - * and the directory an authored `` ensures, in the checkout the person is - * standing in. There is no managed checkout, no `` or `` - * selection, no local Git operation and nothing external. The components for + * This records file and directory work and the managed checkouts a document + * selects to do it in: authored and generated writes, authored deletions, the + * directory an authored `` ensures, and the `` and + * `` selections that say where. There is no local Git operation — + * no Add, Switch, Commit or Push — and nothing external. The components for * those stay declared where they already are and refuse through their own - * unchanged missing-provider defaults. + * unchanged missing-provider defaults, and the selection handlers installed + * here are not the ordinary all-feature provider, which would enable them. * * ## Installing it reaches nothing * - * No directory, no lease, no `git`, no filesystem read. The ambient checkout is - * discovered by the first mutation that needs to know whether it is inside one, - * and an entry that mutates nothing never looks. Being outside a checkout is - * remembered as "there is none" rather than refused, so an entry writing files - * beside the document runs exactly as it would anywhere else. + * No directory, no lease, no `git`, no filesystem read, and no managed root. + * The ambient checkout is discovered by the first mutation that needs to know + * whether it is inside one, and an entry that mutates nothing never looks. The + * managed root is created by the first selection, and an entry that selects + * nothing never makes one. Being outside a checkout is remembered as "there is + * none" rather than refused, so an entry writing files beside the document runs + * exactly as it would anywhere else. */ -import { useScope } from "effection"; +import { until, useScope } from "effection"; import type { Operation, Scope } from "effection"; +import { realpath } from "node:fs/promises"; +import { ensureDir } from "@effectionx/fs"; import { cwd, Files } from "@executablemd/runtime"; import { denoRepositoryHost } from "../composition/host.ts"; import type { RepositoryHost } from "../composition/host.ts"; import { gitSession } from "../composition/git.ts"; import type { GitSession } from "../composition/git.ts"; -import { discoverAmbientCheckout } from "../run-composition/ambient.ts"; +import type { HelperAssembly } from "../composition/credential-helper.ts"; +import type { GitAuthentication } from "../composition/authentication.ts"; +import { Repository } from "../../composition/api.ts"; +import type { RepositoryRequest, WorktreeRequest } from "../../composition/api.ts"; +import type { RepositorySelection } from "../../composition/selection.ts"; +import { discoverAmbientCheckout, discoverAmbientRepository } from "../run-composition/ambient.ts"; +import type { AmbientRepository } from "../run-composition/ambient.ts"; import { lazily } from "../run-composition/lazy.ts"; -import { useCommonDirectoryLease } from "./leases.ts"; +import { useLeases } from "../run-composition/leases.ts"; +import type { Leases, SlotKind } from "../run-composition/leases.ts"; +import { selectionRegistry } from "../selections.ts"; +import { holdManagedPlacement, useCommonDirectoryLease } from "./leases.ts"; import type { ReplAmbientCheckout } from "../../retained-repository/records.ts"; import { currentCoroutine, selectedCheckout } from "./local.ts"; import { replFileMiddleware } from "./files.ts"; -import type { ReplLocalRecorder } from "./files.ts"; +import type { ReplLocalRecorder, ReplManagedPlacement } from "./files.ts"; +import { + placementSubject, + selectReplAmbientOwner, + selectReplRepository, + selectReplWorktree, + slotTurns, +} from "./selections.ts"; +import type { ReplPlacementRegistration, ReplSelectionEntry } from "./selections.ts"; /** What a caller may state about one entry's recording. */ export interface ReplRecordingOptions { @@ -57,6 +80,19 @@ export interface ReplRecordingOptions { * can drive this exact assembly without a `git` subprocess. */ readonly host?: RepositoryHost; + /** + * Where managed checkouts live. + * + * Stated by the selected REPL host, which is the one place that names + * `~/.xmd/repositories`. An entry with no managed root records file work and + * installs no selector at all, so `` and `` reach their + * ordinary missing-provider refusal rather than a half-configured one. + */ + readonly root?: string; + /** How this host writes and starts its own credential helper for a clone. */ + readonly helper?: HelperAssembly; + /** What this host lends a clone that needs to prove who it is. */ + readonly authentication?: GitAuthentication; } /** @@ -70,8 +106,14 @@ export interface ReplRecordingOptions { * directory is discovered from that. */ export function* useReplRepositoryRecording(options: ReplRecordingOptions = {}): Operation { - const host = options.host ?? denoRepositoryHost(); + const host = + options.host ?? + denoRepositoryHost({ + ...(options.authentication === undefined ? {} : { authentication: options.authentication }), + ...(options.helper === undefined ? {} : { helper: options.helper }), + }); const directory = yield* cwd(); + const configuredRoot = options.root; // The entry task. Every capability below is acquired into it, so the session, // its directory and the lease last as long as the entry does rather than as @@ -111,6 +153,12 @@ export function* useReplRepositoryRecording(options: ReplRecordingOptions = {}): // a `git` invocation. const selected = new Map(); + // Registered by a completed or live selection and kept for the rest of the + // entry, so a self-closing `` can bind a path a later `` + // sibling renders at. Empty for an entry with no managed root, which is what + // leaves PR 1's ambient routing the only route there. + const placements: ReplManagedPlacement[] = []; + const recorder: ReplLocalRecorder = { *selection(): Operation { const coroutine = yield* currentCoroutine(); @@ -122,19 +170,157 @@ export function* useReplRepositoryRecording(options: ReplRecordingOptions = {}): selected.set(coroutine, { checkout }); return checkout; }, - *hold(commonDirectory: string): Operation { - let acquire = leases.get(commonDirectory); - if (acquire === undefined) { - acquire = lazily(entry, function* (): Operation { - yield* useCommonDirectoryLease(commonDirectory); - }); - leases.set(commonDirectory, acquire); - } - yield* acquire(); - }, + hold: (commonDirectory: string) => holdCommon(commonDirectory), + placements: () => placements, }; + function holdCommon(commonDirectory: string): Operation { + let acquire = leases.get(commonDirectory); + if (acquire === undefined) { + acquire = lazily(entry, function* (): Operation { + yield* useCommonDirectoryLease(commonDirectory); + }); + leases.set(commonDirectory, acquire); + } + return acquire(); + } + // The outermost band, so what is recorded is the result the whole composition // answered with rather than one layer's own. yield* Files.around(replFileMiddleware(recorder), { at: "max" }); + + // A recorder with no managed root records file work and installs no selector, + // so `` and `` reach their ordinary missing-provider + // refusal. Nothing below this line is reached, named or acquired by such an + // entry. + if (configuredRoot === undefined) { + return; + } + const root = configuredRoot; + + // Created before it is canonicalized, and canonicalized once, exactly as the + // ordinary provider does it. A path that does not exist yet resolves to + // itself, so a root canonicalized before it was made would be one spelling on + // the entry that created it and another on every entry afterwards — two + // spellings, two digests, two slots for one Repository, and no lock between + // them. + const useRoot = lazily(entry, function* (): Operation { + yield* ensureDir(root); + try { + return yield* until(realpath(root)); + } catch { + return root; + } + }); + // One `Leases` per managed root, because a slot's lock sidecar lives under + // the root that slot belongs to. A replayed placement names the root it was + // selected under, which is not necessarily the one configured now — the same + // configured path can resolve somewhere else — and locking a saved slot under + // today's root would open a sidecar nobody else is holding. + const roots = new Map Operation>(); + function useSlots(root: string): Operation { + let acquire = roots.get(root); + if (acquire === undefined) { + acquire = lazily(entry, function* (): Operation { + return yield* useLeases(root); + }); + roots.set(root, acquire); + } + return acquire(); + } + + // Everything a Repository identity needs, and only a root `` needs + // it. An entry that writes files and selects nothing keeps PR 1's smaller + // discovery and never asks these questions. + const useAmbientRepository = lazily(entry, function* (): Operation< + AmbientRepository | undefined + > { + return yield* discoverAmbientRepository(yield* useGit(), directory); + }); + + // One lazy acquisition per managed slot, so a slot taken by a live selection + // is the same hold the first new file operation inside a restored placement + // reacquires. Single-flight, because two concurrent first callers each + // opening their own descriptor would be refused by the operating system for + // contending with their own entry. + const slots = new Map Operation>(); + function holdSlot(root: string, kind: SlotKind, path: string, subject: string): Operation { + // Keyed on the root as well as the slot: two placements of one entry may + // name the same slot path under two roots, and they are two sidecars. + const key = `${root}\u0000${kind}:${path}`; + let acquire = slots.get(key); + if (acquire === undefined) { + acquire = lazily(entry, function* (): Operation { + yield* (yield* useSlots(root)).hold(kind, path, subject); + }); + slots.set(key, acquire); + } + return acquire(); + } + + // This entry's own registry and its own registered placements. Neither is a + // prop, a Context value, a middleware answer or a journal event, and neither + // survives the entry: membership is rebuilt from the placements this + // execution parsed, and rebuilding it acquires nothing. + const selections = selectionRegistry(); + const serialize = slotTurns(); + + const selectionEntry: ReplSelectionEntry = { + host, + git: useGit, + root: useRoot, + ambient: useAmbientRepository, + holdSlot, + holdCommon: (commonDirectory: string) => holdCommon(commonDirectory), + serialize, + selections, + remember(registration: ReplPlacementRegistration): void { + const slot = registration.slot; + // The root this placement recorded, not the one configured now: a + // restored placement's slot belongs to the root it was selected under, + // and that is the sidecar an ordinary owner of it holds. + const placedRoot = registration.placement.managedRoot; + if (slot === undefined || placedRoot === undefined) { + // The ambient owner is a routing answer rather than a managed + // placement, and has neither a slot nor a root. Registering it for file + // coverage would shadow the saved ambient record a coroutine consumes + // at its own cursor. + return; + } + const checkoutRoot = registration.placement.checkoutRoot; + const existing = placements.findIndex((placed) => placed.checkoutRoot === checkoutRoot); + const subject = placementSubject(registration.placement); + const placed: ReplManagedPlacement = { + checkoutRoot, + commonDirectory: registration.placement.commonDirectory, + own(): Operation { + return holdManagedPlacement(() => holdSlot(placedRoot, slot.kind, slot.path, subject)); + }, + }; + if (existing < 0) { + placements.push(placed); + return; + } + placements[existing] = placed; + }, + }; + + // Repository selection, and nothing else this package can do. Installing the + // ordinary all-feature provider here would enable local Git, publication and + // external workers, none of which this delivery retains. + yield* Repository.around( + { + select: ([request]: [RepositoryRequest]): Operation => + selectReplRepository(selectionEntry, request), + + worktree: ([repository, request]: [ + RepositorySelection, + WorktreeRequest, + ]): Operation => selectReplWorktree(selectionEntry, repository, request), + + ambient: (): Operation => + selectReplAmbientOwner(selectionEntry, directory), + }, + { at: "min" }, + ); } diff --git a/packages/git/src/deno/retained-repository/selections.ts b/packages/git/src/deno/retained-repository/selections.ts new file mode 100644 index 000000000..9d97cf32d --- /dev/null +++ b/packages/git/src/deno/retained-repository/selections.ts @@ -0,0 +1,824 @@ +/** + * One REPL entry's Repository and Worktree selections, performed once and + * answered from what they recorded. + * + * A `` or `` in a REPL entry asks the ordinary managed + * selector for a checkout under the host's repository root — the same selector + * `xmd run` asks, with the same reuse, conflict and interrupted-creation policy + * — and records where it ended up. Reopen the entry at a question and that + * recorded placement is the answer: the path is bound, the body expands, and no + * selector is called, nothing is cloned, no `worktree add` runs, no sidecar is + * read, no credential is opened and no lock is taken. + * + * That is the whole of the honesty here. A completed selection says where this + * entry was working. It does not say that the checkout still exists, still + * holds those bytes, or is still a Git checkout at all — somebody may have + * edited it, moved it or deleted it between the question and the answer. + * Genuinely new work afterwards meets today's environment and gets today's + * answer, including an ordinary missing-file failure. + * + * ## Why a saved selector answer is never re-checked + * + * Re-running the selector to "make sure" would undo the exercise. It reads + * metadata, resolves commits and takes the slot's lock, and any of those can + * refuse — so a continuation would fail at a question the person already + * answered, over a checkout whose current state the recorded result was never + * about. The saved facts are parsed, their relationships are checked against + * this call with pure arithmetic, and that is all. + * + * ## Two holds, two jobs + * + * A **slot lease** says one entry at a time owns a managed checkout. A live + * selection takes it before it inspects or creates anything, and it is held + * until the entry tears down so a later `` inside the same entry is still + * protected. A completed replay takes none; the first genuinely new file + * operation inside a restored placement reacquires it then, without revalidating + * the checkout. + * + * The **common-directory lease** is PR 1's kernel lock on the shared Git + * directory, and only a *new* Worktree selector takes it here: `worktree add` + * writes refs and linked administration into the repository everybody else's + * checkouts share. Whether it will create or reuse is not predicted, because + * finding out is the selector's own job. A Repository clone creates its own + * repository under its slot lease and takes no common hold. + * + * Both are acquired inside the recorded operation, so a replayed selection + * acquires neither. When both are needed the slot comes first. + * + * ## The selector finishes before its answer is written down + * + * What the selector itself acquires is shorter-lived than the entry: the + * authentication a clone opens belongs to that one clone. It runs in a scope of + * its own that is torn down and joined before the placement is published, so no + * record and no body work can precede that cleanup. A cleanup that fails takes + * the ordinary fatal path, publishing nothing — a selector whose own teardown + * did not finish has not described a checkout anybody may work in. + */ + +import { Err, Ok, scoped, withResolvers } from "effection"; +import type { Operation, Result, WithResolvers } from "effection"; +import { createDurableOperation, StaleInputError } from "@executablemd/durable-streams"; +import { retainedRecordAhead } from "./local.ts"; +import type { + ActivateDurabilityFailure, + Json, + LiveDurableOperationCoordinator, + Result as ProtocolResult, +} from "@executablemd/durable-streams"; +import type { RepositoryRequest, WorktreeRequest } from "../../composition/api.ts"; +import { + RepositoryCompositionError, + RepositorySelectionError, + WorktreeCompositionError, +} from "../../composition/errors.ts"; +import { + sameRepositoryIdentity, + type RepositoryIdentity, + type RepositorySelection, +} from "../../composition/selection.ts"; +import { + parseReplPlacementRecord, + REPL_REPOSITORY_PLACEMENT, + ReplPlacementRefusal, + replPlacementFingerprint, + replPlacementRecordJson, + replPlacementRecordName, + type ReplManagedRefusalReason, + type ReplPlacement, + type ReplPlacementRefusalData, + type ReplPlacementRequest, +} from "../../retained-repository/placement-records.ts"; +import { admitLocator, locatorFingerprint } from "../composition/locator.ts"; +import { GitRefusal, type GitSession } from "../composition/git.ts"; +import type { RepositoryHost } from "../composition/host.ts"; +import { + CompositionRefusal, + refusalReason, + repositoryRefusal, + worktreeRefusal, +} from "../composition/refusals.ts"; +import { selectManagedRepository, selectManagedWorktree } from "../run-composition/checkouts.ts"; +import { ManagedCheckoutError, NoAmbientRepositoryError } from "../run-composition/errors.ts"; +import type { SlotKind } from "../run-composition/leases.ts"; +import { checkoutOf, repositorySlot, worktreeSlot } from "../run-composition/placement.ts"; +import type { AmbientRepository } from "../run-composition/ambient.ts"; +import type { SelectionRegistry } from "../selections.ts"; + +/** + * A saved placement standing where this selection belongs that it cannot read. + * + * Fatal, on the same terms as a saved local result this host cannot read: a + * record that does not describe this selection is not an outcome of it, so it + * does not belong in the result channel and `` may not carry on + * past it. + */ +export class ReplPlacementRecordError extends StaleInputError { + override name = "ReplPlacementRecordError"; +} + +/** One placement this entry holds, and the managed slot it occupies. */ +export interface ReplPlacementRegistration { + readonly placement: ReplPlacement; + /** + * The managed slot and its lock kind, or absent for an ambient owner. + * + * What the first genuinely new file operation inside a restored placement + * reacquires. Private entry state: a slot is derived from the request and the + * recorded root, never read out of the journal. + */ + readonly slot: { readonly kind: SlotKind; readonly path: string } | undefined; +} + +/** What a recorded selection needs from the entry that owns it. */ +export interface ReplSelectionEntry { + /** The native Git host, which is also what lends clone authentication. */ + readonly host: RepositoryHost; + /** This entry's Git session, acquired once into the entry's own scope. */ + git(): Operation; + /** The canonical managed root, created before it is canonicalized, once. */ + root(): Operation; + /** The repository this invocation started in, discovered at most once. */ + ambient(): Operation; + /** + * Hold a managed slot for the entry, single-flight per root and slot. + * + * The root is stated rather than taken from today's configuration, because a + * slot's lock sidecar lives under the root the slot belongs to: a replayed + * placement's slot belongs to the root its own record carries, and locking it + * under a different one excludes nobody. + */ + holdSlot(root: string, kind: SlotKind, path: string, subject: string): Operation; + /** Hold the shared Git directory a new Worktree selector may change. */ + holdCommon(commonDirectory: string): Operation; + /** Run live selector work for one slot in order within this entry. */ + serialize(key: string, work: () => Operation): Operation; + /** This entry's private map from the selections it minted to what it holds. */ + readonly selections: SelectionRegistry; + /** Remember a placement so file work can route into it and reacquire its slot. */ + remember(registration: ReplPlacementRegistration): void; +} + +/** + * Select the managed Repository this invocation names, or answer from the + * record of selecting it. + * + * The locator is admitted before anything else, because admission is pure and + * its refusal is the same on every run: a url this provider will not hand to + * Git is refused without a record, so a continuation refuses it again rather + * than replaying a verdict about a string it can still see. + */ +export function* selectReplRepository( + entry: ReplSelectionEntry, + request: RepositoryRequest, +): Operation { + const locator = admitLocator(request.locator); + if (locator === undefined) { + throw repositoryRefusal(request.name, "invalid-locator"); + } + const recorded: ReplPlacementRequest = Object.freeze({ + kind: "repository" as const, + name: request.name, + locatorFingerprint: locatorFingerprint(locator), + base: request.base ?? null, + }); + + const registration = yield* recordedPlacement(entry, recorded, { + slot: (root) => repositorySlot(root, locator, request.name), + // The slot lease, before anything inspects or creates the checkout, and + // held until this entry tears down. + acquire: (root, slot) => + entry.holdSlot(root, "repository", slot, `repository ${JSON.stringify(request.name)}`), + *perform(root, slot) { + { + const managed = yield* selectManagedRepository(yield* entry.git(), entry.host, slot, { + name: request.name, + locator, + base: request.base, + }); + const identity: RepositoryIdentity = Object.freeze({ + name: request.name, + locatorFingerprint: managed.metadata.locatorFingerprint, + requestedBase: managed.metadata.requestedBase, + creationCommit: managed.creationCommit, + primaryBranch: managed.metadata.primaryBranch, + objectFormat: managed.objectFormat, + }); + return Object.freeze({ + kind: "repository" as const, + managedRoot: root, + checkoutRoot: managed.checkout, + commonDirectory: managed.commonDirectory, + identity, + displayName: request.name, + // A Repository's own checkout is where a Worktree of it runs + // `worktree add`. + ownerCheckout: managed.checkout, + }); + } + }, + }); + + return mint(entry, registration); +} + +/** + * Select a named linked checkout of a Repository this entry already selected. + * + * The owner is authenticated against this entry's own registry before any + * native work, so a Repository selection that was copied, rebuilt or edited + * names no checkout here. Its stable facts — identity, common directory and + * owner checkout — are what the record is named by, because the selection the + * provider minted for it is fresh every entry and would name a different + * question each time. + */ +export function* selectReplWorktree( + entry: ReplSelectionEntry, + repository: RepositorySelection, + request: WorktreeRequest, +): Operation { + const owner = entry.selections.authenticate( + repository, + () => new RepositorySelectionError(""), + ).placement; + const recorded: ReplPlacementRequest = Object.freeze({ + kind: "worktree" as const, + name: request.name, + branch: request.branch, + base: request.base ?? null, + owner: owner.identity, + commonDirectory: owner.commonDirectory, + ownerCheckout: owner.ownerCheckout, + }); + + const registration = yield* recordedPlacement(entry, recorded, { + slot: (root) => worktreeSlot(root, owner.commonDirectory, request.name), + *acquire(root, slot) { + yield* entry.holdSlot(root, "worktree", slot, `worktree ${JSON.stringify(request.name)}`); + // Before the selector, not after: `worktree add` writes refs and linked + // administration into the Git directory every checkout of that repository + // shares, and whether this call creates or reuses is the selector's own + // answer rather than something to predict. + yield* entry.holdCommon(owner.commonDirectory); + }, + *perform(root, slot) { + { + const managed = yield* selectManagedWorktree(yield* entry.git(), slot, { + name: request.name, + branch: request.branch, + base: request.base, + owner: owner.commonDirectory, + ownerCheckout: owner.ownerCheckout, + }); + return Object.freeze({ + kind: "worktree" as const, + managedRoot: root, + checkoutRoot: managed.checkout, + commonDirectory: managed.commonDirectory, + // The owner's identity, because that is the repository this checkout + // belongs to; the worktree's own name, because that is which checkout + // of it this selection points at. + identity: owner.identity, + displayName: request.name, + ownerCheckout: owner.ownerCheckout, + }); + } + }, + }); + + return mint(entry, registration); +} + +/** + * The Repository this entry is standing in, as a root-level `` needs + * it. + * + * Recorded, so a continuation learns which repository a root Worktree belonged + * to without asking Git again — including when the session was opened inside a + * linked worktree, where the checkout the command ran in and the repository it + * belongs to are two different paths. + * + * Being outside a checkout is not a recordable answer. It is the existing + * actionable refusal, and it is fatal for the same reason it always was: later + * siblings must not run as though a repository had been found. + */ +export function* selectReplAmbientOwner( + entry: ReplSelectionEntry, + from: string, +): Operation { + const recorded: ReplPlacementRequest = Object.freeze({ + kind: "ambient-owner" as const, + cwd: from, + }); + + const registration = yield* recordedPlacement(entry, recorded, { + slot: () => undefined, + // Nothing is held for it: this is the checkout the person was already in. + acquire: () => nothing(), + *perform() { + const ambient = yield* entry.ambient(); + if (ambient === undefined) { + throw new NoAmbientRepositoryError("this element"); + } + const identity: RepositoryIdentity = Object.freeze({ + name: ambient.name, + locatorFingerprint: + ambient.originFingerprint ?? locatorFingerprint(ambient.commonDirectory), + requestedBase: null, + creationCommit: ambient.head, + primaryBranch: ambient.defaultBranch, + objectFormat: ambient.objectFormat, + }); + return Object.freeze({ + kind: "ambient-owner" as const, + // Nothing placed it: this is the checkout the person was already in. + managedRoot: undefined, + checkoutRoot: ambient.checkoutRoot, + commonDirectory: ambient.commonDirectory, + identity, + displayName: ambient.name, + ownerCheckout: ambient.checkoutRoot, + }); + }, + }); + + return mint(entry, registration); +} + +/** + * Perform this selection once, or answer with the placement performing it + * produced. + * + * The durable operation consumes its own position, so two equal selections at + * two positions each record and replay their own. The live work is handed the + * managed root, because that is the one fact a live selection needs and a + * replayed one must not ask for: recanonicalizing today's configured root to + * check a historical answer would make an answer about yesterday depend on what + * is on disk now. + */ +function* recordedPlacement( + entry: ReplSelectionEntry, + request: ReplPlacementRequest, + selector: ReplPlacementSelector, +): Operation { + const fingerprint = yield* replPlacementFingerprint(request); + const name = replPlacementRecordName({ kind: request.kind, fingerprint }); + + // Journal data on a replay, so it arrives as the unknown it is and is parsed + // below rather than trusted as this code's own. + const stored: unknown = yield* performed(entry, request, selector, fingerprint, name); + + const parsed = parseReplPlacementRecord(stored); + if (parsed === undefined) { + throw new ReplPlacementRecordError( + `the saved ${describe(request)} is not one this host can read, so what it says about the ` + + "checkout this entry was working in is unknown.", + ); + } + if (parsed.kind !== request.kind || parsed.fingerprint !== fingerprint) { + throw new ReplPlacementRecordError( + `the saved placement standing where ${describe(request)} belongs describes another ` + + "selection, so this history does not say what happened here.", + ); + } + if (!parsed.outcome.ok) { + throw placementFailure(request, refusalOf(parsed.outcome.error)); + } + return registered(request, parsed.outcome.value, selector.slot); +} + +/** + * How one selection derives its slot, takes its holds and performs itself. + * + * The slot is a function of the root rather than a value, because a live + * selection derives it from the configured root and a replayed one from the + * root its own result recorded. Both derive it from what the document wrote — + * the name, the locator, the owner — which is why the record needs to carry + * neither the slot nor the locator. + */ +interface ReplPlacementSelector { + slot(root: string): string | undefined; + /** The holds a genuinely new selection needs, taken before it runs. */ + acquire(root: string, slot: string): Operation; + /** Call the ordinary selector, and describe what it selected. */ + perform(root: string, slot: string): Operation; +} + +/** + * Perform this selection, or read the record of performing it — and reach the + * holds only for the first. + * + * The retained cursor is read before anything is acquired, because a lease is + * not something a saved answer needs: a continuation standing on a completed + * placement answers from it, and taking the slot to do so would be a hold + * nobody asked for and a refusal nobody can act on. + * + * Acquisition therefore happens *outside* the recorded operation. That is what + * keeps a slot another entry is holding an ordinary refusal the person can act + * on — close the other session and ask again — rather than a durability failure + * the entry cannot recover from, and what keeps temporary contention out of the + * journal. + * + * The peek is a hint and never an authority: the durable engine remains the + * only judge of position. A hint that guessed wrong costs a hold this entry + * would have taken a moment later, which is why {@link attempted} asks for the + * same holds again — idempotently — around work that really does run live. + */ +function* performed( + entry: ReplSelectionEntry, + request: ReplPlacementRequest, + selector: ReplPlacementSelector, + fingerprint: string, + name: string, +): Operation { + function* once(): Operation { + const stored: unknown = yield createDurableOperation( + { type: REPL_REPOSITORY_PLACEMENT, name }, + function* () { + const outcome = yield* attempted(request, selector, yield* entry.root()); + return replPlacementRecordJson({ fingerprint, request, outcome }); + }, + { coordinator: placementCoordinator }, + ); + return stored; + } + + if (yield* retainedRecordAhead(REPL_REPOSITORY_PLACEMENT, name)) { + return yield* once(); + } + const root = yield* entry.root(); + const slot = selector.slot(root); + if (slot === undefined) { + return yield* once(); + } + // One slot, one live selector at a time inside this entry. Two `` + // branches asking for the same checkout both record at their own positions, + // and both must; what they may not do is have two clones racing into one + // directory, so the second waits and then performs its own ordinary reuse + // check — which is what it would have done arriving a moment later. + return yield* entry.serialize(slot, function* () { + yield* selector.acquire(root, slot); + return yield* once(); + }); +} + +/** + * Run the live selector, or classify its refusal as one this record may carry. + * + * A classified refusal is an answer the document asked for and is written down. + * Anything else is infrastructure: it publishes no completion, so a continuation + * reaches the ordinary live path rather than replaying a verdict this host has + * no word for. + */ +function* attempted( + request: ReplPlacementRequest, + selector: ReplPlacementSelector, + root: string, +): Operation> { + const slot = selector.slot(root) ?? ""; + try { + yield* selector.acquire(root, slot); + // Scoped, so everything the selector itself acquired — the authentication a + // clone opened above all — is released and *joined* before this answer + // becomes a result. Without it the selector's resources outlive the + // publication: the durable substrate runs this callback and its append in + // one task, so a credential would still be being torn down while the + // placement was already acknowledged and the body had begun. What stays + // outside is what the entry owns: the Git session, its isolation directory + // and both holds, each acquired into the entry's own scope above. + return Ok(yield* scoped(() => selector.perform(root, slot))); + } catch (error) { + const refusal = classified(request, error); + if (refusal === undefined) { + throw error; + } + return Err(new ReplPlacementRefusal(refusal)); + } +} + +/** + * The closed classification this failure refuses under, or `undefined`. + * + * Three conditions and no more. A native `GitRefusal` from a clone or a + * `worktree add` and a `CompositionRefusal` both carry a word from the + * composition vocabulary and are translated through that vocabulary's own + * sentence, which is how a live refusal and a replayed one end up as the same + * text. A `ManagedCheckoutError` keeps its own word. + * + * `in-use` is not one of them. A slot another entry holds is contention decided + * before this selection begins, and writing it down would make somebody else's + * open session the permanent answer to this question. + */ +function classified( + request: ReplPlacementRequest, + error: unknown, +): ReplPlacementRefusalData | undefined { + if (request.kind === "ambient-owner") { + // Discovery answers with a repository or with nothing, and nothing is the + // existing actionable refusal rather than an outcome worth writing down. + return undefined; + } + if (error instanceof ManagedCheckoutError) { + const reason = MANAGED_REFUSALS.find((word) => word === error.reason); + return reason === undefined + ? undefined + : { vocabulary: "managed", reason, sentence: error.message }; + } + const word = gitWord(error) ?? compositionWord(request, error); + if (word === undefined) { + return undefined; + } + if (request.kind === "worktree") { + const refusal = worktreeRefusal(request.name, word); + return { vocabulary: "worktree", reason: refusal.reason, sentence: refusal.message }; + } + const refusal = repositoryRefusal(request.name, word); + return { vocabulary: "repository", reason: refusal.reason, sentence: refusal.message }; +} + +function gitWord(error: unknown): string | undefined { + return error instanceof GitRefusal ? error.reason : undefined; +} + +function compositionWord(request: ReplPlacementRequest, error: unknown): string | undefined { + if (!(error instanceof CompositionRefusal)) { + return undefined; + } + return refusalReason(error, request.kind === "worktree" ? "worktree" : "repository"); +} + +/** + * The failure a refused selection reaches the document as. + * + * Built here on a live refusal and on a replayed one alike, so the two cannot + * diverge: what the record carries is a word and a sentence, and this is the one + * place either of them becomes an error a component shows. + */ +function placementFailure(request: ReplPlacementRequest, data: ReplPlacementRefusalData): Error { + if (data.vocabulary === "managed") { + return new ManagedCheckoutError(data.reason, data.sentence); + } + if (data.vocabulary === "worktree") { + return new WorktreeCompositionError(named(request), data.reason, data.sentence); + } + return new RepositoryCompositionError(named(request), data.reason, data.sentence); +} + +/** + * The managed words a completed selection may carry, as this adapter writes + * them. + * + * Assigned to the provider's own vocabulary beside it, so a renamed or removed + * managed reason fails to compile here rather than silently leaving a word the + * record format still names. + */ +const MANAGED_REFUSALS: readonly ReplManagedRefusalReason[] = [ + "incompatible-reuse", + "partial-creation", + "unusable-checkout", +]; + +/** + * The refusal data a parsed failure carries. + * + * The codec's failure side is this class by construction, so anything else here + * is this host's own invariant rather than something a record did. + */ +function refusalOf(error: Error): ReplPlacementRefusalData { + if (!(error instanceof ReplPlacementRefusal)) { + throw new ReplPlacementRecordError( + "a saved placement refusal carries no closed classification, so what it refused is unknown.", + ); + } + return error.refusal; +} + +/** + * Hold a parsed placement to the call it is standing at, with pure arithmetic. + * + * Three questions, and none of them touches a filesystem. The result describes + * the same request it is named by; the identity it carries is the one that + * request asked for; and the checkout it names is the one this request's own + * slot would hold under the root the result itself records. A record that + * disagrees with any of them is not an answer to this call. + */ +function registered( + request: ReplPlacementRequest, + placement: ReplPlacement, + slotOf: (root: string) => string | undefined, +): ReplPlacementRegistration { + if (placement.kind !== request.kind) { + throw mismatch(request); + } + if (request.kind === "ambient-owner") { + if (placement.managedRoot !== undefined || placement.ownerCheckout !== placement.checkoutRoot) { + throw mismatch(request); + } + return Object.freeze({ placement, slot: undefined }); + } + const root = placement.managedRoot; + if (root === undefined || placement.displayName !== request.name) { + throw mismatch(request); + } + if (request.kind === "repository") { + if ( + placement.identity.name !== request.name || + placement.identity.locatorFingerprint !== request.locatorFingerprint || + placement.identity.requestedBase !== request.base || + placement.ownerCheckout !== placement.checkoutRoot + ) { + throw mismatch(request); + } + } else if ( + !sameRepositoryIdentity(placement.identity, request.owner) || + placement.commonDirectory !== request.commonDirectory || + placement.ownerCheckout !== request.ownerCheckout + ) { + throw mismatch(request); + } + // Derived from the root the result itself records, never from today's + // configured one: a check of a historical answer that recanonicalized the + // root would answer about the filesystem rather than about the record. + const path = slotOf(root); + if (path === undefined || checkoutOf(path) !== placement.checkoutRoot) { + throw mismatch(request); + } + return Object.freeze({ + placement, + slot: Object.freeze({ kind: request.kind, path }), + }); +} + +function mismatch(request: ReplPlacementRequest): ReplPlacementRecordError { + return new ReplPlacementRecordError( + `the saved ${describe(request)} describes a checkout that is not the one this selection ` + + "names, so this history does not say where the entry was working.", + ); +} + +/** + * Remember this placement for the entry, and mint this execution's selection. + * + * The selection is fresh every entry and private to this provider: the record + * holds the stable facts, and nothing a document can see, copy or rebuild names + * a checkout on its own. + */ +function mint( + entry: ReplSelectionEntry, + registration: ReplPlacementRegistration, +): RepositorySelection { + entry.remember(registration); + const placement = registration.placement; + const key = + registration.slot?.path ?? `ambient ${placement.commonDirectory} ${placement.checkoutRoot}`; + return entry.selections.mint( + key, + placement.displayName, + placement.identity, + placement.checkoutRoot, + registration, + ); +} + +/** + * How a managed slot's refusal names what it is about. + * + * Derived from the placement rather than from the request, so a live selection + * and a restored one that reacquires the same slot say the same thing about it. + */ +export function placementSubject(placement: ReplPlacement): string { + return `${placement.kind} ${JSON.stringify(placement.displayName)}`; +} + +/** An operation that acquires nothing, for a selection that holds nothing. */ +// deno-lint-ignore require-yield +function* nothing(): Operation {} + +function named(request: ReplPlacementRequest): string { + return request.kind === "ambient-owner" ? "" : request.name; +} + +function describe(request: ReplPlacementRequest): string { + if (request.kind === "ambient-owner") { + return "ambient repository for this entry"; + } + return `${request.kind} selection ${JSON.stringify(request.name)}`; +} + +/** + * The live boundary one recorded selection publishes through. + * + * The same contract PR 1's local operations publish under, for the same reason: + * the default coordinator would serialize whatever was thrown and publish it as + * this selection's outcome, and a provider that broke its contract may or may + * not have cloned a repository. So a throw activates the durability fence and is + * re-raised with its identity intact, publishing nothing — and a continuation + * takes the ordinary live path, where the existing partial-creation policy + * decides what to do about whatever native work did happen. + * + * A valid outcome — a placement or a classified refusal — is published once. A + * failing append is the engine's own persistence fence and is not caught here. + */ +const placementCoordinator: LiveDurableOperationCoordinator = { + *run( + execute: () => Operation, + publish: (result: ProtocolResult) => Operation, + activateFailure: ActivateDurabilityFailure, + ): Operation { + let value: T; + try { + value = yield* execute(); + } catch (error) { + throw activateFailure(error); + } + const result: ProtocolResult = { status: "ok", value }; + yield* publish(result); + return result; + }, +}; + +/** + * Run work keyed on one slot in order, within one entry. + * + * Two `` branches may ask for the same Repository at the same time. Both + * record at their own positions and both must, so neither may be suppressed — + * but both calling the selector at once would have two clones racing into one + * directory. So the second waits for the first and then performs its own + * ordinary reuse check, which is what it would have done arriving a moment + * later. + * + * ## A waiter that leaves releases nothing + * + * The turn is a queue with one holder, rather than a chain of waiters each + * signalling the next. Chaining is what made a cancelled waiter grant the slot: + * its cleanup resolved its own completion, which is all its successor was + * waiting for, so the successor entered while the *holder* was still working. + * + * Here a waiter that is cancelled before it is granted the turn takes itself + * out of the queue and signals nothing, so exclusion stays with whoever has it. + * Its own teardown is prompt and waits for no live work. A waiter that was + * granted the turn and then cancelled before it resumed hands the turn on + * rather than stranding it, which is the one case the two cleanups have to tell + * apart. + */ +export function slotTurns(): (key: string, work: () => Operation) => Operation { + interface Waiting { + readonly granted: WithResolvers; + /** Whether the turn has been handed to this waiter. */ + taken: boolean; + } + interface Slot { + held: boolean; + readonly queue: Waiting[]; + } + const slots = new Map(); + + /** Hand the turn to the next waiter, or leave the slot free. */ + function release(key: string, slot: Slot): void { + const next = slot.queue.shift(); + if (next === undefined) { + slot.held = false; + slots.delete(key); + return; + } + next.taken = true; + next.granted.resolve(); + } + + return function* (key: string, work: () => Operation): Operation { + let slot = slots.get(key); + if (slot === undefined) { + slot = { held: false, queue: [] }; + slots.set(key, slot); + } + const here = slot; + if (here.held) { + const mine: Waiting = { granted: withResolvers(), taken: false }; + here.queue.push(mine); + let entered = false; + try { + yield* mine.granted.operation; + entered = true; + } finally { + if (!entered) { + if (mine.taken) { + // Granted and never taken up: the turn is this waiter's to hand on. + release(key, here); + } else { + // Never granted: there is nothing to release, and the holder keeps + // what it has. + const at = here.queue.indexOf(mine); + if (at >= 0) { + here.queue.splice(at, 1); + } + } + } + } + } + here.held = true; + try { + return yield* work(); + } finally { + release(key, here); + } + }; +} diff --git a/packages/git/src/deno/run-composition/checkouts.ts b/packages/git/src/deno/run-composition/checkouts.ts index aa4c2c236..9aa199bf8 100644 --- a/packages/git/src/deno/run-composition/checkouts.ts +++ b/packages/git/src/deno/run-composition/checkouts.ts @@ -31,6 +31,7 @@ */ import { open, rename } from "node:fs/promises"; +import type { FileHandle } from "node:fs/promises"; import { ensureDir, exists, readTextFile, readdir } from "@effectionx/fs"; import { realpath } from "node:fs/promises"; import { ensure, scoped, until, type Operation } from "effection"; @@ -97,20 +98,63 @@ function unusable(sentence: string): never { * already writing; atomic because a reader under the same lock must see either * the whole sidecar or none of it, and a partially written one would describe a * checkout nobody made. + * + * The release is armed before the open is started, and it joins the open rather + * than only closing what the open returned. Registering it afterwards would + * leave a window where a halt arrives while the descriptor is still being + * delivered: the cleanup has not been established, so nothing closes it, and + * the open completes into a scope that has already finished unwinding. Arming + * first is not enough on its own either — a cleanup that ran while the open was + * still in flight would find no handle to close and let teardown finish while + * one was still coming. */ -function* writeMetadata(slot: string, metadata: ManagedMetadata): Operation { +export function* writeMetadata( + slot: string, + metadata: ManagedMetadata, + /** + * How the exclusive temporary sibling is opened. + * + * Package-private, for the cancellation regression: the window this + * discipline exists for is between starting the open and being handed the + * descriptor, and holding that open is the only way to cut it. Production + * states none. + */ + openExclusive: (path: string) => Promise = (path) => open(path, "wx"), +): Operation { const temporary = `${metadataOf(slot)}.${randomUUID()}`; yield* scoped(function* () { - const handle = yield* until(open(temporary, "wx")); - // Registered before the write, so a halt closes the descriptor: closing is - // asynchronous, and a `finally` that suspended would not be guaranteed to - // finish. - yield* ensure(() => until(handle.close())); + let opening: Promise | undefined; + yield* ensure(function* () { + if (opening === undefined) { + return; + } + const opened = yield* until(opening.then(settled, unopened)); + if (opened !== undefined) { + yield* until(opened.close()); + } + }); + opening = openExclusive(temporary); + const handle = yield* until(opening); yield* until(handle.writeFile(metadataBytes(metadata), "utf8")); }); yield* until(rename(temporary, metadataOf(slot))); } +function settled(handle: FileHandle): FileHandle { + return handle; +} + +/** + * What an open that failed delivered: nothing to close. + * + * The failure itself is the caller's, raised where the open was awaited. A + * cleanup that re-raised it would replace whatever ended the scope with a + * second account of the same thing. + */ +function unopened(): undefined { + return undefined; +} + /** The sidecar this slot holds, or `undefined` when it holds none. */ function* readMetadata(slot: string): Operation { const path = metadataOf(slot); diff --git a/packages/git/src/retained-repository/admission.ts b/packages/git/src/retained-repository/admission.ts index 0f94be0af..e6488a723 100644 --- a/packages/git/src/retained-repository/admission.ts +++ b/packages/git/src/retained-repository/admission.ts @@ -39,6 +39,19 @@ import { REPL_REPOSITORY_VERSION, statedReplRecordVersion, } from "./records.ts"; +import { + parseReplPlacementRecord, + replPlacementFingerprint, + replPlacementRecordName, + REPL_REPOSITORY_PLACEMENT, +} from "./placement-records.ts"; + +/** The durable types this reader owns, and the only ones it reads. */ +const OWNED: ReadonlySet = new Set([ + REPL_REPOSITORY_LOCAL, + REPL_REPOSITORY_SELECTION, + REPL_REPOSITORY_PLACEMENT, +]); /** Why a history cannot be continued. */ export type ReplRepositoryRefusal = @@ -80,7 +93,7 @@ export function* admitReplRepositoryHistory(retained: readonly DurableEvent[]): continue; } const type = event.description.type; - if (type !== REPL_REPOSITORY_LOCAL && type !== REPL_REPOSITORY_SELECTION) { + if (!OWNED.has(type)) { continue; } if (event.result.status !== "ok") { @@ -99,6 +112,10 @@ export function* admitReplRepositoryHistory(retained: readonly DurableEvent[]): yield* admitSelection(payload, event.description.name); continue; } + if (type === REPL_REPOSITORY_PLACEMENT) { + yield* admitPlacement(payload, event.description.name); + continue; + } yield* admitLocal(payload, event.description.name); } } @@ -120,6 +137,33 @@ function* admitLocal(payload: unknown, name: string): Operation { } } +/** + * Hold one recorded selection to the request it carries. + * + * The same two questions the other two records answer, over a record whose + * request is larger: a placement states its kind, and a request parsed as that + * kind is what the fingerprint and the name are recomputed from. A record whose + * outcome states a managed root for a checkout nothing placed — or omits one for + * a checkout something did — does not parse, so it is refused here rather than + * binding a path on replay. + * + * It opens nothing. Whether the checkout a placement names is still there is + * not this reader's question and is not a question a continuation asks. + */ +function* admitPlacement(payload: unknown, name: string): Operation { + const record = parseReplPlacementRecord(payload); + if (record === undefined) { + throw malformed(REPL_REPOSITORY_PLACEMENT); + } + const fingerprint = yield* replPlacementFingerprint(record.request); + if (record.fingerprint !== fingerprint) { + throw changed(REPL_REPOSITORY_PLACEMENT); + } + if (name !== replPlacementRecordName({ kind: record.kind, fingerprint })) { + throw misnamed(REPL_REPOSITORY_PLACEMENT); + } +} + function* admitSelection(payload: unknown, name: string): Operation { const record = parseReplSelectionRecord(payload); if (record === undefined) { diff --git a/packages/git/src/retained-repository/placement-records.ts b/packages/git/src/retained-repository/placement-records.ts new file mode 100644 index 000000000..62d0b0984 --- /dev/null +++ b/packages/git/src/retained-repository/placement-records.ts @@ -0,0 +1,553 @@ +/** + * What one REPL entry's Repository and Worktree selections look like on the + * wire. + * + * An entry that writes `` selects a managed + * checkout: a directory under the host's repository root, created once and + * reused afterwards. Reopen the entry and that selection answers again from + * what it recorded — the paths the checkout was found at, the identity it was + * pinned with — without asking a selector anything, cloning, adding a worktree, + * reading a sidecar, opening a credential or taking a lock. The checkout may + * have been edited or removed since; a completed selection says where this + * entry was working, which says nothing about what is there now. + * + * Three kinds are recorded. A **repository** selection names a managed clone, a + * **worktree** selection names a linked checkout of one, and an + * **ambient owner** records the Repository a root-level `` belongs + * to — the checkout the person opened the session in, which a Worktree written + * outside a lexical `` is a worktree *of*. + * + * ## Identity is the request, and position is the journal's + * + * A record's name is the digest of the record version, the kind and the + * normalized request. Two selections asking for exactly the same thing produce + * the same name, and that is allowed: telling two of them apart is the durable + * engine's own positional matching. Nothing here searches history by name, and + * a repeated name is valid. + * + * The request carries what was asked for and nothing more. A Repository's + * locator appears as its fingerprint alone, because the locator is already + * admitted and handed to Git by the time this names it and a second copy in the + * journal would publish a url the identity deliberately does not. A Worktree's + * request carries its owner's repository identity, common directory and owner + * checkout — the stable facts that say which repository this is a worktree of, + * rather than the fresh opaque selection the provider minted for it, which + * names nothing after the entry ends. + * + * ## What a result may carry + * + * Routing and identity: where the checkout is, which Git directory it belongs + * to, the credential-free repository identity, the display name a document + * sees, and the owner checkout a later Worktree would run `worktree add` in. An + * ambient owner carries the same facts without a managed root, because nothing + * placed it. + * + * Nothing else. No origin, no publication evidence, no object source, no Git + * session, no authentication, no lock, no handle, and no opaque selection + * identifier. A later delivery that needs one of those retains it then. + * + * A refusal carries the closed classification the ordinary selector refused + * under and the fixed sentence it refused with, so a continuation reconstructs + * the same failure the live call produced. A native exception, a stack, a + * `stderr` line and a credential are none of those and reach no record: an + * unrecognized condition is infrastructure, and publishing it as something the + * document asked for is exactly what must not happen. + */ + +import { Err, Ok } from "effection"; +import type { Operation, Result } from "effection"; +import type { Json } from "@executablemd/durable-streams"; +import { canonicalJson } from "@executablemd/workflow"; +import { members, optionalText, text } from "../composition/parse.ts"; +import { + parseRepositoryIdentity, + repositoryIdentityJson, + type RepositoryIdentity, +} from "../composition/selection.ts"; +import type { RepositoryFailureReason, WorktreeFailureReason } from "../composition/errors.ts"; +import { digest, RECORD_PREFIX, REPL_REPOSITORY_VERSION } from "./records.ts"; + +/** The durable type one completed Repository or Worktree selection is journaled under. */ +export const REPL_REPOSITORY_PLACEMENT = "repl_repository_placement"; + +/** Which selection a placement record describes. */ +export type ReplPlacementKind = "repository" | "worktree" | "ambient-owner"; + +const PLACEMENT_KINDS: readonly ReplPlacementKind[] = ["repository", "worktree", "ambient-owner"]; + +/** + * What a `` selection asked for. + * + * The locator by fingerprint, because the bytes themselves are the one fact an + * identity never publishes. The base as the caller wrote it or `null`, because + * "no base" and "this base" select differently and a reuse compares them. + */ +export interface ReplRepositoryRequest { + readonly kind: "repository"; + readonly name: string; + readonly locatorFingerprint: string; + readonly base: string | null; +} + +/** + * What a `` selection asked for, owner included. + * + * The owner is three stable facts rather than the selection the provider minted + * for it: a fresh opaque identifier names nothing once the entry that minted it + * is gone, and a durable name built from one would differ on every reopen of + * the same question. + */ +export interface ReplWorktreeRequest { + readonly kind: "worktree"; + readonly name: string; + readonly branch: string; + readonly base: string | null; + /** The authenticated owner's credential-free repository identity. */ + readonly owner: RepositoryIdentity; + /** The owner's canonical common Git directory. */ + readonly commonDirectory: string; + /** The owner checkout `worktree add` would run in. */ + readonly ownerCheckout: string; +} + +/** What a root-level `` asked about the Repository it belongs to. */ +export interface ReplAmbientOwnerRequest { + readonly kind: "ambient-owner"; + /** The directory this invocation started in, which discovery starts from. */ + readonly cwd: string; +} + +export type ReplPlacementRequest = + | ReplRepositoryRequest + | ReplWorktreeRequest + | ReplAmbientOwnerRequest; + +/** + * Where one selected checkout is, and which repository it belongs to. + * + * Routing facts. Holding them is knowing where this entry was working; it is + * not permission to work there and not evidence that anything is still on disk. + */ +export interface ReplPlacement { + readonly kind: ReplPlacementKind; + /** + * The canonical managed root this placement was selected under. + * + * Absent for an ambient owner, which nothing placed: it is the checkout the + * person was already standing in. + */ + readonly managedRoot: string | undefined; + /** The canonical root of the checkout this selection points at. */ + readonly checkoutRoot: string; + /** The canonical common Git directory, which identifies the repository. */ + readonly commonDirectory: string; + /** The credential-free identity every operation and record carries. */ + readonly identity: RepositoryIdentity; + /** The name a document sees: the authored one, or an ambient checkout's own. */ + readonly displayName: string; + /** A checkout of the owning repository, which is where `worktree add` runs. */ + readonly ownerCheckout: string; +} + +/** + * The closed classification one refused selection carries. + * + * Three vocabularies, each the one its own condition is already reported in. A + * `repository` or `worktree` word is the composition vocabulary the components + * speak, which is also what a native `GitRefusal` from a clone or a + * `worktree add` is translated into before it is recorded. A `managed` word is + * the managed-checkout vocabulary a shared host root adds. + * + * `in-use` is deliberately not one of the managed words. A slot another entry + * holds is temporary contention, decided before the recorded selection begins, + * and saving it would make a condition that lasts as long as somebody else's + * work the permanent answer every later continuation replays. + * + * `no-repository-context` is likewise absent: there being no Repository in + * scope is decided by `` before any selector is asked, so it cannot + * arrive back through a record. + */ +export type ReplPlacementRefusalData = + | { + readonly vocabulary: "repository"; + readonly reason: RepositoryFailureReason; + readonly sentence: string; + } + | { + readonly vocabulary: "worktree"; + readonly reason: WorktreeFailureReason; + readonly sentence: string; + } + | { + readonly vocabulary: "managed"; + readonly reason: ReplManagedRefusalReason; + readonly sentence: string; + }; + +/** + * The managed-checkout words a completed selection may carry. + * + * A subset of the provider's own vocabulary, named here because which words a + * durable format carries is part of that format. The adapter that writes and + * rebuilds them is where the compiler holds the two in agreement. + */ +export type ReplManagedRefusalReason = + | "incompatible-reuse" + | "partial-creation" + | "unusable-checkout"; + +const REPOSITORY_REFUSALS: readonly RepositoryFailureReason[] = [ + "invalid-locator", + "authentication-unavailable", + "unresolved-base", + "missing-remote-default", + "incompatible-reuse", + "unusable-repository", +]; + +const WORKTREE_REFUSALS: readonly WorktreeFailureReason[] = [ + "unresolved-base", + "branch-checked-out-elsewhere", + "incompatible-reuse", + "unusable-repository", +]; + +const MANAGED_REFUSALS: readonly ReplManagedRefusalReason[] = [ + "incompatible-reuse", + "partial-creation", + "unusable-checkout", +]; + +/** + * A refused selection, as the record carries it. + * + * The failure side of the internal `Result`, and never what a component sees: + * the adapter turns this into the ordinary error its own vocabulary names, by + * the same route on a live refusal and on a replayed one, so the two cannot + * diverge. + */ +export class ReplPlacementRefusal extends Error { + override name = "ReplPlacementRefusal"; + + readonly refusal: ReplPlacementRefusalData; + + constructor(refusal: ReplPlacementRefusalData) { + super(refusal.sentence); + this.refusal = refusal; + } +} + +/** One completed selection, as a continuation reads it back. */ +export interface ReplPlacementRecord { + readonly version: typeof REPL_REPOSITORY_VERSION; + readonly kind: ReplPlacementKind; + /** The digest of the version, the kind and the normalized request. */ + readonly fingerprint: string; + readonly request: ReplPlacementRequest; + /** The placement it selected, or the refusal it answered with. */ + readonly outcome: Result; +} + +/** The stable name one placement record is journaled under. */ +export function replPlacementRecordName(record: { + readonly kind: ReplPlacementKind; + readonly fingerprint: string; +}): string { + return canonicalJson([ + RECORD_PREFIX, + "placement", + REPL_REPOSITORY_VERSION, + record.kind, + record.fingerprint, + ]); +} + +/** The digest of everything one selection asks for. */ +export function replPlacementFingerprint(request: ReplPlacementRequest): Operation { + return digest({ + version: REPL_REPOSITORY_VERSION, + kind: request.kind, + request: requestJson(request), + }); +} + +/** The payload one completed selection is journaled with. */ +export function replPlacementRecordJson(input: { + readonly fingerprint: string; + readonly request: ReplPlacementRequest; + readonly outcome: Result; +}): Json { + const stated = { + version: REPL_REPOSITORY_VERSION, + kind: input.request.kind, + fingerprint: input.fingerprint, + request: requestJson(input.request), + }; + if (input.outcome.ok) { + return { ...stated, placement: placementJson(input.outcome.value) }; + } + return { ...stated, refusal: refusalJson(refusalData(input.outcome.error)) }; +} + +/** + * The selection this saved value describes, or `undefined`. + * + * Total, and exact about its members. A payload carrying anything this version + * did not write describes something other than a selection this host made, and + * answering a document from it would bind a path nobody selected. + */ +export function parseReplPlacementRecord(value: unknown): ReplPlacementRecord | undefined { + const refused = members(value, ["version", "kind", "fingerprint", "request", "refusal"]); + const selected = + refused === undefined + ? members(value, ["version", "kind", "fingerprint", "request", "placement"]) + : undefined; + const read = refused ?? selected; + if (read === undefined || read["version"] !== REPL_REPOSITORY_VERSION) { + return undefined; + } + const kind = parseReplPlacementKind(read["kind"]); + const fingerprint = text(read["fingerprint"]); + if (kind === undefined || fingerprint === undefined) { + return undefined; + } + const request = parseRequest(read["request"], kind); + if (request === undefined) { + return undefined; + } + const outcome = + refused === undefined ? parsePlacement(read["placement"], kind) : parseRefusal(read["refusal"]); + if (outcome === undefined) { + return undefined; + } + return Object.freeze({ version: REPL_REPOSITORY_VERSION, kind, fingerprint, request, outcome }); +} + +/** The selection kind this value names, or `undefined`. */ +export function parseReplPlacementKind(value: unknown): ReplPlacementKind | undefined { + return PLACEMENT_KINDS.find((kind) => kind === value); +} + +/** + * The refusal data this error carries, or `undefined` when it carries none. + * + * What decides whether a failed selection may be written down at all: a + * classified refusal is an answer the document asked for, and anything else is + * a condition this host has no word for. + */ +export function replPlacementRefusalOf(error: unknown): ReplPlacementRefusalData | undefined { + return error instanceof ReplPlacementRefusal ? error.refusal : undefined; +} + +function refusalData(error: Error): ReplPlacementRefusalData { + const data = replPlacementRefusalOf(error); + if (data === undefined) { + throw new TypeError("a retained placement refusal carries no closed classification"); + } + return data; +} + +function requestJson(request: ReplPlacementRequest): Json { + if (request.kind === "repository") { + return { + name: request.name, + locatorFingerprint: request.locatorFingerprint, + base: request.base, + }; + } + if (request.kind === "worktree") { + return { + name: request.name, + branch: request.branch, + base: request.base, + owner: repositoryIdentityJson(request.owner), + commonDirectory: request.commonDirectory, + ownerCheckout: request.ownerCheckout, + }; + } + return { cwd: request.cwd }; +} + +function placementJson(placement: ReplPlacement): Json { + return { + managedRoot: placement.managedRoot ?? null, + checkoutRoot: placement.checkoutRoot, + commonDirectory: placement.commonDirectory, + identity: repositoryIdentityJson(placement.identity), + displayName: placement.displayName, + ownerCheckout: placement.ownerCheckout, + }; +} + +function refusalJson(refusal: ReplPlacementRefusalData): Json { + return { + vocabulary: refusal.vocabulary, + reason: refusal.reason, + sentence: refusal.sentence, + }; +} + +function parseRequest(value: unknown, kind: ReplPlacementKind): ReplPlacementRequest | undefined { + if (kind === "repository") { + return parseRepositoryRequest(value); + } + if (kind === "worktree") { + return parseWorktreeRequest(value); + } + const read = members(value, ["cwd"]); + const cwd = read === undefined ? undefined : text(read["cwd"]); + return cwd === undefined ? undefined : Object.freeze({ kind: "ambient-owner" as const, cwd }); +} + +function parseRepositoryRequest(value: unknown): ReplRepositoryRequest | undefined { + const read = members(value, ["name", "locatorFingerprint", "base"]); + if (read === undefined) { + return undefined; + } + const name = text(read["name"]); + const locatorFingerprint = text(read["locatorFingerprint"]); + const base = optionalText(read["base"]); + if ( + name === undefined || + locatorFingerprint === undefined || + !/^[0-9a-f]{64}$/.test(locatorFingerprint) || + base === undefined + ) { + return undefined; + } + return Object.freeze({ kind: "repository" as const, name, locatorFingerprint, base }); +} + +function parseWorktreeRequest(value: unknown): ReplWorktreeRequest | undefined { + const read = members(value, [ + "name", + "branch", + "base", + "owner", + "commonDirectory", + "ownerCheckout", + ]); + if (read === undefined) { + return undefined; + } + const name = text(read["name"]); + const branch = text(read["branch"]); + const base = optionalText(read["base"]); + const owner = parseRepositoryIdentity(read["owner"]); + const commonDirectory = text(read["commonDirectory"]); + const ownerCheckout = text(read["ownerCheckout"]); + if ( + name === undefined || + branch === undefined || + base === undefined || + owner === undefined || + commonDirectory === undefined || + ownerCheckout === undefined + ) { + return undefined; + } + return Object.freeze({ + kind: "worktree" as const, + name, + branch, + base, + owner, + commonDirectory, + ownerCheckout, + }); +} + +/** + * The placement a saved success describes, or `undefined`. + * + * A managed root is required of exactly the two kinds a managed root places, + * and refused for the ambient owner: a record stating one for a checkout + * nothing placed, or omitting one for a checkout something did, disagrees with + * the kind it is named under. + * + * Absence is exactly `null`, and nothing else is read as it. A value that is + * present and is not a managed root — a number, an object, the empty string — + * is a record this version did not write, not a record that carried no root: + * reading the two as one answer would admit a malformed ambient placement and + * bind a path from it. + */ +function parsePlacement( + value: unknown, + kind: ReplPlacementKind, +): Result | undefined { + const read = members(value, [ + "managedRoot", + "checkoutRoot", + "commonDirectory", + "identity", + "displayName", + "ownerCheckout", + ]); + if (read === undefined) { + return undefined; + } + const stated = read["managedRoot"]; + if (stated !== null && text(stated) === undefined) { + return undefined; + } + const managedRoot = stated === null ? undefined : text(stated); + if ((kind === "ambient-owner") !== (managedRoot === undefined)) { + return undefined; + } + const checkoutRoot = text(read["checkoutRoot"]); + const commonDirectory = text(read["commonDirectory"]); + const identity = parseRepositoryIdentity(read["identity"]); + const displayName = text(read["displayName"]); + const ownerCheckout = text(read["ownerCheckout"]); + if ( + checkoutRoot === undefined || + commonDirectory === undefined || + identity === undefined || + displayName === undefined || + ownerCheckout === undefined + ) { + return undefined; + } + return Ok( + Object.freeze({ + kind, + managedRoot, + checkoutRoot, + commonDirectory, + identity, + displayName, + ownerCheckout, + }), + ); +} + +function parseRefusal(value: unknown): Result | undefined { + const read = members(value, ["vocabulary", "reason", "sentence"]); + if (read === undefined) { + return undefined; + } + const sentence = text(read["sentence"]); + const stated = read["reason"]; + if (sentence === undefined) { + return undefined; + } + if (read["vocabulary"] === "repository") { + const reason = REPOSITORY_REFUSALS.find((word) => word === stated); + return reason === undefined + ? undefined + : Err(new ReplPlacementRefusal({ vocabulary: "repository", reason, sentence })); + } + if (read["vocabulary"] === "worktree") { + const reason = WORKTREE_REFUSALS.find((word) => word === stated); + return reason === undefined + ? undefined + : Err(new ReplPlacementRefusal({ vocabulary: "worktree", reason, sentence })); + } + if (read["vocabulary"] !== "managed") { + return undefined; + } + const reason = MANAGED_REFUSALS.find((word) => word === stated); + return reason === undefined + ? undefined + : Err(new ReplPlacementRefusal({ vocabulary: "managed", reason, sentence })); +} diff --git a/packages/git/src/retained-repository/records.ts b/packages/git/src/retained-repository/records.ts index 891f68257..2890b4bb7 100644 --- a/packages/git/src/retained-repository/records.ts +++ b/packages/git/src/retained-repository/records.ts @@ -66,7 +66,13 @@ export const REPL_REPOSITORY_LOCAL = "repl_repository_local"; /** The durable type the entry's ambient routing facts are journaled under. */ export const REPL_REPOSITORY_SELECTION = "repl_repository_selection"; -const RECORD_PREFIX = "executablemd.git.repl-repository"; +/** + * The namespace every record name of this feature begins with. + * + * Shared with the placement records beside these, because a second prefix + * would be a second namespace for one feature's history. + */ +export const RECORD_PREFIX = "executablemd.git.repl-repository"; /** Which local operation a record describes. */ export type ReplLocalOperation = "file-read" | "file-write" | "file-delete" | "dir-ensure"; @@ -405,7 +411,13 @@ function requestJson(request: ReplLocalRequest): Json { : { cwd: request.cwd, path: request.path, content: request.content }; } -function* digest(value: Json): Operation { +/** + * The SHA-256 digest of one canonical value, as every name here is built from. + * + * Shared with the placement records beside these: two implementations of one + * digest would be two answers to what a record is named. + */ +export function* digest(value: Json): Operation { const encoded = new TextEncoder().encode(canonicalJson(value)); const computed = yield* until(crypto.subtle.digest("SHA-256", encoded)); return Array.from(new Uint8Array(computed), (byte) => byte.toString(16).padStart(2, "0")).join( diff --git a/packages/git/tests/module-partition.test.ts b/packages/git/tests/module-partition.test.ts index 02a014bea..2614bc429 100644 --- a/packages/git/tests/module-partition.test.ts +++ b/packages/git/tests/module-partition.test.ts @@ -92,6 +92,7 @@ const PROVIDER_NEUTRAL: readonly string[] = [ "src/issue/tracker.ts", "src/plugin.ts", "src/retained-repository/admission.ts", + "src/retained-repository/placement-records.ts", "src/retained-repository/records.ts", ]; @@ -162,6 +163,7 @@ const DENO_ADAPTER: readonly string[] = [ "src/deno/retained-repository/leases.ts", "src/deno/retained-repository/local.ts", "src/deno/retained-repository/provider.ts", + "src/deno/retained-repository/selections.ts", "src/deno/run-composition/ambient.ts", "src/deno/run-composition/checkouts.ts", "src/deno/run-composition/errors.ts", diff --git a/packages/git/tests/retained-repository-lifecycle.test.ts b/packages/git/tests/retained-repository-lifecycle.test.ts index f84cac4ef..ef8b77ca2 100644 --- a/packages/git/tests/retained-repository-lifecycle.test.ts +++ b/packages/git/tests/retained-repository-lifecycle.test.ts @@ -21,6 +21,7 @@ import { type Operation, type Result, ensure, + race, resource, scoped, sleep, @@ -30,7 +31,7 @@ import { withResolvers, } from "effection"; import { exists, readTextFile, rm, writeTextFile } from "@effectionx/fs"; -import { mkdir, mkdtemp, realpath } from "node:fs/promises"; +import { mkdir, mkdtemp, realpath, symlink, unlink } from "node:fs/promises"; import { tmpdir } from "node:os"; import { join } from "node:path"; import { API, hostFilesHandler } from "@executablemd/runtime"; @@ -49,9 +50,15 @@ import { CommonDirectoryLeaseError, useCommonDirectoryLease, } from "../src/deno/retained-repository/leases.ts"; -import { commonDirectoryLock } from "../src/deno/run-composition/placement.ts"; +import { + commonDirectoryLock, + repositorySlot, + worktreeSlot, +} from "../src/deno/run-composition/placement.ts"; +import { useLeases } from "../src/deno/run-composition/leases.ts"; import { useCompositionComponents } from "../src/composition/installation.ts"; import { REPL_REPOSITORY_LOCAL } from "../src/retained-repository/records.ts"; +import { REPL_REPOSITORY_PLACEMENT } from "../src/retained-repository/placement-records.ts"; /** Let every ready task run, so "reached its wait" is about turns. */ function* settled(turns = 8): Operation { @@ -158,8 +165,20 @@ function entry( * released it. A source ending in `` is how a held entry is held in * production too — somebody left it at a question — so this is that, with the * answer never arriving. + * + * A callback that answers returns the answer; one that returns nothing leaves + * the entry where it is, which is what every row holding a checkout open + * needs and what a continuation past the question does not. + */ + holding?: () => Operation, + /** + * Where this entry's managed checkouts live. + * + * Absent for every row about the checkout the entry was opened in: a recorder + * with no managed root installs no selector at all, which is the assembly + * those rows are already about. */ - holding?: () => Operation, + root?: string, ): Operation { return scoped(function* (): Operation { const path = join(checkout, "entry.md"); @@ -189,7 +208,7 @@ function entry( yield* useCompositionComponents(); yield* Execution.around({ *init(_args, next): Operation { - yield* useReplRepositoryRecording(); + yield* useReplRepositoryRecording(root === undefined ? {} : { root }); yield* next(); }, }); @@ -199,8 +218,11 @@ function entry( // the lease its initialization took — until that callback returns. yield* Elicitation.around({ *elicit(): Operation { - yield* holding(); - return yield* suspend(); + const answer = yield* holding(); + if (answer === undefined) { + return yield* suspend(); + } + return answer; }, }); } @@ -221,6 +243,12 @@ function entry( }); } +function placementRecords(events: readonly DurableEvent[]): number { + return events.filter( + (event) => event.type === "yield" && event.description.type === REPL_REPOSITORY_PLACEMENT, + ).length; +} + function localRecords(events: readonly DurableEvent[]): number { return events.filter( (event) => event.type === "yield" && event.description.type === REPL_REPOSITORY_LOCAL, @@ -230,6 +258,14 @@ function localRecords(events: readonly DurableEvent[]): number { /** The question a held entry stands at, never answered. */ const QUESTION = 'Holding'; +/** A question that is answered at once, for a continuation past one. */ +function answeringOnce(): () => Operation { + // deno-lint-ignore require-yield + return function* (): Operation { + return {}; + }; +} + describe("an entry's hold on the checkout it is working in", () => { it("E8: refuses a second live entry, preserving what the first recorded", function* () { const fixture = yield* useFixture(); @@ -538,6 +574,343 @@ describe("an entry's hold on the checkout it is working in", () => { }); }); + it("E4: refuses another entry the managed slot one is holding, until teardown", function* () { + const fixture = yield* useFixture(); + const managed = join(fixture.root, "repositories"); + const origin = yield* useOrigin(fixture.root, "origin"); + const source = [ + ``, + QUESTION, + ].join("\n"); + + yield* scoped(function* (): Operation { + const holding = counters(); + const acquired = withResolvers(); + const release = withResolvers(); + yield* spawn(function* (): Operation { + yield* entry( + fixture.checkout, + source, + new InMemoryStream(), + holding, + function* (): Operation { + acquired.resolve(); + yield* release.operation; + }, + managed, + ); + }); + yield* acquired.operation; + + // A second live entry of the same process, asking for the same managed + // Repository. It is refused by the operating system: the first entry's + // descriptor is a different handle on the same sidecar. + const blocked = counters(); + const second = yield* entry( + fixture.checkout, + source, + new InMemoryStream(), + blocked, + answeringOnce(), + managed, + ); + expect(second.refusal).toContain("another process is working in the managed checkout"); + // Refused before the selector: nothing was written down for a selection + // that never reached one. + expect(placementRecords(second.events)).toBe(0); + + release.resolve(); + }); + + // The holder is gone, so the same selection now reuses the checkout it + // left behind. + const after = counters(); + const later = yield* entry( + fixture.checkout, + source, + new InMemoryStream(), + after, + answeringOnce(), + managed, + ); + expect(later.refusal).toBe(undefined); + }); + + it("E4: lets a restored placement read under a held mutation lease, and not write", function* () { + const fixture = yield* useFixture(); + const managed = join(fixture.root, "repositories"); + const origin = yield* useOrigin(fixture.root, "shared"); + const commonLock = commonDirectoryLock(yield* until(realpath(join(origin, ".git")))); + + // Two linked placements of one repository, selected from the repository the + // entry was opened in. `reader` and `writer` are two histories over the + // same second worktree: each was left at a question, with its own suffix + // still to run. + const reading = counters(); + const readerSource = [ + '', + 'probe', + QUESTION, + '', + "{seen}", + "", + ].join("\n"); + const readerPrefix = yield* leftEntry(origin, readerSource, reading, managed); + + const writing = counters(); + const writerSource = [ + '', + QUESTION, + 'later', + "", + ].join("\n"); + const writerPrefix = yield* leftEntry(origin, writerSource, writing, managed); + + yield* scoped(function* (): Operation { + // One entry holding a *different* linked placement of the same + // repository, and with it the shared Git directory's mutation lease its + // own worktree selector took. + const holding = counters(); + const acquired = withResolvers(); + const release = withResolvers(); + yield* spawn(function* (): Operation { + yield* entry( + origin, + ['', QUESTION, ""].join("\n"), + new InMemoryStream(), + holding, + function* (): Operation { + acquired.resolve(); + yield* release.operation; + }, + managed, + ); + }); + yield* acquired.operation; + + // Watched from here, after the holder has its descriptors: what these two + // continuations do is the subject. + const observed = yield* useObservedDescriptors(); + + const readAgain = counters(); + const read = yield* entry( + origin, + readerSource, + new InMemoryStream([...readerPrefix]), + readAgain, + answeringOnce(), + managed, + ); + // The read happened, in a placement restored from its record, while + // somebody else holds the repository for mutation. + expect(read.refusal).toBe(undefined); + expect(readAgain.reads).toEqual(["probe.md"]); + expect(String(read.output)).toContain("probe"); + // It took the placement's own slot back and never opened the shared Git + // directory's lock: being retained and being exclusive are different + // questions, and so are the two holds. + expect(observed.opened().filter((path) => path === commonLock)).toEqual([]); + expect(observed.events().includes("lock")).toBe(true); + + observed.forget(); + const writeAgain = counters(); + const refused = yield* entry( + origin, + writerSource, + new InMemoryStream([...writerPrefix]), + writeAgain, + answeringOnce(), + managed, + ); + // The mutation refused before it happened, and before any result was + // written down for it. + expect(refused.refusal).toContain("Another XMD entry or process holds this checkout"); + expect(writeAgain.writes).toEqual([]); + expect(yield* exists(join(fixture.root, "later.md"))).toBe(false); + expect(observed.opened().includes(commonLock)).toBe(true); + + release.resolve(); + }); + + // The holder is gone, so the same mutation now takes the lease it always + // did — the positive control for the refusal above. + const after = counters(); + const wrote = yield* entry( + origin, + writerSource, + new InMemoryStream([...writerPrefix]), + after, + answeringOnce(), + managed, + ); + expect(wrote.refusal).toBe(undefined); + expect(after.writes).toEqual(["later.md"]); + }); + + it("E4: reacquires a restored placement's slot under the root it recorded", function* () { + const fixture = yield* useFixture(); + const origin = yield* useOrigin(fixture.root, "recorded-origin"); + // Two real managed roots and one configured spelling that resolves to one + // of them. The configured string never changes; only what it resolves to + // does, which is what makes this about the root a placement *recorded* + // rather than about a caller passing a different option. + const managedA = join(fixture.root, "managed-a"); + const managedB = join(fixture.root, "managed-b"); + const configured = join(fixture.root, "managed-link"); + yield* until(mkdir(managedA, { recursive: true })); + yield* until(mkdir(managedB, { recursive: true })); + yield* until(symlink(managedA, configured, "dir")); + + const source = [ + ``, + QUESTION, + 'after', + "", + ].join("\n"); + + const live = counters(); + const prefix = yield* leftEntry(fixture.checkout, source, live, configured); + expect(placementRecords(prefix)).toBe(1); + + yield* scoped(function* (): Operation { + // An ordinary owner of that placement, holding its slot under the root + // the placement was selected under. + const slot = repositorySlot(managedA, origin, "project"); + const held = yield* useLeases(managedA); + yield* held.hold("repository", slot, 'repository "project"'); + + // The positive control: a second independent acquisition of the same + // slot, under the same root, is refused. Without it, a continuation that + // "succeeded" would prove nothing about which sidecar was opened. + let control: string | undefined; + try { + yield* scoped(function* (): Operation { + const competing = yield* useLeases(managedA); + yield* competing.hold("repository", slot, 'repository "project"'); + }); + } catch (error) { + control = error instanceof Error ? error.message : String(error); + } + expect(control).toContain("another process is working in the managed checkout"); + + // The configured spelling now resolves somewhere else. The placement + // still names the root it recorded, so the continuation's new write has + // to meet the holder above — a lock taken under today's root would open a + // sidecar nobody holds and let the write through. + yield* until(unlink(configured)); + yield* until(symlink(managedB, configured, "dir")); + + const after = counters(); + const continued = yield* entry( + fixture.checkout, + source, + new InMemoryStream([...prefix]), + after, + answeringOnce(), + configured, + ); + expect(continued.refusal).toContain("another process is working in the managed checkout"); + expect(after.writes).toEqual([]); + }); + + // The holder is gone, so the same continuation now performs its write in + // the checkout the placement recorded. + const released = counters(); + const later = yield* entry( + fixture.checkout, + source, + new InMemoryStream([...prefix]), + released, + answeringOnce(), + configured, + ); + expect(later.refusal).toBe(undefined); + expect(released.writes).toEqual(["after.md"]); + // And it landed in the checkout the placement recorded, under root A, + // while the configured spelling still resolves to B. + expect( + yield* readTextFile( + join(repositorySlot(managedA, origin, "project"), "checkout", "after.md"), + ), + ).toBe("after"); + }); + + it("E4: refuses an absolute Dir in a restored placement another owner holds", function* () { + const fixture = yield* useFixture(); + const managed = join(fixture.root, "repositories"); + const origin = yield* useOrigin(fixture.root, "absolute-dir-origin"); + const owner = yield* until(realpath(join(origin, ".git"))); + + // A root Worktree of the repository the entry was opened in, and an + // absolute `` beneath the checkout it bound — reached only after the + // question, so the prefix stops before it. + const source = [ + '', + QUESTION, + "", + 'inside', + "", + ].join("\n"); + + const live = counters(); + const prefix = yield* leftEntry(origin, source, live, managed); + expect(placementRecords(prefix)).toBe(2); + const slot = worktreeSlot(managed, owner, "observed"); + const made = join(slot, "checkout", "made"); + expect(yield* exists(made)).toBe(false); + + yield* scoped(function* (): Operation { + // An ordinary owner of that worktree's slot. + const held = yield* useLeases(managed); + yield* held.hold("worktree", slot, 'worktree "observed"'); + + // The positive control: a second independent acquisition of the same + // slot is refused, so a continuation that "succeeded" below could not be + // read as the lock being free. + let control: string | undefined; + try { + yield* scoped(function* (): Operation { + const competing = yield* useLeases(managed); + yield* competing.hold("worktree", slot, 'worktree "observed"'); + }); + } catch (error) { + control = error instanceof Error ? error.message : String(error); + } + expect(control).toContain("another process is working in the managed checkout"); + + const blocked = counters(); + const refused = yield* entry( + origin, + source, + new InMemoryStream([...prefix]), + blocked, + answeringOnce(), + managed, + ); + // Refused before the ensure reached the filesystem, before the directory + // existed, before the body ran and before any result was written down. + expect(refused.refusal).toContain("another process is working in the managed checkout"); + expect(yield* exists(made)).toBe(false); + expect(blocked.writes).toEqual([]); + expect(localRecords(refused.events)).toBe(localRecords(prefix)); + }); + + // The holder is gone, so the same continuation makes the directory and + // writes in it through ordinary creation. + const after = counters(); + const later = yield* entry( + origin, + source, + new InMemoryStream([...prefix]), + after, + answeringOnce(), + managed, + ); + expect(later.refusal).toBe(undefined); + expect(after.writes).toEqual(["inside.md"]); + expect(yield* readTextFile(join(made, "inside.md"))).toBe("inside"); + }); + it("E8: opens, locks, releases and closes each descriptor exactly once", function* () { const fixture = yield* useFixture(); const lock = commonDirectoryLock(yield* common(fixture.checkout)); @@ -601,9 +974,77 @@ describe("an entry's hold on the checkout it is working in", () => { }); }); +/** + * A source repository with a commit in it, for the rows that select one. + * + * Made inside the row rather than in the shared fixture, so the rows about the + * ambient checkout keep the exact repository they have always had. + */ +function* useOrigin(root: string, name: string): Operation { + const origin = join(root, name); + yield* until(mkdir(origin, { recursive: true })); + const git = gitSession(denoRepositoryHost(), root); + yield* git.run(["init", "-b", "main", origin], root); + yield* writeTextFile(join(origin, "README.md"), "origin\n"); + yield* git.run(["add", "README.md"], origin); + yield* git.run(["commit", "-m", "first"], origin); + return origin; +} + +/** + * The prefix an entry left behind when it was left at a question. + * + * A real unfinished history: the entry is spawned, its question is reached and + * never answered, the journal is read as the interruption left it, and only + * then is the entry halted. The continuation that follows runs the same + * document and reaches the work after the question. + */ +function leftEntry( + checkout: string, + source: string, + counted: Counters, + root: string, +): Operation { + return scoped(function* () { + const stream = new InMemoryStream(); + const reached = withResolvers(); + const task = yield* spawn(() => + entry( + checkout, + source, + stream, + counted, + function* (): Operation { + reached.resolve(); + yield* suspend(); + }, + root, + ), + ); + yield* race([ + reached.operation, + (function* (): Operation { + yield* sleep(30_000); + throw new Error("the entry never reached its question"); + })(), + ]); + const prefix = [...(yield* stream.readAll())]; + yield* task.halt(); + return prefix; + }); +} + /** What this host was asked to do with each lock descriptor, in order. */ interface ObservedDescriptors { events(): readonly string[]; + /** + * Every path a descriptor was opened for, in order. + * + * What distinguishes the two holds: a managed slot's lock and a shared Git + * directory's lock are different sidecars, and a row about one of them has to + * be able to say the other was never opened. + */ + opened(): readonly string[]; /** Start a fresh list, so one row is not read against another's calls. */ forget(): void; /** Make the next lock attempt throw, which is a partial acquisition. */ @@ -630,6 +1071,7 @@ function useObservedDescriptors(): Operation { throw new Error("this host opens no files, so there is no descriptor to observe"); } let events: string[] = []; + let paths: string[] = []; let throwing = false; yield* ensure(() => { @@ -639,6 +1081,7 @@ function useObservedDescriptors(): Operation { Reflect.set(Object(runtime), "openSync", (path: string, options: unknown) => { const file = Reflect.apply(original, runtime, [path, options]); events.push("open"); + paths.push(path); return { tryLockSync(exclusive: boolean): boolean { if (throwing) { @@ -662,8 +1105,10 @@ function useObservedDescriptors(): Operation { yield* provide({ events: () => [...events], + opened: () => [...paths], forget(): void { events = []; + paths = []; }, failLock(): void { throwing = true; diff --git a/packages/git/tests/retained-repository-records.test.ts b/packages/git/tests/retained-repository-records.test.ts index c2f02e9e0..61f3a8258 100644 --- a/packages/git/tests/retained-repository-records.test.ts +++ b/packages/git/tests/retained-repository-records.test.ts @@ -40,9 +40,77 @@ import { admitReplRepositoryHistory, ReplRepositoryHistoryError, } from "../src/retained-repository/admission.ts"; +import { + parseReplPlacementRecord, + REPL_REPOSITORY_PLACEMENT, + ReplPlacementRefusal, + replPlacementFingerprint, + replPlacementRecordJson, + replPlacementRecordName, +} from "../src/retained-repository/placement-records.ts"; +import type { + ReplPlacement, + ReplPlacementRequest, +} from "../src/retained-repository/placement-records.ts"; +import type { RepositoryIdentity } from "../src/composition/selection.ts"; const WRITE: ReplLocalRequest = { cwd: "/work", path: "note.md", content: "first" }; +const IDENTITY: RepositoryIdentity = Object.freeze({ + name: "project", + locatorFingerprint: "a".repeat(64), + requestedBase: null, + creationCommit: "b".repeat(40), + primaryBranch: "main", + objectFormat: "sha1", +}); + +const REPOSITORY_REQUEST: ReplPlacementRequest = Object.freeze({ + kind: "repository", + name: "project", + locatorFingerprint: "a".repeat(64), + base: null, +}); + +const PLACEMENT: ReplPlacement = Object.freeze({ + kind: "repository", + managedRoot: "/managed", + checkoutRoot: "/managed/repositories/slot/checkout", + commonDirectory: "/managed/repositories/slot/checkout/.git", + identity: IDENTITY, + displayName: "project", + ownerCheckout: "/managed/repositories/slot/checkout", +}); + +/** The payload one journaled event carries, as an object this suite can edit. */ +function payloadOf(event: DurableEvent): Record { + if (event.type !== "yield" || event.result.status !== "ok") { + throw new Error("the event carries no payload"); + } + return { ...Object(event.result.value) }; +} + +/** One journaled placement, named the way the host names it. */ +function* placementEvent( + request: ReplPlacementRequest = REPOSITORY_REQUEST, + outcome: Result = Ok(PLACEMENT), + name?: string, +): Operation { + const fingerprint = yield* replPlacementFingerprint(request); + return { + type: "yield", + coroutineId: "root", + description: { + type: REPL_REPOSITORY_PLACEMENT, + name: name ?? replPlacementRecordName({ kind: request.kind, fingerprint }), + }, + result: { + status: "ok", + value: replPlacementRecordJson({ fingerprint, request, outcome }), + }, + }; +} + /** The publication a recorded write's success names, where it is one. */ function receipt(outcome: Result | undefined): string | undefined { if (outcome === undefined || !outcome.ok) { @@ -269,6 +337,155 @@ describe("what one entry's records say", () => { }); }); +describe("what one entry's placement records say", () => { + it("names a selection by its kind and its complete normalized request", function* () { + const repository = yield* replPlacementFingerprint(REPOSITORY_REQUEST); + const renamed = yield* replPlacementFingerprint({ ...REPOSITORY_REQUEST, name: "other" }); + const based = yield* replPlacementFingerprint({ ...REPOSITORY_REQUEST, base: "main" }); + const worktree = yield* replPlacementFingerprint({ + kind: "worktree", + name: "project", + branch: "main", + base: null, + owner: IDENTITY, + commonDirectory: "/managed/repositories/slot/checkout/.git", + ownerCheckout: "/managed/repositories/slot/checkout", + }); + const ambient = yield* replPlacementFingerprint({ kind: "ambient-owner", cwd: "/work" }); + + // Every fact in the request is identity, and the kind is too. + expect(new Set([repository, renamed, based, worktree, ambient]).size).toBe(5); + // And the same request twice is the same name, which is what makes two + // equal selections one name at two ordered positions. + expect(yield* replPlacementFingerprint(REPOSITORY_REQUEST)).toBe(repository); + expect( + replPlacementRecordName({ kind: "repository", fingerprint: repository }) === + replPlacementRecordName({ kind: "worktree", fingerprint: repository }), + ).toBe(false); + }); + + it("reads back the routing a successful selection recorded", function* () { + const fingerprint = yield* replPlacementFingerprint(REPOSITORY_REQUEST); + const payload = replPlacementRecordJson({ + fingerprint, + request: REPOSITORY_REQUEST, + outcome: Ok(PLACEMENT), + }); + + const record = parseReplPlacementRecord(payload); + expect(record?.kind).toBe("repository"); + expect(record?.outcome.ok === true ? record.outcome.value.checkoutRoot : undefined).toBe( + PLACEMENT.checkoutRoot, + ); + expect(record?.outcome.ok === true ? record.outcome.value.identity.name : undefined).toBe( + "project", + ); + // A payload carrying anything this version did not write describes + // something other than a selection this host made. + expect(parseReplPlacementRecord({ ...Object(payload), extra: 1 })).toBe(undefined); + expect(parseReplPlacementRecord({ ...Object(payload), version: 2 })).toBe(undefined); + expect(parseReplPlacementRecord({ ...Object(payload), kind: "elsewhere" })).toBe(undefined); + }); + + it("requires a managed root of exactly the kinds a managed root places", function* () { + const ambient: ReplPlacementRequest = { kind: "ambient-owner", cwd: "/work" }; + const fingerprint = yield* replPlacementFingerprint(ambient); + const owner = replPlacementRecordJson({ + fingerprint, + request: ambient, + outcome: Ok({ ...PLACEMENT, kind: "ambient-owner", managedRoot: undefined }), + }); + + // An ambient owner is the checkout the person was already in, so nothing + // placed it. + expect(parseReplPlacementRecord(owner)?.kind).toBe("ambient-owner"); + const placed = Object(owner); + expect( + parseReplPlacementRecord({ + ...placed, + placement: { ...Object(placed["placement"]), managedRoot: "/managed" }, + }), + ).toBe(undefined); + + // And a repository selection without one disagrees with its own kind. + const repository = Object( + replPlacementRecordJson({ + fingerprint: yield* replPlacementFingerprint(REPOSITORY_REQUEST), + request: REPOSITORY_REQUEST, + outcome: Ok(PLACEMENT), + }), + ); + expect( + parseReplPlacementRecord({ + ...repository, + placement: { ...Object(repository["placement"]), managedRoot: null }, + }), + ).toBe(undefined); + }); + + it("reads an ambient absence as null alone, and refuses anything else", function* () { + const ambient: ReplPlacementRequest = { kind: "ambient-owner", cwd: "/work" }; + const fingerprint = yield* replPlacementFingerprint(ambient); + const payload = Object( + replPlacementRecordJson({ + fingerprint, + request: ambient, + outcome: Ok({ ...PLACEMENT, kind: "ambient-owner", managedRoot: undefined }), + }), + ); + const placement = Object(payload["placement"]); + + // The control: absence is written as null, and reads back as absence. + expect(placement["managedRoot"]).toBe(null); + expect(parseReplPlacementRecord(payload)?.kind).toBe("ambient-owner"); + + // A value that is present and is not a managed root is a record this + // version did not write, not a record that carried no root. Reading the two + // as one answer would let a malformed ambient placement bind a path. + for (const stated of [42, "", true, {}, [], 0]) { + expect( + parseReplPlacementRecord({ ...payload, placement: { ...placement, managedRoot: stated } }), + ).toBe(undefined); + } + }); + + it("carries only a refusal word its own vocabulary names", function* () { + const fingerprint = yield* replPlacementFingerprint(REPOSITORY_REQUEST); + const refused = Object( + replPlacementRecordJson({ + fingerprint, + request: REPOSITORY_REQUEST, + outcome: Err( + new ReplPlacementRefusal({ + vocabulary: "managed", + reason: "incompatible-reuse", + sentence: "the managed checkout was created from a different url.", + }), + ), + }), + ); + + const record = parseReplPlacementRecord(refused); + expect(record?.outcome.ok).toBe(false); + expect(record?.outcome.ok === false ? record.outcome.error.message : undefined).toContain( + "different url", + ); + + // A word outside the closed set is not a refusal this build may rebuild, + // and neither is a vocabulary it does not speak. `in-use` is deliberately + // absent: contention is decided before the recorded selection begins. + for (const refusal of [ + { vocabulary: "managed", reason: "in-use", sentence: "busy" }, + { vocabulary: "managed", reason: "invented", sentence: "busy" }, + { vocabulary: "elsewhere", reason: "incompatible-reuse", sentence: "busy" }, + { vocabulary: "worktree", reason: "no-repository-context", sentence: "busy" }, + { vocabulary: "managed", reason: "incompatible-reuse", sentence: "" }, + ]) { + expect(parseReplPlacementRecord({ ...refused, refusal })).toBe(undefined); + } + }); +}); + describe("which histories one entry will continue", () => { it("admits its own records, including repeated equal names", function* () { const once = yield* localEvent("file-write", WRITE); @@ -355,6 +572,78 @@ describe("which histories one entry will continue", () => { expect(refusal?.message).toContain(`version ${REPL_REPOSITORY_VERSION + 1}`); }); + it("admits a placement record, including repeated equal names", function* () { + const first = yield* placementEvent(); + const second = yield* placementEvent(); + expect(first.type === "yield" && second.type === "yield").toBe(true); + expect(yield* refusalOf([first, second])).toBe(undefined); + }); + + it("refuses a placement record it cannot read", function* () { + const event = yield* placementEvent(); + const broken: DurableEvent = { + ...event, + result: { status: "ok", value: { version: REPL_REPOSITORY_VERSION, kind: "repository" } }, + }; + expect((yield* refusalOf([broken]))?.reason).toBe("malformed-record"); + }); + + it("refuses a placement record stating a version this host does not implement", function* () { + const event = yield* placementEvent(); + const stated: DurableEvent = { + ...event, + result: { + status: "ok", + value: { ...payloadOf(event), version: 2 }, + }, + }; + expect((yield* refusalOf([stated]))?.reason).toBe("unknown-version"); + }); + + it("refuses a placement whose request no longer produces its fingerprint", function* () { + const event = yield* placementEvent(); + const changed: DurableEvent = { + ...event, + result: { + status: "ok", + value: { + ...payloadOf(event), + request: { name: "elsewhere", locatorFingerprint: "a".repeat(64), base: null }, + }, + }, + }; + expect((yield* refusalOf([changed]))?.reason).toBe("request-fingerprint"); + }); + + it("refuses a placement journaled under a name it does not produce", function* () { + const event = yield* placementEvent(REPOSITORY_REQUEST, Ok(PLACEMENT), "another name"); + expect((yield* refusalOf([event]))?.reason).toBe("request-fingerprint"); + }); + + it("refuses an ambient placement whose managed root is not one", function* () { + const ambient: ReplPlacementRequest = { kind: "ambient-owner", cwd: "/work" }; + const event = yield* placementEvent( + ambient, + Ok({ + ...PLACEMENT, + kind: "ambient-owner", + managedRoot: undefined, + }), + ); + // The valid-null control is admitted. + expect(yield* refusalOf([event])).toBe(undefined); + + const placement = Object(payloadOf(event)["placement"]); + const malformed: DurableEvent = { + ...event, + result: { + status: "ok", + value: { ...payloadOf(event), placement: { ...placement, managedRoot: 42 } }, + }, + }; + expect((yield* refusalOf([malformed]))?.reason).toBe("malformed-record"); + }); + it("admits an empty history", function* () { const events: readonly DurableEvent[] = []; expect(yield* refusalOf(events)).toBe(undefined); diff --git a/packages/git/tests/retained-repository-selection.test.ts b/packages/git/tests/retained-repository-selection.test.ts new file mode 100644 index 000000000..e1feb1ae9 --- /dev/null +++ b/packages/git/tests/retained-repository-selection.test.ts @@ -0,0 +1,1468 @@ +/** + * What one REPL entry's Repository and Worktree selections retain, replay and + * refuse to repeat. + * + * Everything here runs a document through canonical execution with the entry + * host installed and a real managed root, so what is proven is the whole path: + * the authored `` asks the composition Api, the ordinary managed + * selector clones or reuses a checkout, the placement is recorded, and a + * continuation answers from that record. + * + * The contract these rows hold the host to is the REPL's execution model. + * Replay restores recorded answers; it does not restore the environment or + * require it to match. So the rows that matter most are the ones where the two + * disagree: a managed checkout edited behind a completed selection, a selected + * worktree deleted outright, and a placement whose result was never + * acknowledged. + * + * Each scenario gets its own source repository, managed root, HOME and journal, + * because what most of them assert is what a *second* open of one history does. + */ + +import { describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import { + ensure, + race, + resource, + scoped, + sleep, + spawn, + suspend, + until, + withResolvers, +} from "effection"; +import type { Operation, Result } from "effection"; +import { exists, readTextFile, rm, writeTextFile } from "@effectionx/fs"; +import { chmod, mkdir, mkdtemp, open, readdir, realpath, writeFile } from "node:fs/promises"; +import type { FileHandle } from "node:fs/promises"; +import process from "node:process"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { API, cwd, hostFilesHandler } from "@executablemd/runtime"; +import type { + FilePathInput, + FilesHandler, + FileWriteInput, + FileWriteSuccess, +} from "@executablemd/runtime"; +import { InMemoryStream } from "@executablemd/durable-streams"; +import type { DurableEvent, DurableStream, Json } from "@executablemd/durable-streams"; +import { Execution } from "@executablemd/core/api"; +import { executeInstalled } from "@executablemd/core/host"; +import { collect, Elicitation } from "@executablemd/core"; +import type { ElicitationRequest } from "@executablemd/core"; +import { denoRepositoryHost } from "../src/deno/composition/host.ts"; +import type { GitInvocation, GitOutcome, RepositoryHost } from "../src/deno/composition/host.ts"; +import { gitSession } from "../src/deno/composition/git.ts"; +import type { GitAuthenticationSession } from "../src/deno/composition/authentication.ts"; +import { UNAUTHENTICATED } from "../src/deno/composition/authentication.ts"; +import { useReplRepositoryRecording } from "../src/deno/retained-repository/provider.ts"; +import { slotTurns } from "../src/deno/retained-repository/selections.ts"; +import { admitReplRepositoryHistory } from "../src/retained-repository/admission.ts"; +import { useCompositionComponents } from "../src/composition/installation.ts"; +import { + parseReplPlacementRecord, + REPL_REPOSITORY_PLACEMENT, +} from "../src/retained-repository/placement-records.ts"; +import type { ReplPlacementRecord } from "../src/retained-repository/placement-records.ts"; +import { REPL_REPOSITORY_LOCAL } from "../src/retained-repository/records.ts"; +import { metadataOf, repositorySlot, worktreeSlot } from "../src/deno/run-composition/placement.ts"; +import { writeMetadata } from "../src/deno/run-composition/checkouts.ts"; +import { METADATA_VERSION } from "../src/deno/run-composition/metadata.ts"; +import type { ManagedRepositoryMetadata } from "../src/deno/run-composition/metadata.ts"; + +interface Fixture { + /** The scenario's own directory: source repository, managed root and HOME. */ + readonly root: string; + /** The repository every `` in these rows clones from. */ + readonly origin: string; + /** Where this scenario's managed checkouts live. */ + readonly managed: string; + /** An ordinary checkout an entry is opened in, for the ambient rows. */ + readonly ambient: string; +} + +/** + * One source repository with a commit in it, plus a managed root of this + * scenario's own. + * + * The locator is an absolute path, so the transport carries no identity and no + * credential is ever asked for — which is also what makes "no authentication + * session was opened" say something about replay rather than about the fixture. + */ +function useFixture(): Operation { + return resource(function* (provide) { + const root = yield* until(realpath(yield* until(mkdtemp(join(tmpdir(), "repl-select-"))))); + yield* ensure(() => rm(root, { recursive: true, force: true })); + const git = gitSession(denoRepositoryHost(), root); + const origin = join(root, "origin"); + yield* until(mkdir(origin, { recursive: true })); + yield* git.run(["init", "-b", "main", origin], root); + yield* until(writeFile(join(origin, "README.md"), "origin\n")); + yield* git.run(["add", "README.md"], origin); + yield* git.run(["commit", "-m", "first"], origin); + + const ambient = join(root, "ambient"); + yield* until(mkdir(ambient, { recursive: true })); + yield* git.run(["init", "-b", "main", ambient], root); + yield* until(writeFile(join(ambient, "seed.md"), "seed\n")); + // Committed, because a Repository identity is pinned at the commit HEAD + // named when the entry opened: a checkout with no commit is one this + // provider cannot say the shape of, and discovery answers with nothing. + yield* git.run(["add", "seed.md"], ambient); + yield* git.run(["commit", "-m", "seed"], ambient); + + const managed = join(root, "repositories"); + yield* provide({ root, origin, managed, ambient }); + }); +} + +/** Let every ready task run, so "reached its wait" is about turns. */ +function* settled(turns = 8): Operation { + for (let turn = 0; turn < turns; turn += 1) { + yield* sleep(0); + } +} + +/** + * A promise a row releases by hand. + * + * What makes the two filesystem cuts below deterministic rather than a race: an + * acquisition is started, the halt is delivered while it is demonstrably still + * pending, and only then is it allowed to deliver. + */ +function gate(): { readonly released: Promise; release(): void } { + let open = (): void => {}; + const released = new Promise((resolve) => { + open = () => resolve(); + }); + return { released, release: () => open() }; +} + +/** The sidecar the metadata cut below writes, which is ordinary record data. */ +const REPOSITORY_SIDECAR: ManagedRepositoryMetadata = Object.freeze({ + kind: "repository" as const, + version: METADATA_VERSION, + name: "project", + locator: "/origin", + locatorFingerprint: "a".repeat(64), + requestedBase: null, + creationCommit: "b".repeat(40), + primaryBranch: "main", + objectFormat: "sha1" as const, + commonDirectory: "/origin/.git", +}); + +/** What one observed `git` child reported about its own lifetime. */ +interface WatchedChildren { + /** Resolves once a watched child has been spawned. */ + readonly spawned: Operation; + /** Handed to the host, which forwards it to the one place a child exists. */ + readonly observe: NonNullable; + /** How many watched children were spawned. */ + observed(): number; + /** Whether every observed child has closed both of its pipes. */ + closed(): boolean; + /** Whether any observed process is still running. */ + alive(): boolean; +} + +/** + * Watch the real children a row's chosen invocation spawns. + * + * The production host forwards this to `runProcess`, which calls it with the + * child as it is spawned — the only point at which a row can see the process + * this command really started. A wrapper around the host sees the call and its + * answer and never the child between them, which is why observing residue + * afterwards cannot tell an interrupted clone from one that never began. + */ +function watchChildren(): WatchedChildren { + const pids: number[] = []; + let count = 0; + let closes = 0; + const first = withResolvers(); + return { + spawned: first.operation, + observe(child): void { + count += 1; + if (child.pid !== undefined) { + pids.push(child.pid); + } + child.on("close", () => { + closes += 1; + }); + first.resolve(); + }, + observed: () => count, + closed: () => count > 0 && closes === count, + alive: () => + pids.some((pid) => { + try { + // Signal 0 asks the operating system whether the process is there. + process.kill(pid, 0); + return true; + } catch { + return false; + } + }), + }; +} + +/** + * A script that reports it ran and then waits. + * + * What makes a child cut deterministic: the marker says the command really + * reached this point, and the wait keeps it there until the halt arrives. The + * process group teardown is what ends it. + */ +function* waitingScript(path: string, marker: string): Operation { + yield* writeTextFile(path, `#!/bin/sh\ntouch ${JSON.stringify(marker)}\nsleep 120\n`); + yield* until(chmod(path, 0o755)); +} + +/** + * Make a `file://` clone of this fixture's origin wait inside its transfer. + * + * `uploadpack.packObjectsHook` is honored from the configuration of the + * repository being read, which is this fixture's own, so the clone's child is a + * real `git clone` with a real descendant doing real work when the halt + * arrives. Without it a local clone of one commit finishes in milliseconds and + * nothing is ever interrupted. + */ +function* blockPackObjects(fixture: Fixture): Operation { + const script = join(fixture.root, "pack-objects-hook"); + const marker = join(fixture.root, "pack-objects-ran"); + yield* waitingScript(script, marker); + const git = gitSession(denoRepositoryHost(), fixture.root); + yield* git.run(["config", "uploadpack.packObjectsHook", script], fixture.origin); + return marker; +} + +/** + * A hooks directory whose one hook waits, for the commands that run hooks. + * + * `worktree add` runs `post-checkout` after it has made the checkout, so the + * marker also says the partial state this row is about is already on disk. + */ +function* blockingHooks(fixture: Fixture, hook: string): Operation { + const hooks = join(fixture.root, "hooks"); + yield* until(mkdir(hooks, { recursive: true })); + const marker = join(fixture.root, `${hook}-ran`); + yield* waitingScript(join(hooks, hook), marker); + return marker; +} + +/** Wait until this path is there, or say what never appeared. */ +function* appears(path: string): Operation { + const deadline = Date.now() + 30_000; + while (!(yield* exists(path))) { + if (Date.now() > deadline) { + throw new Error(`${path} never appeared`); + } + yield* sleep(25); + } +} + +/** What the composition and the host beneath the recorder were asked to do. */ +interface Counters { + reads: string[]; + writes: string[]; + deletes: string[]; + ensures: string[]; + /** Every `git` invocation this entry ran, as its argument list. */ + commands: string[][]; + /** Every locator an authentication session was opened for. */ + sessions: string[]; + /** How many isolation directories this entry's Git session acquired. */ + directories: number; +} + +function counters(): Counters { + return { + reads: [], + writes: [], + deletes: [], + ensures: [], + commands: [], + sessions: [], + directories: 0, + }; +} + +/** The Git subcommands one entry issued, in order. */ +function subcommands(counted: Counters): string[] { + return counted.commands.map((args) => args.find((arg) => !arg.startsWith("-")) ?? ""); +} + +/** + * The host filesystem, with the four document operations counted. + * + * Installed at `min`, the implementation slot, so the recorder's own middleware + * composes above it exactly as it does in production. + */ +function countingFiles(counted: Counters): FilesHandler { + const host = hostFilesHandler(); + return { + checkFilePath: (input) => host.checkFilePath(input), + *readTextFile(input: FilePathInput) { + counted.reads.push(input.path); + return yield* host.readTextFile(input); + }, + globFiles: (input) => host.globFiles(input), + temporaryDirectory: () => host.temporaryDirectory(), + *writeTextFile(input: FileWriteInput) { + counted.writes.push(input.path); + return yield* host.writeTextFile(input); + }, + *deleteFile(input: FilePathInput) { + counted.deletes.push(input.path); + return yield* host.deleteFile(input); + }, + *ensureDirectory(input: FilePathInput) { + counted.ensures.push(input.path); + return yield* host.ensureDirectory(input); + }, + }; +} + +/** What a row may do to one `git` invocation before the real one runs. */ +type GitBarrier = (invocation: GitInvocation) => Operation; + +/** + * The production host, counted, and interruptible at a named subcommand. + * + * Every leaf is wrapped rather than replaced: what these rows need to know is + * whether a selector ran at all, and the only honest way to answer is to let + * the real clone, the real `worktree add` and the real session happen and + * watch. A barrier runs *before* the real invocation is delegated to, so a row + * that halts there halts with the native work still to come — and one that + * delegates first halts with a real child in flight. + */ +function countingHost( + counted: Counters, + barrier?: GitBarrier, + intercept?: (invocation: GitInvocation) => Partial | undefined, +): RepositoryHost { + const host = denoRepositoryHost(); + return { + *git(invocation: GitInvocation): Operation { + counted.commands.push([...invocation.args]); + if (barrier !== undefined) { + yield* barrier(invocation); + } + return yield* host.git({ ...invocation, ...(intercept?.(invocation) ?? {}) }); + }, + *useDirectory(): Operation { + counted.directories += 1; + return yield* host.useDirectory(); + }, + *useAuthentication(locator: string): Operation { + counted.sessions.push(locator); + return host.useAuthentication === undefined + ? UNAUTHENTICATED + : yield* host.useAuthentication(locator); + }, + }; +} + +/** + * The host, with the authentication a clone opens reporting its own release. + * + * Substituted at the one seam the production host already offers, so what this + * observes is the *lifetime* of a selector-owned resource rather than anything + * about a credential transport. + */ +function tracedAuthentication(host: RepositoryHost, trace: string[] | undefined): RepositoryHost { + if (trace === undefined) { + return host; + } + return { + ...host, + useAuthentication(): Operation { + return resource(function* (provide) { + yield* ensure(function* () { + trace.push("auth-cleanup-start"); + yield* sleep(20); + trace.push("auth-cleanup-end"); + }); + yield* provide(UNAUTHENTICATED); + }); + }, + }; +} + +interface Run { + readonly output: string; + readonly events: readonly DurableEvent[]; + /** What ended the entry, when something did. */ + readonly refusal: string | undefined; +} + +interface RunOptions { + /** The directory the entry is opened in. Defaults to the fixture root. */ + readonly cwd?: string; + /** What this row installs between the host and the recording. */ + readonly between?: () => Operation; + /** A barrier inside one `git` invocation. */ + readonly barrier?: GitBarrier; + /** + * What this row changes about one invocation before the real host runs it. + * + * Where a row reaches the child a command spawns, and where it adds the `-c` + * settings that make one wait. Returning nothing leaves the invocation + * exactly as the provider built it. + */ + readonly intercept?: (invocation: GitInvocation) => Partial | undefined; + /** Whether this entry installs a managed root at all. */ + readonly root?: string | undefined; + /** + * Where this row records the selector's own cleanup and the work around it. + * + * Present only for the ownership row: the authentication a clone opens is + * substituted for one that says when it is released, and the first document + * write says when the body began. + */ + readonly trace?: string[]; +} + +/** One entry, over one journal, with a real managed root. */ +function run( + fixture: Fixture, + source: string, + stream: DurableStream, + counted: Counters, + options: RunOptions = {}, +): Operation { + return scoped(function* () { + const path = join(fixture.root, "entry.md"); + yield* writeTextFile(path, source); + const working = options.cwd ?? fixture.root; + yield* API.Env.around( + { + // deno-lint-ignore require-yield + *cwd() { + return working; + }, + }, + { at: "min" }, + ); + const files = countingFiles(counted); + yield* API.Files.around( + { + checkFilePath: ([input]) => files.checkFilePath(input), + readTextFile: ([input]) => files.readTextFile(input), + globFiles: ([input]) => files.globFiles(input), + temporaryDirectory: () => files.temporaryDirectory(), + writeTextFile: ([input]) => files.writeTextFile(input), + deleteFile: ([input]) => files.deleteFile(input), + ensureDirectory: ([input]) => files.ensureDirectory(input), + }, + { at: "min" }, + ); + const trace = options.trace; + if (trace !== undefined) { + yield* API.Files.around({ + *writeTextFile([input], next): Operation> { + trace.push("body-file-write"); + return yield* next(input); + }, + }); + } + yield* useCompositionComponents(); + if (options.between !== undefined) { + yield* options.between(); + } + const managed = "root" in options ? options.root : fixture.managed; + // Registered the way the REPL host registers it: core initialization is + // what establishes the recording and the selectors, inside the scope the + // invocation owns. + yield* Execution.around({ + *init(_args, next): Operation { + yield* useReplRepositoryRecording({ + host: tracedAuthentication( + countingHost(counted, options.barrier, options.intercept), + trace, + ), + ...(managed === undefined ? {} : { root: managed }), + }); + yield* next(); + }, + }); + let refusal: string | undefined; + let output = ""; + try { + output = String( + yield* collect( + yield* executeInstalled({ path, stream, includes: [fixture.root] }, [ + { admissions: [(retained) => admitReplRepositoryHistory(retained)] }, + ]), + ), + ); + } catch (error) { + refusal = error instanceof Error ? error.message : String(error); + } + return { output, events: yield* stream.readAll(), refusal }; + }); +} + +/** + * The prefix an entry left behind when it was left at a question. + * + * A *real* unfinished history, not a complete one with its closes removed: a + * journal whose run finished replays every position it holds and runs no + * component, so a continuation with a new suffix would never reach the suffix. + * What these rows are about is the other case — somebody answered nothing, the + * entry is still standing there, and the work after the question has never run. + * + * So the entry is spawned, its question is reached and never answered, the + * journal is read exactly as the interruption left it, and only then is the + * entry halted. + */ +function left( + fixture: Fixture, + source: string, + counted: Counters, + options: RunOptions = {}, +): Operation { + return scoped(function* () { + const stream = new InMemoryStream(); + const reached = withResolvers(); + const task = yield* spawn(() => + run(fixture, source, stream, counted, { + ...options, + *between(): Operation { + if (options.between !== undefined) { + yield* options.between(); + } + yield* Elicitation.around({ + *elicit(): Operation { + reached.resolve(); + // Nobody answers, so this is where the entry is when it is left. + return yield* suspend(); + }, + }); + }, + }), + ); + yield* race([ + reached.operation, + (function* (): Operation { + yield* sleep(30_000); + throw new Error("the entry never reached its question"); + })(), + ]); + const prefix = [...(yield* stream.readAll())]; + yield* task.halt(); + return prefix; + }); +} + +/** Every payload of one record type, in journal order. */ +function recordsOf(events: readonly DurableEvent[], type: string): readonly Json[] { + const held: Json[] = []; + for (const event of events) { + if (event.type === "yield" && event.description.type === type && event.result.status === "ok") { + held.push(event.result.value ?? null); + } + } + return held; +} + +/** The names, in order, this journal recorded placements under. */ +function placementNames(events: readonly DurableEvent[]): readonly string[] { + return events + .filter( + (event) => event.type === "yield" && event.description.type === REPL_REPOSITORY_PLACEMENT, + ) + .map((event) => (event.type === "yield" ? event.description.name : "")); +} + +/** What each placement record says about the selection it describes. */ +function placements(events: readonly DurableEvent[]): ReplPlacementRecord[] { + return recordsOf(events, REPL_REPOSITORY_PLACEMENT).map((payload) => { + const parsed = parseReplPlacementRecord(payload); + if (parsed === undefined) { + throw new Error("a recorded placement is not readable"); + } + return parsed; + }); +} + +/** The checkout a successful placement record names. */ +function checkoutOf(record: ReplPlacementRecord | undefined): string | undefined { + return record?.outcome.ok === true ? record.outcome.value.checkoutRoot : undefined; +} + +/** The question an entry is left at, never answered. */ +const QUESTION = 'Continue?'; + +/** + * An elicitation that answers at once. + * + * The question is in the document because what these rows are about is leaving + * an entry at one and coming back: the prefix a continuation reopens is the + * journal with its close removed, which is the same history a person leaves + * behind when they answer nothing and the entry is reopened later. + */ +function answered(count: { asked: number } = { asked: 0 }): Operation { + return Elicitation.around({ + // deno-lint-ignore require-yield + *elicit(): Operation { + count.asked += 1; + return {}; + }, + }); +} + +describe("one entry's managed Repository and Worktree selections", () => { + it("E2: replays acknowledged placements and results with no selector work", function* () { + const fixture = yield* useFixture(); + const live = counters(); + // One document, run twice. The first run is left at the question, so the + // suffix after it has never run; the second answers and reaches it. A + // continuation is the same document — a different one is a divergence, not + // a resumption. + const source = [ + ``, + '', + "", + 'first', + '', + "Recorded value: {held}", + QUESTION, + "", + 'new suffix', + "", + ].join("\n"); + const prefix = yield* left(fixture, source, live); + + // Two placements and the file work inside them: one ensure, one write, one + // read. The ensure is the managed worktree root itself, reached through the + // path the self-closing `` bound. + const recorded = placements(prefix); + expect(recorded.map((record) => record.kind)).toEqual(["repository", "worktree"]); + const worktree = checkoutOf(recorded[1]); + expect(worktree).toBe( + `${worktreeSlot(fixture.managed, yield* commonOf(fixture, "project"), "experiment")}/checkout`, + ); + expect(live.ensures.length).toBe(1); + expect(live.writes).toEqual(["note.md"]); + expect(live.reads).toEqual(["note.md"]); + expect(subcommands(live)).toContain("clone"); + expect(subcommands(live)).toContain("worktree"); + expect(yield* readTextFile(join(worktree ?? "", "note.md"))).toBe("first"); + + // The environment and the record now disagree, which the REPL's execution + // model allows. The managed checkout's bytes change, and the selected + // worktree is removed outright — the state a selector would refuse. + yield* writeTextFile(join(worktree ?? "", "note.md"), "changed by somebody else"); + yield* rm(worktree ?? "", { recursive: true, force: true }); + + const replayed = counters(); + const repository = checkoutOf(recorded[0]) ?? ""; + const again = yield* run(fixture, source, new InMemoryStream([...prefix]), replayed, { + between: answered, + }); + + expect(again.refusal).toBe(undefined); + // Not one selector call, clone, `worktree add`, metadata read, credential + // or isolation directory: the Git session is acquired lazily, and nothing + // on this path needed it. + expect(replayed.commands).toEqual([]); + expect(replayed.sessions).toEqual([]); + expect(replayed.directories).toBe(0); + // No replayed file work either: the ensure, the first write and the read + // all answered from their records, and only the new suffix ran. + expect(replayed.ensures).toEqual([]); + expect(replayed.reads).toEqual([]); + expect(replayed.writes).toEqual(["suffix.md"]); + // The binding a continuation offers is the text the read answered with the + // first time, from a checkout that no longer exists. + expect(again.output).toContain("Recorded value: first"); + // Nothing was recreated or repaired to produce it, and the new suffix + // reached the restored Repository placement's own checkout. + expect(yield* exists(worktree ?? "")).toBe(false); + expect(yield* readTextFile(join(repository, "suffix.md"))).toBe("new suffix"); + }); + + it("E2: retains an absolute captured-path Dir, and replays it over a removed checkout", function* () { + const fixture = yield* useFixture(); + const live = counters(); + // The supported spelling this row is about: a self-closing Repository binds + // its checkout path, and a `` renders at exactly that absolute path. + // The entry is opened outside any checkout, so nothing ambient can answer + // for it — the placement is the only route there is. + const source = [ + ``, + "", + QUESTION, + "", + ].join("\n"); + + const prefix = yield* left(fixture, source, live); + const checkout = checkoutOf(placements(prefix)[0]) ?? ""; + expect(checkout).not.toBe(""); + + // The ensure happened, at the checkout root itself, and it was retained: + // an absolute path routed through the working directory would have landed + // inside no placement and been recorded by nothing. + expect(live.ensures).toEqual([checkout]); + expect(recordsOf(prefix, REPL_REPOSITORY_LOCAL).length).toBe(1); + + // The checkout is gone, and the continuation answers the ensure from its + // record rather than making the directory again. + yield* rm(checkout, { recursive: true, force: true }); + + const replayed = counters(); + const again = yield* run(fixture, source, new InMemoryStream([...prefix]), replayed, { + between: answered, + }); + + expect(again.refusal).toBe(undefined); + expect(replayed.ensures).toEqual([]); + expect(replayed.commands).toEqual([]); + expect(replayed.directories).toBe(0); + // And nothing recreated it. A replayed ensure that reached the filesystem + // would have put the checkout root back. + expect(yield* exists(checkout)).toBe(false); + }); + + it("E2: reports an ordinary live failure for a new read of a missing file", function* () { + const fixture = yield* useFixture(); + const live = counters(); + // The paired form, so the read after the question is relative to the + // checkout this selection names without anything creating a directory. + const source = [ + ``, + QUESTION, + "", + '', + "", + "", + ].join("\n"); + + const prefix = yield* left(fixture, source, live); + const checkout = checkoutOf(placements(prefix)[0]); + expect(checkout).not.toBe(undefined); + + // The checkout is gone, and the continuation asks for a file in it. + yield* rm(checkout ?? "", { recursive: true, force: true }); + + const replayed = counters(); + const count = { asked: 0 }; + const again = yield* run(fixture, source, new InMemoryStream([...prefix]), replayed, { + between: () => answered(count), + }); + + // The question replayed from the journal like everything before it, so + // nothing asked it again. + expect(count.asked).toBe(1); + // The read failed where the file is not, in the ordinary vocabulary, and + // nothing cloned the checkout back into existence to answer it. + expect(again.output).toContain('cannot read "absent.md": no such file'); + expect(subcommands(replayed)).not.toContain("clone"); + expect(yield* exists(checkout ?? "")).toBe(false); + }); + + it("E2: replays an ordinary selector refusal without native work", function* () { + const fixture = yield* useFixture(); + const live = counters(); + const stream = new InMemoryStream(); + const source = [ + "", + ``, + "", + QUESTION, + ].join("\n"); + + // A slot carrying a sidecar this version cannot read, which is the + // ordinary `incompatible-reuse` refusal and changes nothing on disk. + const slot = repositorySlot(fixture.managed, fixture.origin, "project"); + yield* until(mkdir(slot, { recursive: true })); + yield* writeTextFile(metadataOf(slot), "{}\n"); + + const prefix = yield* left(fixture, source, live); + expect(subcommands(live)).not.toContain("clone"); + + // Recorded as the refusal it was, and replayed as the same one without the + // selector being asked again. + const recorded = placements(prefix); + expect(recorded.length).toBe(1); + expect(recorded[0]?.outcome.ok).toBe(false); + + const replayed = counters(); + const again = yield* run(fixture, source, new InMemoryStream([...prefix]), replayed, { + between: answered, + }); + const repeated = again.refusal === undefined ? again.output : again.refusal; + expect(repeated).toContain("cannot read"); + expect(replayed.commands).toEqual([]); + expect(replayed.directories).toBe(0); + }); + + it("E3: restores the enclosing Repository and its directory after nesting", function* () { + const fixture = yield* useFixture(); + const live = counters(); + const source = [ + ``, + '', + 'worktree', + "", + ``, + 'nested', + "", + 'outer', + "", + ].join("\n"); + + const ran = yield* run(fixture, source, new InMemoryStream(), live); + expect(ran.refusal).toBe(undefined); + + const recorded = placements(ran.events); + expect(recorded.map((record) => record.kind)).toEqual(["repository", "worktree", "repository"]); + const outer = checkoutOf(recorded[0]) ?? ""; + const tree = checkoutOf(recorded[1]) ?? ""; + const nested = checkoutOf(recorded[2]) ?? ""; + expect(outer === nested).toBe(false); + + // Each write landed in the checkout its own lexical scope named, and the + // write after the nested Repository landed back in the outer one. + expect(yield* readTextFile(join(tree, "in-worktree.md"))).toBe("worktree"); + expect(yield* readTextFile(join(nested, "in-nested.md"))).toBe("nested"); + expect(yield* readTextFile(join(outer, "back-in-outer.md"))).toBe("outer"); + // All three writes were recorded, because all three were inside a managed + // placement this entry registered. + expect(recordsOf(ran.events, REPL_REPOSITORY_LOCAL).length).toBe(3); + }); + + it("E3: consumes its own record for each of two equal requests", function* () { + const fixture = yield* useFixture(); + const live = counters(); + const source = [ + ``, + ``, + "{first} and {second}", + QUESTION, + ].join("\n"); + + const prefix = yield* left(fixture, source, live); + + // Two records, under one name, at two ordered positions: one clone, and + // the second selection reused the slot this entry already held. + const names = placementNames(prefix); + expect(names.length).toBe(2); + expect(names[0]).toBe(names[1]); + expect(subcommands(live).filter((name) => name === "clone").length).toBe(1); + + const replayed = counters(); + const again = yield* run(fixture, source, new InMemoryStream([...prefix]), replayed, { + between: answered, + }); + expect(again.refusal).toBe(undefined); + expect(replayed.commands).toEqual([]); + expect(again.output).toContain("and"); + }); + + it("E3: keeps two concurrent equal selections to one creation and two records", function* () { + const fixture = yield* useFixture(); + const live = counters(); + const selection = ``; + const source = [ + "", + `${selection}{selected}`, + `${selection}{selected}`, + "", + ].join("\n"); + + const ran = yield* run(fixture, source, new InMemoryStream(), live); + expect(ran.refusal).toBe(undefined); + + // Neither record was suppressed, and neither branch raced the other into + // the directory: one clone for the two of them. + expect(placementNames(ran.events).length).toBe(2); + expect(subcommands(live).filter((name) => name === "clone").length).toBe(1); + const recorded = placements(ran.events); + expect(checkoutOf(recorded[0])).toBe(checkoutOf(recorded[1])); + }); + + it("E3: keeps a cancelled slot waiter from releasing the active holder", function* () { + const serialize = slotTurns(); + const active = withResolvers(); + const release = withResolvers(); + let third = false; + + yield* scoped(function* (): Operation { + // A holding the turn, B waiting on A, C waiting behind B. + yield* spawn(() => + serialize("one-slot", function* (): Operation { + active.resolve(); + yield* release.operation; + }), + ); + yield* active.operation; + const waiting = yield* spawn(() => + serialize("one-slot", function* (): Operation { + yield* suspend(); + }), + ); + yield* sleep(0); + yield* spawn(() => + serialize("one-slot", function* (): Operation { + third = true; + yield* suspend(); + }), + ); + yield* sleep(0); + + // B leaves. It was never granted the turn, so it releases nothing and + // the exclusion stays with A — a waiter that signalled its own + // completion here would let C run a selector beside A's. + yield* waiting.halt(); + yield* sleep(0); + expect(third).toBe(false); + + // A ends, and C is the one that inherits the turn. + release.resolve(); + yield* sleep(0); + expect(third).toBe(true); + }); + }); + + it("E5: finishes the selector's own cleanup before the placement is recorded", function* () { + const fixture = yield* useFixture(); + const live = counters(); + const trace: string[] = []; + const events: DurableEvent[] = []; + // The journal, with the placement append marked where it happens. + const stream: DurableStream = { + *readAll(): Operation { + return [...events]; + }, + *append(event: DurableEvent): Operation { + if (event.type === "yield" && event.description.type === REPL_REPOSITORY_PLACEMENT) { + trace.push("placement-append"); + } + events.push(event); + yield* sleep(0); + }, + }; + + const ran = yield* run( + fixture, + [ + ``, + 'body', + "", + ].join("\n"), + stream, + live, + { trace }, + ); + + expect(ran.refusal).toBe(undefined); + // The clone's authentication is the selector's own, not the entry's. Its + // cleanup is joined before the answer becomes a record, so no placement and + // no body work precede it. + expect(trace).toEqual([ + "auth-cleanup-start", + "auth-cleanup-end", + "placement-append", + "body-file-write", + ]); + }); + + it("E3: resumes two spawned branches in their own placements, in reverse order", function* () { + const fixture = yield* useFixture(); + const live = counters(); + // Two branches, two different managed repositories, two questions. Each + // branch records its own placement and its own file work at its own + // position in its own coroutine. + const branch = (name: string) => + [ + ``, + `${name}`, + `Question for ${name}`, + `suffix`, + "", + ].join("\n"); + const source = ["", branch("alpha"), branch("beta"), ""].join("\n"); + + const prefix = yield* scoped(function* () { + const stream = new InMemoryStream(); + const waiting = withResolvers(); + let asked = 0; + const task = yield* spawn(() => + run(fixture, source, stream, live, { + *between(): Operation { + yield* Elicitation.around({ + *elicit(): Operation { + asked += 1; + if (asked === 2) { + waiting.resolve(); + } + return yield* suspend(); + }, + }); + }, + }), + ); + yield* race([ + waiting.operation, + (function* (): Operation { + yield* sleep(30_000); + throw new Error("both branches never reached their questions"); + })(), + ]); + const held = [...(yield* stream.readAll())]; + yield* task.halt(); + return held; + }); + + // Two placements, on two different coroutines, and neither branch's record + // is in the other's. + const selections = prefix.filter( + (event) => event.type === "yield" && event.description.type === REPL_REPOSITORY_PLACEMENT, + ); + expect(selections.length).toBe(2); + expect( + new Set(selections.map((event) => (event.type === "yield" ? event.coroutineId : ""))).size, + ).toBe(2); + expect([...live.writes].sort()).toEqual(["alpha.md", "beta.md"]); + + // Continued with the answers arriving in the other order: `alpha` reached + // its question first and is answered last here. + const replayed = counters(); + const again = yield* run(fixture, source, new InMemoryStream([...prefix]), replayed, { + *between(): Operation { + yield* Elicitation.around({ + *elicit([request]: [ElicitationRequest]): Operation { + yield* sleep(request.message.includes("alpha") ? 30 : 0); + return {}; + }, + }); + }, + }); + + expect(again.refusal).toBe(undefined); + // Each branch's suffix ran once, in its own placement, and neither + // replayed write happened again. + expect([...replayed.writes].sort()).toEqual(["alpha-suffix.md", "beta-suffix.md"]); + const recorded = placements(prefix); + const alpha = recorded.find( + (record) => record.request.kind === "repository" && record.request.name === "alpha", + ); + const beta = recorded.find( + (record) => record.request.kind === "repository" && record.request.name === "beta", + ); + expect(checkoutOf(alpha) === checkoutOf(beta)).toBe(false); + expect(yield* readTextFile(join(checkoutOf(alpha) ?? "", "alpha-suffix.md"))).toBe("suffix"); + expect(yield* readTextFile(join(checkoutOf(beta) ?? "", "beta-suffix.md"))).toBe("suffix"); + }); + + it("E3: selects a root Worktree of the repository the entry was opened in", function* () { + const fixture = yield* useFixture(); + const live = counters(); + const source = [ + '', + "", + 'root', + "", + QUESTION, + ].join("\n"); + + const prefix = yield* left(fixture, source, live, { cwd: fixture.ambient }); + + // The ambient owner is recorded as its own selection, so a continuation + // learns which repository the root Worktree belonged to without asking Git. + const recorded = placements(prefix); + expect(recorded.map((record) => record.kind)).toEqual(["ambient-owner", "worktree"]); + const tree = checkoutOf(recorded[1]) ?? ""; + expect(yield* readTextFile(join(tree, "from-root.md"))).toBe("root"); + + const replayed = counters(); + const again = yield* run(fixture, source, new InMemoryStream([...prefix]), replayed, { + cwd: fixture.ambient, + between: answered, + }); + expect(again.refusal).toBe(undefined); + expect(replayed.commands).toEqual([]); + expect(replayed.ensures).toEqual([]); + expect(replayed.writes).toEqual([]); + }); + + it("E3: refuses a root Worktree outside a checkout with the actionable sentence", function* () { + const fixture = yield* useFixture(); + const live = counters(); + const outside = join(fixture.root, "outside"); + yield* until(mkdir(outside, { recursive: true })); + + const ran = yield* run( + fixture, + '', + new InMemoryStream(), + live, + { cwd: outside }, + ); + + const stated = ran.refusal === undefined ? ran.output : ran.refusal; + expect(stated).toContain("needs a repository"); + // Nothing was written down for a selection that found nothing. + expect(placements(ran.events)).toEqual([]); + }); + + it("E6: leaves no placement when the acknowledgement is cancelled", function* () { + const fixture = yield* useFixture(); + const counted = counters(); + const events: DurableEvent[] = []; + const reached = withResolvers(); + const held = withResolvers(); + const source = + `` + + '\nsentinel'; + + // A barrier between the native clone and its acknowledgement: the checkout + // exists, and the append has not happened. + const barrier: DurableStream = { + *readAll(): Operation { + return [...events]; + }, + *append(event: DurableEvent): Operation { + if (event.type === "yield" && event.description.type === REPL_REPOSITORY_PLACEMENT) { + reached.resolve(); + yield* held.operation; + } + events.push(event); + }, + }; + + yield* scoped(function* () { + const task = yield* spawn(() => run(fixture, source, barrier, counted)); + yield* reached.operation; + yield* task.halt(); + }); + + // The clone happened, and nothing says it did — so no body ran past it. + expect(subcommands(counted)).toContain("clone"); + expect(recordsOf(events, REPL_REPOSITORY_PLACEMENT)).toEqual([]); + expect(yield* exists(join(fixture.root, "sentinel.md"))).toBe(false); + + // A continuation therefore takes the live path, where the ordinary reuse + // policy decides what the slot now holds. + const again = counters(); + const continued = yield* run(fixture, source, new InMemoryStream([...events]), again); + expect(continued.refusal).toBe(undefined); + expect(placements(continued.events).length).toBe(1); + }); + + it("E6: bypasses the selector for a placement acknowledged before cancellation", function* () { + const fixture = yield* useFixture(); + const live = counters(); + const source = [ + ``, + QUESTION, + 'new suffix', + "", + ].join("\n"); + + const prefix = yield* left(fixture, source, live); + expect(placements(prefix).length).toBe(1); + const checkout = checkoutOf(placements(prefix)[0]) ?? ""; + + // The suffix the interrupted run never reached: genuinely new work, inside + // a placement that was restored rather than selected again. + const replayed = counters(); + const count = { asked: 0 }; + const again = yield* run(fixture, source, new InMemoryStream([...prefix]), replayed, { + between: () => answered(count), + }); + expect(again.refusal).toBe(undefined); + // The question replayed, the selector was never asked, and the new write + // reached the restored placement's own checkout. + expect(count.asked).toBe(1); + expect(replayed.commands).toEqual([]); + expect(replayed.writes).toEqual(["suffix.md"]); + expect(yield* readTextFile(join(checkout, "suffix.md"))).toBe("new suffix"); + }); + + it("E6: keeps a rejected placement append fatal, with nothing after it", function* () { + const fixture = yield* useFixture(); + const counted = counters(); + const events: DurableEvent[] = []; + const rejecting: DurableStream = { + *readAll(): Operation { + return [...events]; + }, + *append(event: DurableEvent): Operation { + if (event.type === "yield" && event.description.type === REPL_REPOSITORY_PLACEMENT) { + throw new Error("the journal refused this event"); + } + events.push(event); + }, + }; + + const ran = yield* run( + fixture, + [ + "", + ``, + "", + 'sentinel', + ].join("\n"), + rejecting, + counted, + ); + + // The first persistence failure is preserved, no placement was published, + // and the sentinel after it never ran — printing region or not. + const stated = ran.refusal === undefined ? ran.output : ran.refusal; + expect(stated).toContain("persist"); + expect(stated).not.toContain("sentinel"); + expect(recordsOf(events, REPL_REPOSITORY_PLACEMENT)).toEqual([]); + expect(yield* exists(join(fixture.root, "sentinel.md"))).toBe(false); + }); + + it("E5: joins an actually running clone child and its blocked descendant", function* () { + const fixture = yield* useFixture(); + const counted = counters(); + // A clone that is genuinely running when the halt arrives. The transport + // command is a script that reports it was reached and then waits, so the + // child is a real `git clone` holding a real descendant open — not a + // command that had yet to be delegated to. + // + // Git creates the destination only once the connection has succeeded, so + // this cut leaves no bytes behind. What it discriminates is the teardown: + // whether the process this entry started, and the process that one + // started, are joined before the owner finishes. The worktree cut below is + // the one that interrupts a command *after* it has written something, and + // proves the ordinary handling of what it left. + const reached = join(fixture.root, "transport-ran"); + const transport = join(fixture.root, "transport"); + yield* waitingScript(transport, reached); + const locator = "ssh://example.invalid/repo.git"; + const source = [ + ``, + 'sentinel', + ].join("\n"); + + const watched = watchChildren(); + const journal = new InMemoryStream(); + yield* scoped(function* () { + const task = yield* spawn(() => + run(fixture, source, journal, counted, { + intercept: (invocation) => + invocation.args.includes("clone") + ? { + observe: watched.observe, + args: ["-c", `core.sshCommand=${transport}`, ...invocation.args], + } + : undefined, + }), + ); + yield* watched.spawned; + // Not merely spawned: inside its own transport, with a descendant of its + // own waiting. Halting before the delegation would interrupt a command + // that had done nothing, which is the hole this row closes. + yield* appears(reached); + // The guard against a vacuous row: a clone that had already ended would + // make everything below true without anything being interrupted. + expect(watched.closed()).toBe(false); + // Teardown kills the process group and waits for `close`, so this does + // not return until the child and its descendant are gone. + yield* task.halt(); + }); + + expect(watched.observed()).toBe(1); + expect(watched.closed()).toBe(true); + expect(watched.alive()).toBe(false); + + // No placement was published, and no body ran after it. + expect(placements(yield* journal.readAll())).toEqual([]); + expect(yield* exists(join(fixture.root, "sentinel.md"))).toBe(false); + // The isolation directory the session took is gone with the entry. + expect(counted.directories).toBe(1); + + // And nothing was published for a creation that never reached a checkout. + const slot = repositorySlot(fixture.managed, locator, "project"); + expect(yield* exists(metadataOf(slot))).toBe(false); + expect(yield* temporarySiblings(slot)).toEqual([]); + }); + + it("E5: joins an actually running worktree-add child and adopts what it left", function* () { + const fixture = yield* useFixture(); + const counted = counters(); + // `worktree add` runs the `post-checkout` hook in the checkout it just + // made, so a hook that waits holds the real child open after it has done + // its work — which is the partial state the next attempt has to account + // for. + const checkedOut = yield* blockingHooks(fixture, "post-checkout"); + const hooks = join(fixture.root, "hooks"); + const source = [ + ``, + '', + 'sentinel', + "", + ].join("\n"); + + const watched = watchChildren(); + const journal = new InMemoryStream(); + yield* scoped(function* () { + const task = yield* spawn(() => + run(fixture, source, journal, counted, { + intercept: (invocation) => + invocation.args.includes("worktree") + ? { + observe: watched.observe, + args: ["-c", `core.hooksPath=${hooks}`, ...invocation.args], + } + : undefined, + }), + ); + yield* watched.spawned; + // The hook runs after the checkout, so this says the worktree is on disk + // and the command is still the live owner of it. + yield* appears(checkedOut); + expect(watched.closed()).toBe(false); + yield* task.halt(); + }); + + expect(watched.observed()).toBe(1); + expect(watched.closed()).toBe(true); + expect(watched.alive()).toBe(false); + expect(placements(yield* journal.readAll()).map((record) => record.kind)).toEqual([ + "repository", + ]); + expect(yield* exists(join(fixture.root, "sentinel.md"))).toBe(false); + + // The worktree slot holds a checkout nothing describes yet. + const owner = yield* commonOf(fixture, "project"); + const slot = worktreeSlot(fixture.managed, owner, "experiment"); + expect(yield* exists(join(slot, "checkout"))).toBe(true); + expect(yield* exists(metadataOf(slot))).toBe(false); + + // The next attempt proves it under the ordinary stricter proof and adopts + // it, which is what keeps the bytes somebody's work rather than something + // to recreate. + const again = counters(); + const continued = yield* run(fixture, source, new InMemoryStream(), again); + expect(continued.refusal).toBe(undefined); + expect(yield* readTextFile(metadataOf(slot))).toContain('"kind": "worktree"'); + }); + + it("E5: waits for a pending metadata open and closes what it delivers", function* () { + const fixture = yield* useFixture(); + const slot = join(fixture.root, "metadata-slot"); + yield* until(mkdir(slot, { recursive: true })); + + const asked = withResolvers(); + const held = gate(); + let closed = false; + let opened = false; + // The open is started and then held, so the halt below lands in exactly the + // window this ownership discipline exists for: after the acquisition began + // and before its descriptor arrived. + const openExclusive = (path: string): Promise => { + asked.resolve(path); + return held.released + .then(() => open(path, "wx")) + .then((handle) => { + opened = true; + const close = handle.close.bind(handle); + return Object.assign(handle, { + close: (): Promise => { + closed = true; + return close(); + }, + }); + }); + }; + + let returned = false; + yield* scoped(function* (): Operation { + const writing = yield* spawn(() => writeMetadata(slot, REPOSITORY_SIDECAR, openExclusive)); + const temporary = yield* asked.operation; + const halting = yield* spawn(function* (): Operation { + yield* writing.halt(); + returned = true; + }); + yield* settled(); + // Teardown is waiting for the acquisition it armed its release before. + expect(returned).toBe(false); + expect(opened).toBe(false); + + held.release(); + yield* halting; + expect(temporary.startsWith(`${metadataOf(slot)}.`)).toBe(true); + }); + + // It waited for delivery, and closed what was delivered. + expect(returned).toBe(true); + expect(opened).toBe(true); + expect(closed).toBe(true); + // And published nothing: a sidecar is renamed into place only by a write + // that finished. + expect(yield* exists(metadataOf(slot))).toBe(false); + }); + + it("E5: waits for a pending isolation directory and removes what it delivers", function* () { + const made: string[] = []; + const asked = withResolvers(); + const held = gate(); + const create = (prefix: string): Promise => { + asked.resolve(); + return held.released + .then(() => mkdtemp(prefix)) + .then((directory) => { + made.push(directory); + return directory; + }); + }; + const host = denoRepositoryHost({ directories: { create } }); + + let returned = false; + yield* scoped(function* (): Operation { + const taking = yield* spawn(function* (): Operation { + yield* host.useDirectory(); + yield* suspend(); + }); + yield* asked.operation; + const halting = yield* spawn(function* (): Operation { + yield* taking.halt(); + returned = true; + }); + yield* settled(); + // The creation is still pending, and teardown has not finished over it. + expect(returned).toBe(false); + expect(made).toEqual([]); + + held.release(); + yield* halting; + }); + + expect(returned).toBe(true); + // It waited for the directory to arrive, and removed it. + expect(made.length).toBe(1); + expect(yield* exists(made[0] ?? "")).toBe(false); + }); +}); + +/** The canonical common Git directory of one managed repository of this fixture. */ +function* commonOf(fixture: Fixture, name: string): Operation { + const slot = repositorySlot(fixture.managed, fixture.origin, name); + return yield* until(realpath(join(slot, "checkout", ".git"))); +} + +/** Every orphan metadata sibling this slot holds. */ +function* temporarySiblings(slot: string): Operation { + if (!(yield* exists(slot))) { + return []; + } + const entries = yield* until(readdir(slot)); + return entries.filter((entry) => entry.startsWith("metadata.json.")).sort(); +} + +/** The isolation directories this host's sessions have left in the temporary root. */ +function* isolationDirectories(): Operation { + const entries = yield* until(readdir(tmpdir())); + return entries.filter((entry) => entry.startsWith("xmd-workflow-git-")).sort(); +} diff --git a/scripts/runtime-test-exclusions.ts b/scripts/runtime-test-exclusions.ts index 10f67676f..a584e260d 100644 --- a/scripts/runtime-test-exclusions.ts +++ b/scripts/runtime-test-exclusions.ts @@ -682,6 +682,12 @@ const DENO_ONLY_REPOSITORY_PROVIDER: RuntimeExclusion[] = [ "the subject is a REPL entry's native repository host: it discovers a real checkout with a `git` subprocess and records what a real filesystem answered. The record format and the admission are provider-neutral and are proven on every runtime by packages/git/tests/retained-repository-records.test.ts", issue: DERIVED_SCOPE, }, + { + path: "packages/git/tests/retained-repository-selection.test.ts", + reason: + "the subject is a REPL entry's managed selection: it clones and adds worktrees with real `git` children, holds their slots under a kernel-released exclusive advisory lock taken through the Deno runtime, and interrupts those children to observe their close. The placement record and its admission are provider-neutral and are proven on every runtime by packages/git/tests/retained-repository-records.test.ts; that a runtime without the provider refuses a managed selector before it reaches a clone is proven by packages/cli/tests/repl-repositories-pty.test.ts, which runs everywhere", + issue: DERIVED_SCOPE, + }, { path: "packages/git/tests/retained-repository-lifecycle.test.ts", reason: diff --git a/scripts/tests/repl-repositories-compiled.test.ts b/scripts/tests/repl-repositories-compiled.test.ts index 6145c1259..582d16c78 100644 --- a/scripts/tests/repl-repositories-compiled.test.ts +++ b/scripts/tests/repl-repositories-compiled.test.ts @@ -22,19 +22,42 @@ import { exists } from "@effectionx/fs"; import type { Operation } from "effection"; import { fileURLToPath as fromFileUrl } from "node:url"; import { join } from "node:path"; -import { replFilesJourney } from "../../packages/cli/tests/support/repl-files-journey.ts"; +import { + replFilesJourney, + replRepositoryJourney, +} from "../../packages/cli/tests/support/repl-files-journey.ts"; const ROOT = fromFileUrl(new URL("../..", import.meta.url)); const BINARY = join(ROOT, "dist", "xmd"); +/** How this suite invokes the compiled `xmd`. */ +function compiled(args: readonly string[]): string { + return [shellQuote(BINARY), ...args].join(" "); +} + +/** Refuse the row rather than skip it: a missing binary is missing evidence. */ +function* requireBinary(): Operation { + if (!(yield* exists(BINARY))) { + throw new Error( + `this row drives the compiled binary and ${BINARY} is not there. Run \`deno task build\` ` + + "first; a skipped row is missing evidence rather than a passing behavior.", + ); + } +} + describe("the compiled binary's REPL entry files", () => { it("E1: carries the entry host, so a completed prefix answers again", function* (): Operation { - if (!(yield* exists(BINARY))) { - throw new Error( - `this row drives the compiled binary and ${BINARY} is not there. Run \`deno task build\` ` + - "first; a skipped row is missing evidence rather than a passing behavior.", - ); - } - yield* replFilesJourney((args) => [shellQuote(BINARY), ...args].join(" ")); + yield* requireBinary(); + yield* replFilesJourney(compiled); + }); + + it("E1: carries the managed selectors, so a Repository answers again", function* (): Operation { + yield* requireBinary(); + yield* replRepositoryJourney(compiled, "repository"); + }); + + it("E1: carries the managed selectors, so a Worktree's path routes a Dir", function* (): Operation { + yield* requireBinary(); + yield* replRepositoryJourney(compiled, "worktree"); }); }); diff --git a/specs/executable-mdx-spec.md b/specs/executable-mdx-spec.md index 84f243feb..e902765d3 100644 --- a/specs/executable-mdx-spec.md +++ b/specs/executable-mdx-spec.md @@ -10086,8 +10086,8 @@ bytes. The contracts below describe the two providers a document can reach. A document execution reaches repository operations only through the provider the command installed inside the execution scope, before the root document is -imported. There are two, and they differ in lifetime and permission rather than -in what an author writes. +imported. There are three, and they differ in lifetime and permission rather +than in what an author writes. The **ordinary provider** is what the Deno source entrypoint and the compiled binary install for `xmd run`. Constructing it acquires nothing at all: no @@ -10128,7 +10128,22 @@ The **retained provider** is what a workflow host installs inside its Workspace attachment. Its operations are durable effects keyed by the WorkflowRun, so a completed one restores from the journal without contacting anything. -Node and Bun install neither. They register the same declarations (§5.3), and +The **entry provider** is what the Deno source entrypoint and the compiled +binary install for one executing `xmd repl` entry, registered by the selected +REPL host rather than by the bundled Plugin and established by core +initialization inside the scope that entry owns. It answers `` and +`` by calling the *ordinary* selector above, with that selector's own +creation, reuse, conflict and interrupted-adoption policy, and it records what +the selector answered: a completed selection is answered from its record without +calling a selector, cloning, adding a worktree, reading a sidecar, opening a +credential or taking a lock, and only genuinely new work meets the current +filesystem. It installs nothing else the ordinary provider carries — no local +Git, no publication, no issue tracker and no external worker — so those +components refuse there exactly as they do under a host that installs no +provider for them. Constructing it acquires nothing, including no managed root, +and a passive reading of a settled history initializes nothing at all. + +Node and Bun install none of them. They register the same declarations (§5.3), and every repository operation there reports an absent provider before a lock, a credential, a subprocess or a request exists. @@ -14083,6 +14098,22 @@ user's own `~/.xmd/repositories`. | ORC20 | Retained workflow regression | Repository and Worktree replay, transactional Git, Push and pull-request history evidence, Issue effects, forks and completed replay keep their records, identities, provider call counts and native-launch refusal unchanged; directory ensure adds only its own `workspace_file` event and resulting retained root | | ORC21 | Compiled binary | A compiled smoke creates a root-level ambient Worktree, runs a command there, proves `.git` is a file and the checkout persists after exit; a second gated process proves lock refusal and release | +### Tier ERC — Repository composition inside a REPL entry (§5.3, §8.1) + +Bounded to what one executing entry retains about the checkouts it selected. +Everything an ordinary run proves about selection itself stays in Tier ORC: the +rows below are about the record and the holds around it. + +| # | Test | Verify | +|---|------|--------| +| ERC1 | The public journey | A real-terminal source and compiled `xmd repl` each run a Repository-only and a Repository-plus-Worktree entry with ``, write, delete and read, leave at the question, have the files changed from a shell, reopen the printed location, answer, and run the suffix once; the Worktree journey uses the self-closing captured path with a sibling ``, and a new entry selecting the same placement reads what is there now | +| ERC2 | Completed selection reaches nothing | An unfinished prefix holding acknowledged Repository and Worktree selections and file results replays with zero selector, clone, `worktree add`, sidecar, credential, lock and document-filesystem calls, after a managed checkout's bytes change and the selected worktree is removed; a new read of a missing file reports its ordinary live failure, and one ordinary selector refusal replays without native work | +| ERC3 | Position, not name | Nested Repository, Worktree and another Repository restore the enclosing owner and working directory; a captured self-closing path routes a later ``; two equal requests consume their own records; two spawned branches keep their records coroutine-local and resume in reverse order; two concurrent requests for one slot take one acquisition and serialize creation without suppressing either record | +| ERC4 | Two holds, two jobs | A managed slot one entry holds refuses another entry's new selection until teardown; a restored placement records a new read under a held shared-Git-directory lease with zero opens of that lock, while a new mutation refuses before any result is written and succeeds after release | +| ERC5 | Interruption | A clone and a `worktree add` each interrupted while their child is genuinely running join that child and its descendant before the entry ends, leaving no placement, no body sentinel and no orphan sidecar; what a `worktree add` did write is accounted for by the ordinary interrupted-creation policy rather than recreated or deleted; a metadata open and an isolation-directory creation each cut while still pending are joined before teardown finishes, and what they deliver is closed and removed | +| ERC6 | Acknowledgement, not completion | A native placement completed before its append is cancelled leaves no record and the continuation runs live; one acknowledged before cancellation bypasses the selector; a rejected placement append preserves the first persistence failure with no body after it | +| ERC7 | Admission and ceiling | Malformed, unknown-version, misnamed and request-mismatched placement records refuse before native work; an admitted generated `` inside a managed body is retained on the same terms as an authored one while a generated ``, `` or `` still refuses preflight | +| ERC8 | Unsupported runtimes | A REPL on a runtime that installs no entry provider performs ordinary file and directory work and reports an absent Repository provider before any clone, creating no managed root | --- diff --git a/specs/repl-spec.md b/specs/repl-spec.md index 869dff948..884690f5c 100644 --- a/specs/repl-spec.md +++ b/specs/repl-spec.md @@ -824,7 +824,70 @@ hold, and a completed operation of any kind takes nothing at all. Starting outside a checkout retains none of this and changes nothing else: the entry performs ordinary file work, exactly as it would anywhere else, and never -looks for a repository twice. +looks for a repository twice. Selecting a Repository is a separate matter and +works from anywhere, which is the next section. + +### The repositories and worktrees an entry selected + +An entry may also work in a checkout it asks for rather than the one you are +standing in: + +```xmd + + + + first + + Recorded value: {held} + continued + + +``` + +`` selects a persistent checkout beneath +`~/.xmd/repositories`, creating it the first time and reusing it afterwards. +`` selects a linked checkout of the Repository +in scope, or — written outside one, in a directory inside a Git checkout — of +the repository you opened the session in. Written outside one with no checkout +around you, it refuses with the same actionable sentence it has always had: run +from inside a checkout, or write a `` around it. Both forms behave +as they do anywhere else: paired, they scope their content and the working +directory; self-closing, they render nothing and return the checkout path, which +a sibling `` may then render at. Who creates, reuses or refuses a managed +checkout, and in what words, is unchanged — it is the same selection an ordinary +`xmd run` performs. + +What is new is that the selection is retained. Leave the entry at the question +above and come back to the printed location: the Repository and the Worktree +answer from what they recorded. No checkout is cloned, no `worktree add` runs, no +sidecar is read, no credential is opened and no lock is taken to answer them. +The file work inside them answers from its own records exactly as the ambient +section describes, and the write after the question happens for the first time +when you answer. + +A completed selection says where the entry was working. It is not a claim that +the checkout is still there. Remove it between leaving and coming back and the +entry still continues — the selection answers, the completed reads and writes +answer, and nothing is recreated or repaired to make that true. Only *new* work +meets today's filesystem: a new read of a file that is not there fails exactly as +it would have the first time, and a new `` creates the directory it names, +because `` creates directories and is not checkout validation. A new entry +selecting the same Repository selects it afresh, which is an ordinary reuse of +whatever is on disk. + +Two holds answer two different questions, and a message tells you which one you +met. One entry at a time *owns* a managed checkout: a second entry asking for +the same Repository or Worktree is refused immediately, naming the checkout, and +may have it once the first entry has been torn down. Separately, one entry at a +time *changes* one repository, which is the shared-Git-directory hold the +ambient section describes — a new `` takes it, because creating a +linked checkout writes into administration every checkout of that repository +shares, and a new write, deletion or directory takes it too. Neither hold is +taken to answer a completed selection or a completed file operation, and a read +takes only the ownership of the checkout it is reading in. + +Local Git, publication and issues remain unavailable here; the exclusions below +say which components those are. ## What the head says the execution is doing @@ -874,8 +937,11 @@ recorded output replaces it. Repository behavior belongs to the entry that is executing, and to nothing wider. The command selects it; the entry's own task prepares it, once, inside that task, and everything it takes — the Git session it reads through, where it -looked for a checkout, the hold on that repository — is released when the entry -is torn down. Each entry reads and writes its own range of the one history, so +looked for a checkout, the managed checkouts it selected, the hold on each of +those and the hold on the repository it changes — is released when the entry is +torn down. The selections an entry made are its own: a continuation rebuilds +which checkouts it was working in from what it recorded, and rebuilding them +reaches nothing. Each entry reads and writes its own range of the one history, so the same text typed twice is two sets of records rather than one replayed, and the entry after an interrupted one replays nothing of it however identical its text. No second name for the entry appears in a record: which history an entry @@ -912,11 +978,21 @@ schedules no timer. do not open the same location twice for writing. The hold on a checkout is a different thing: it says one entry at a time may mutate one repository, and it says nothing about who may write this execution's history. -- Repository components are not part of this. ``, ``, the - local Git operations, `` and `` refuse in a REPL entry - exactly as they do under any host that installs no provider for them. What an - entry retains is file and directory work in the checkout it was started in, - and nothing is managed, cloned, staged, committed or published. +- Local Git and publication are not part of this. ``, ``, + ``, ``, `` and `` refuse in a REPL + entry exactly as they do under any host that installs no provider for them. + What an entry retains is `` and `` selection and the + file and directory work inside those checkouts and the one it was started in; + nothing is staged, committed, pushed or published. +- Generated selection is unchanged. An admitted generated `` read, write + or `` is retained exactly as an authored one is, because both + cross the one document filesystem. `` stays ordinary and is retained by + nothing here, authored or generated. A generated ``, `` + or `` refuses before any provider is asked. +- Nothing inventories the managed root. There is no listing of the checkouts an + entry or a session has selected, no snapshot of their contents, and no + restoration pass; a managed checkout is somebody's work and nothing here + deletes one. - Continuing an entry is explicit. Returning to the live head starts the one unfinished entry, once; a frozen position starts nothing at all. - A continuation does not reconcile. Nothing compares a saved result with what diff --git a/specs/workflow-workspace-spec.md b/specs/workflow-workspace-spec.md index 386d7ddfb..149908941 100644 --- a/specs/workflow-workspace-spec.md +++ b/specs/workflow-workspace-spec.md @@ -130,22 +130,33 @@ The interactive REPL is specified in `specs/repl-spec.md` rather than here. One distinction belongs in this document, because it is about mutation replay. A REPL entry performs ordinary file work against the caller's own filesystem, -like `xmd run`. What differs is that the host retains the writes, deletions and -`` ensures the entry performed in the ambient Git checkout it was started -in, so an interrupted entry can be continued: a completed operation answers from -its record and touches nothing, and only an operation with no record executes. +like `xmd run`. What differs is that the host retains the reads, writes, +deletions and `` ensures the entry performed in a checkout it is working in +— the ambient Git checkout it was started in, or a managed one it selected — so +an interrupted entry can be continued: a completed operation answers from its +record and touches nothing, and only an operation with no record executes. Nothing is compared with the live filesystem, nothing is rolled back and no frontier is advanced — a recorded result and what is on disk may disagree, and the person reading the screen is what resolves that. +`` and `` are available to a REPL entry and are retained +on the same terms. The selector is the ordinary managed one §6 describes, with +its own creation, reuse, conflict and interrupted-adoption policy; what the +entry adds is a small routing record of what the selector answered, so a +completed selection is restored without calling a selector, cloning, adding a +worktree, reading a sidecar, opening a credential or taking a lock. A completed +selection says where the entry was working and nothing about what that checkout +now holds, so a continuation works after the checkout has been changed or +removed, while new work meets the current filesystem. Nothing is inventoried, +snapshotted or restored, and no managed checkout is deleted. + So the three environments retain mutation differently, and none of the other two changes. `xmd run` journals no filesystem mutation at all. A workflow run's mutations are Workspace effects inside a transaction, ordered by the frontier §9 and §10 describe, and the REPL has none of that: no Workspace, no -transaction, no frontier and no logical filesystem. Repository components — -``, ``, the local Git operations, `` and -`` — are not available to a REPL entry and refuse there as they do -under any host that installs no provider for them. +transaction, no frontier and no logical filesystem. The local Git operations, +`` and `` are not available to a REPL entry and refuse there +as they do under any host that installs no provider for them. ## 3. Workflow lifecycle @@ -942,6 +953,14 @@ outside a lexical `` belongs to the ambient Repository the command was run in. Nothing there is retained, replayed or forkable, and none of the durability this section states applies to it. +A REPL entry selects through that same ordinary selector, and records the answer +rather than the checkout (§2.3). The hold lasts for the entry rather than for +one document execution, so a later `` inside the same entry is still +protected, and a completed selection takes no hold at all — the first genuinely +new file operation inside a restored placement reacquires it. None of the +durability this section states applies there either: there is still no +Workspace, no transaction and no fork. + What both share is the composition value a component observes: a Repository selection carries a provider-minted identifier, a display name, the credential-free repository identity and the selected checkout path, and grants @@ -1126,10 +1145,13 @@ only cwd. It neither selects nor replaces the Repository in scope, so a Git operation inside a directory beneath a selected checkout still addresses that same Repository. -Under the interactive `xmd repl` the ensure is retained where the target is -inside the checkout the entry was started in, so continuing an interrupted entry -answers a completed ensure from its record and asks the provider for nothing -(`specs/repl-spec.md`). Everything stated above is unchanged by that: the ensure +Under the interactive `xmd repl` the ensure is retained where the target is in a +checkout the entry is working in — the one it was started in, or a managed +placement it selected, the managed checkout root itself included — so continuing +an interrupted entry answers a completed ensure from its record and asks the +provider for nothing (`specs/repl-spec.md`). The root counting is what makes the +self-closing Worktree and sibling `` pairing above retained there: the +`` stands at exactly the checkout the selection named. Everything stated above is unchanged by that: the ensure still precedes the cwd installation and every child, the refusal is still fixed and sanitized, the cwd is still restored on success, failure and cancellation, and `` still neither selects nor replaces the Repository in scope. A @@ -3540,6 +3562,7 @@ fetch operation requires its own language and durability contract. | ``, `` and `` composition under a workflow run | Repository and Worktree built by #293, Deno provider only; ``'s mandatory `API.Files.ensureDirectory` is specified by #643 as one transactional `workspace_file` mutation followed by lexical cwd installation | | the same twelve declarations under every runtime | built by #643: one shadowable array declared by the bundled Git Plugin and consumed wherever its profile is assembled — `xmd syntax`, `xmd plan`, an ordinary document execution and a workflow action that executes a document. `` left that array when canonical core took the name, and is present under every runtime without it | | ``, `` and the ambient Repository under an ordinary run | built by #643, Deno and compiled only: managed checkouts under `~/.xmd/repositories` with version 1 sidecars and execution-owned non-blocking locks, and the checkout the command was run in as the default Repository. Node and Bun install no operational provider | +| `` and `` inside a REPL entry | built on the #181 stack, Deno and compiled only: the ordinary managed selector above, with what it answered retained as a small routing record so a completed selection restores without a selector, a clone, a `worktree add`, a sidecar read, a credential or a lock — and only new work meets the current filesystem. One entry at a time owns a managed slot, held until the entry is torn down and reacquired lazily by the first new operation inside a restored placement; that hold is distinct from the shared Git directory's mutation lease. Nothing is inventoried, snapshotted, restored or deleted, and Node and Bun install no operational provider | | local Git operations and `Git.Push` evidence under an ordinary run | built by #643, Deno and compiled only: the same authored transitions with no transaction and no replay, commits recorded under the invoking user's own captured Git identity with an actionable refusal when the host can name none, and a private per-execution Push evidence entry that authorizes `` and crosses no run | | `` and pull-request reads under an ordinary run | built by #643, Deno and compiled only: the same transports and ceilings with no durable envelope, keyed by this execution's own invocation identity | | transactional Git components (`Git.Switch`, `Git.Add`, `Git.Commit`) | built by #294, Deno provider only |