From aac3a54cc02aef279543ce89d28ffb3c8af4094c Mon Sep 17 00:00:00 2001 From: willbot Date: Mon, 2 Nov 2026 11:17:26 +0100 Subject: [PATCH 1/2] Run prisma dev's converges through alchemy/Alchemist in the prisma process prisma dev, and the dev operation, now lower the app against the local target and run Alchemy's plan and apply in the prisma process, for the first converge and every rebuild. No .prisma-composer/dev//alchemy.run.ts is written and no Alchemy child is spawned. destroy keeps the child. - The app's @prisma/composer/internal/converge also exports localTargetProviders and devState; CONVERGE_CONTRACT is 2. - Each converge has its own scope, so the state lease is held only while it runs. Rebuilds stay one at a time and coalesce. - Converge progress reaches the user as converge-resource and converge-message dev events; prisma dev renders them as steps and messages, with console stdout sent to stderr for the session. - A failed first converge is DEV.CONVERGE_FAILED with the reasons; a failed rebuild is a converge-failed event with the cause, keeping the app. No stack file path or reproduce command. - DevInput.signal interrupts a running converge: the first one fails with DEV.INTERRUPTED (prisma dev exits 130); stop() interrupts a rebuild's converge and waits for it before stopping services. - The app is imported once per process, so an edit to the entry module or a service's authoring module is reported (definition-changed) and rebuilds pause until dev restarts. - Deleted generate-dev-stack.ts, family/dev-converge-child.ts, the orphaned fake-child fixture, and renderDevStackFile from the testing entry. DEV.STACK_WRITE_FAILED becomes DEV.STATE_WRITE_FAILED. - ADR-0064 extended to dev; deploy-cli.md, local-dev.md, guides, the skill and the amended ADRs updated. Co-Authored-By: Claude Opus 5.5 Signed-off-by: willbot Signed-off-by: Will Madden --- docs/design/03-domain-model/glossary.md | 2 +- docs/design/10-domains/deploy-cli.md | 8 +- docs/design/10-domains/local-dev.md | 38 +- ...-alchemy-through-a-generated-stack-file.md | 2 +- ...-containers-are-an-extension-descriptor.md | 2 +- ...path-is-the-programmatic-deploy-surface.md | 2 +- ...l-envelopes-with-dotted-namespace-codes.md | 12 +- ...s-the-composer-section-of-prisma-config.md | 2 +- ...vice-with-its-build-adapters-dev-server.md | 2 + ...s-come-from-resolve-and-config-adapters.md | 2 +- ...ADR-0064-deploy-runs-alchemy-in-process.md | 40 +- docs/design/90-decisions/README.md | 14 +- docs/guides/deploying.md | 4 +- docs/guides/running-locally.md | 17 +- .../cli/src/__tests__/app-converge.test.ts | 31 +- .../src/__tests__/converge-in-process.test.ts | 3 +- .../fixtures/stack-import/stub-composer.ts | 8 +- .../src/__tests__/stack-file-import.test.ts | 41 +- .../3-tooling/cli/src/app-converge.ts | 29 +- .../3-tooling/cli/src/converge-contract.ts | 11 +- .../dev/__tests__/generate-dev-stack.test.ts | 165 ------ .../cli/src/dev/generate-dev-stack.ts | 131 ----- .../exports/__tests__/control-import.test.ts | 2 +- .../3-tooling/cli/src/exports/converge.ts | 1 + .../3-tooling/cli/src/exports/testing.ts | 8 +- .../__tests__/dev-converge-child.test.ts | 116 ---- .../src/family/__tests__/dev-converge.test.ts | 178 ++++++ .../__tests__/dev-emulators-dir.test.ts | 11 +- .../src/family/__tests__/fake-child.test.ts | 83 --- .../family/__tests__/fixtures/fake-child.mjs | 59 -- .../3-tooling/cli/src/family/commands/dev.ts | 201 ++++--- .../cli/src/family/dev-converge-child.ts | 94 ---- .../operations/__tests__/operations.test.ts | 522 ++++++++++++++---- .../3-tooling/cli/src/operations/dev.ts | 32 +- .../src/operations/execute-deploy-destroy.ts | 22 +- .../cli/src/operations/execute-dev.ts | 318 ++++++----- .../3-tooling/cli/src/operations/shared.ts | 22 +- .../3-tooling/cli/src/run-alchemy.ts | 13 +- .../lowering/src/state/empty-scope.ts | 4 +- .../composer-cli/src/exports/testing.ts | 4 +- skills/prisma-composer-core-concepts/SKILL.md | 30 +- .../test/local-dev-store.integration.ts | 128 +---- .../integration/test/local-dev.integration.ts | 183 +++--- 43 files changed, 1202 insertions(+), 1395 deletions(-) delete mode 100644 packages/0-framework/3-tooling/cli/src/dev/__tests__/generate-dev-stack.test.ts delete mode 100644 packages/0-framework/3-tooling/cli/src/dev/generate-dev-stack.ts delete mode 100644 packages/0-framework/3-tooling/cli/src/family/__tests__/dev-converge-child.test.ts create mode 100644 packages/0-framework/3-tooling/cli/src/family/__tests__/dev-converge.test.ts delete mode 100644 packages/0-framework/3-tooling/cli/src/family/__tests__/fake-child.test.ts delete mode 100644 packages/0-framework/3-tooling/cli/src/family/__tests__/fixtures/fake-child.mjs delete mode 100644 packages/0-framework/3-tooling/cli/src/family/dev-converge-child.ts diff --git a/docs/design/03-domain-model/glossary.md b/docs/design/03-domain-model/glossary.md index 7370245c6..2dc0f2039 100644 --- a/docs/design/03-domain-model/glossary.md +++ b/docs/design/03-domain-model/glossary.md @@ -421,7 +421,7 @@ to a plan. ### Converge -Making an environment match the lowered stack: Alchemy plans what to create, update, replace and delete, then applies it. `prisma deploy` converges in its own process through Alchemy's programmatic API; `prisma dev` and `destroy` converge in an Alchemy child process. The *app converge* is the app install's `lower` and in-process converge, which `deploy` loads from `@prisma/composer/internal/converge` ([ADR-0064](../90-decisions/ADR-0064-deploy-runs-alchemy-in-process.md)). Codes: `DEV.CONVERGE_FAILED`, `DEPLOY.CONVERGE_RUNNING`, `DEPS.CONVERGE_UNRESOLVABLE`, `DEPS.CONVERGE_CONTRACT_MISMATCH`. +Making an environment match the lowered stack: Alchemy plans what to create, update, replace and delete, then applies it. `prisma deploy` and `prisma dev` converge in their own process through Alchemy's programmatic API; `destroy` converges in an Alchemy child process. The *app converge* is the app install's `lower` and in-process converge, with the local target's providers and state store for dev, which `deploy` and `dev` load from `@prisma/composer/internal/converge` ([ADR-0064](../90-decisions/ADR-0064-deploy-runs-alchemy-in-process.md)). Codes: `DEV.CONVERGE_FAILED`, `DEV.INTERRUPTED`, `DEPLOY.CONVERGE_RUNNING`, `DEPS.CONVERGE_UNRESOLVABLE`, `DEPS.CONVERGE_CONTRACT_MISMATCH`. ### Control plane / Execution plane diff --git a/docs/design/10-domains/deploy-cli.md b/docs/design/10-domains/deploy-cli.md index 52abc1c48..1a9d78337 100644 --- a/docs/design/10-domains/deploy-cli.md +++ b/docs/design/10-domains/deploy-cli.md @@ -96,9 +96,9 @@ that relative imports inside the entry graph may use `./x.js` or extensionless bun APIs can only deploy under bun, since loading the graph imports that module — the app's choice, not a CLI limit. -**Where Alchemy runs.** `prisma deploy` runs Alchemy in the `prisma` process, through Alchemy's programmatic API (`alchemy/Alchemist`) ([ADR-0064](../90-decisions/ADR-0064-deploy-runs-alchemy-in-process.md)). It loads core's `lower()` and the converge from `@prisma/composer/internal/converge`, resolved from the directory of the app's `prisma.config.ts`, so they use the app's own install of `alchemy` and `effect`, the same one its extensions and providers come from. The `prisma` host's Composer calls that install, so the two must speak the same converge contract, a number the install exports from `@prisma/composer/internal/converge-contract`; their package versions may differ. A different contract fails with `DEPS.CONVERGE_CONTRACT_MISMATCH`, naming both contracts and both versions and saying which side to upgrade. An app whose `@prisma/composer` cannot be resolved, or predates that entry, fails with `DEPS.CONVERGE_UNRESOLVABLE`. Both fail before anything is created. Alchemy and `effect` load only behind the deploy operation's lazy executor import, so `prisma --version` never loads them. +**Where Alchemy runs.** `prisma deploy` and `prisma dev` run Alchemy in the `prisma` process, through Alchemy's programmatic API (`alchemy/Alchemist`) ([ADR-0064](../90-decisions/ADR-0064-deploy-runs-alchemy-in-process.md)). It loads core's `lower()` and the converge from `@prisma/composer/internal/converge`, resolved from the directory of the app's `prisma.config.ts`, so they use the app's own install of `alchemy` and `effect`, the same one its extensions and providers come from. The `prisma` host's Composer calls that install, so the two must speak the same converge contract, a number the install exports from `@prisma/composer/internal/converge-contract`; their package versions may differ. A different contract fails with `DEPS.CONVERGE_CONTRACT_MISMATCH`, naming both contracts and both versions and saying which side to upgrade. An app whose `@prisma/composer` cannot be resolved, or predates that entry, fails with `DEPS.CONVERGE_UNRESOLVABLE`. Both fail before anything is created. Alchemy and `effect` load only behind the operations' lazy executor imports, so `prisma --version` never loads them. `prisma dev` loads the local target's providers and state store from the same entry, and converges once at start and once per rebuild, one at a time ([local-dev.md](local-dev.md) § Pipeline). -**The Alchemy child's runtime (`destroy` and `prisma dev`).** These still start an Alchemy child. Composer runs the `bin` of the `alchemy` +**The Alchemy child's runtime (`destroy`).** `destroy` still starts an Alchemy child. Composer runs the `bin` of the `alchemy` package that the app's `@prisma/composer` depends on, found from the app directory the way the generated stack file's own imports are, so the CLI and the stack code come from one install. It starts that bin with Node, as the @@ -192,7 +192,7 @@ user can rerun it by hand (ADR-0007). - **Process state.** Alchemy reads its configuration only from `process.env` and `.env` in the working directory. For the length of the plan and apply, `deploy` sets the service token, the workspace id and each reporter's variables in `process.env`, and changes to the deploy's working directory when the process is elsewhere. It restores all of them on every exit path. One process runs one converge at a time, across every install it loads (`DEPLOY.CONVERGE_RUNNING`), and a worker thread cannot change directory (`DEPLOY.WORKER_THREAD`). - **Credentials.** Alchemy needs a service token. The engine gives a command no way to read a `prisma login` session's token, so the command passes a token only when the active credential came from `PRISMA_SERVICE_TOKEN`, with that credential's workspace. The operation checks for it before preflight and before any container is ensured, and fails with `DEPLOY.SERVICE_TOKEN_REQUIRED` without one. - `destroy` (and `prisma dev`) write the pipeline's results as a runnable stack module at `.prisma-composer/alchemy.run.ts` and drive the `alchemy` CLI against it (ADR-0007), started as § Runtime describes, carrying every resolved container across to the child as one environment variable per extension — content the CLI writes but never reads (ADR-0038). `alchemy destroy` re-imports and re-evaluates the same stack, so its target reconstruction needs the containers. The generated file and Alchemy's state live in the process's working directory — tool state lives where you run the tool, like any other CLI (ADR-0004). The generated stack file reads the config pool again, from the same adapters; it never reads the secrets pool. + `destroy` writes the pipeline's results as a runnable stack module at `.prisma-composer/alchemy.run.ts` and drive the `alchemy` CLI against it (ADR-0007), started as § Runtime describes, carrying every resolved container across to the child as one environment variable per extension — content the CLI writes but never reads (ADR-0038). `alchemy destroy` re-imports and re-evaluates the same stack, so its target reconstruction needs the containers. The generated file and Alchemy's state live in the process's working directory — tool state lives where you run the tool, like any other CLI (ADR-0004). The generated stack file reads the config pool again, from the same adapters; it never reads the secrets pool. The pass that assembles a service is the same pass that lowers it, so the correlation between services and their built bundles never exists as @@ -217,7 +217,7 @@ inputs carry the same `environment`. ([ADR-0038](../90-decisions/ADR-0038-containers-are-an-extension-descriptor.md)). The CLI resolves every configured extension's container before Alchemy runs (pipeline step 8) and holds each result as an opaque value. -- **Containers reach the lowering as values.** `deploy` hands each resolved container to `lower()` directly. For `destroy` and `prisma dev`, which start an Alchemy child, each container crosses to the child as one environment variable, named from the extension's id and holding whatever that extension's own `serialize()` produced; the child reads it back and calls the same extension's `deserialize`. The transport is content-blind: the CLI never reads the value, and an extension never touches `process.env` directly — it receives its container as a parameter everywhere. Preflight payloads travel the same two ways and reach the lowering as `LowerContext.preflight`. +- **Containers reach the lowering as values.** `deploy` and `prisma dev` hand each resolved container to `lower()` directly. For `destroy`, which starts an Alchemy child, each container crosses to the child as one environment variable, named from the extension's id and holding whatever that extension's own `serialize()` produced; the child reads it back and calls the same extension's `deserialize`. The transport is content-blind: the CLI never reads the value, and an extension never touches `process.env` directly — it receives its container as a parameter everywhere. Preflight payloads travel the same two ways and reach the lowering as `LowerContext.preflight`. - **State rides the container.** `PrismaAppConfig.state` names its owning extension; core hands that extension's own resolved container to `state.create()`, so the state layer is built from it rather than from the diff --git a/docs/design/10-domains/local-dev.md b/docs/design/10-domains/local-dev.md index ef6d42708..b3c639664 100644 --- a/docs/design/10-domains/local-dev.md +++ b/docs/design/10-domains/local-dev.md @@ -107,7 +107,7 @@ and hooks receive, and it is Alchemy's stage, so Alchemy keeps each copy's state at `.alchemy/state//`. Every other local file that belongs to one copy lives in `.prisma-composer/dev//`: the record of instances, the environment variable values, the fingerprint key -(ADR-0058), the generated stack file, the artifacts and the bucket objects. The folder name is +(ADR-0058), the artifacts and the bucket objects. The folder name is the environment name, encoded, because an environment name may contain `/`. Alchemy keeps the stage `feature/x` inside the folder of the stage `feature`, so `--fresh` removes only the files directly in its own stage's folder. @@ -162,12 +162,12 @@ Dev re-runs [deploy's pipeline](deploy-cli.md#the-pipeline) with these deltas: A recorded database the last session stopped is started here: on a warm start the converge may change nothing and never call the Database provider. -5. **Lower + converge** — a dev-generated stack file (ADR-0007's pattern, at - `.prisma-composer/dev//alchemy.run.ts`), driven with the extension's - `localTarget.providers()` layer and `devState()` (Alchemy's built-in file store, exported by `@prisma/composer/local-target`), - with the environment name as Alchemy's stage. Providers provision emulator instances (a running +5. **Lower + converge** — in the `prisma` process + ([ADR-0064](../90-decisions/ADR-0064-deploy-runs-alchemy-in-process.md)): `lower()` with the + extensions' `localTarget.providers()` layers (`localTargetProviders`) and `devState()` (Alchemy's built-in file store), all from the app's install of `@prisma/composer/internal/converge`, + then Alchemy's plan and apply with the environment name as Alchemy's stage. No stack file is written and no child is started. Providers provision emulator instances (a running service, a database, a bucket) by talking to the emulators; converge - terminates as always, and the Compute emulator keeps serving. The dev stack file reads both pools again and passes them to the local providers. + terminates as always, and the Compute emulator keeps serving. The local providers get both pools as values. Each rebuild converges the same way, one at a time; Alchemy's progress reaches the user through the dev operation's events. 6. **Attach** — new, dev-only: through `localTarget.attach`, render the front door (every service's endpoint) and watch for rebuilds, loop. Ctrl-C, SIGTERM or a failure stops the app's service processes and database @@ -433,9 +433,9 @@ instances had generated ids is discarded when it loads. ### Process lifetimes -1. **Converge-scoped** — the Alchemy child (ADR-0007), started as - [deploy-cli.md § Runtime](deploy-cli.md) describes. Providers run here; - nothing started here survives its exit. +1. **Converge-scoped** — each converge's scope, in the `prisma` process + (ADR-0064). Providers run here, and the state lease is held only while a + converge runs; nothing started here outlives the scope. 2. **Machine-scoped emulators** — one per node kind per registry root, multi-tenant, holding every long-lived process: the Compute emulator (service children), the Postgres emulator (one `@prisma/dev` server per @@ -550,7 +550,7 @@ The same table the port's hand-rolled script implemented, now standard: | minted needs (ADR-0030/0031: service keys, streams keys) | minted locally by the unchanged provisioners | | origin (ADR-0039) | `http://localhost:` via the unchanged origin channel | -Whether a missing secret is required is decided by validation, not by reading the schema (ADR-0042): the dev preflight validates each service's input with every missing secret left out, and then with each one left out alone, and names as required only the secrets whose absence alone fails. The preflight sends the alchemy process, on the preflight transport, the names of the absent secrets and an HMAC-SHA256 fingerprint of the value of each variable that a service's input or params point at, keyed by the environment's own `.prisma-composer/dev//fingerprint.key`. The serializer leaves out the absent secrets, and the fingerprint fills each deployment's `:updatedAt` trigger member, so a value changed between two runs redeploys exactly the services that read it. The local `Deployment` provider reads current values from the pools the dev stack file read and gives each service only the variables its own input and params point at. +Whether a missing secret is required is decided by validation, not by reading the schema (ADR-0042): the dev preflight validates each service's input with every missing secret left out, and then with each one left out alone, and names as required only the secrets whose absence alone fails. The preflight hands the lowering, as its payload, the names of the absent secrets and an HMAC-SHA256 fingerprint of the value of each variable that a service's input or params point at, keyed by the environment's own `.prisma-composer/dev//fingerprint.key`. The serializer leaves out the absent secrets, and the fingerprint fills each deployment's `:updatedAt` trigger member, so a value changed between two runs redeploys exactly the services that read it. The local `Deployment` provider reads current values from the pools dev read and gives each service only the variables its own input and params point at. To load `.env` files, list `dotenv()` in `config` and `secrets`, with `environmentVariables()` after it to keep the shell. It reads `.env`, `.env.local`, `.env.` and `.env..local` beside `prisma.config.ts`. Values in `.env.dev` are read only by the environment `dev`; values in `.env` and `.env.local` are read by every environment, `prisma deploy` to `production` included, whose preflight uploads any secret the environment does not have yet. `import 'dotenv/config'` at the top of `prisma.config.ts` still works with the default lists. @@ -579,6 +579,10 @@ Deploy's rule holds: every failure names its fix. | required secret unset or empty | hard error listing every missing variable, the service that needs it, the schema's message, and where an adapter in `secrets` reads it; an optional one is not an error | | env-sourced param unbound | hard error listing the missing names, deploy-preflight style | | an adapter returns different values at a rebuild than at start | that the values changed and rebuilds are paused until dev restarts | +| the entry module or a file a service is declared in changes while dev runs | which files changed, that dev cannot load the app's definition again, and that rebuilds are paused until dev restarts | +| the first converge fails | `DEV.CONVERGE_FAILED`, with Alchemy's errors in the message and in `meta.reasons`; `dev` stops the app's services before it returns the failure | +| a rebuild's converge fails | a warning with Alchemy's errors; the running app is kept and dev keeps watching | +| Ctrl-C during the first converge | `DEV.INTERRUPTED`, reported as a message; `prisma dev` exits 130 | | a service exits before it listens, or does not listen within the readiness timeout | the address, the port, the exit code if it exited, and the log path; `dev` stops the app's services before it returns the failure | | an emulator does not accept connections (retried briefly; an error response from the emulator is never retried) | `DEV.EMULATOR_UNREACHABLE`, with the emulator's URL, from `dev` and `log` alike | | `PRISMA_COMPOSER_SERVICE_READY_TIMEOUT_MS` is not a positive whole number | `DEV.SERVICE_READY_TIMEOUT_INVALID`, with the value | @@ -630,13 +634,13 @@ The design's founding claim was proven against open-chat — a real, pre-existin loop picks rebuilds up on its own. A build descriptor that still returns no `watch` entries reports `[dev]
has no watchable inputs` at startup and needs its rebuilds triggered manually. -- **A topology edit reconverges, but the session's own view of the app is - the one it started with.** The entry module is imported once per process, - so after an edit to `module.ts` or a `service.ts` the Alchemy child (a fresh - process) converges the new topology while the session keeps the graph, the - bundles and the dev servers it loaded. Edits that keep the set of services - and their build adapters — wiring, inputs, bindings — apply fully; adding, - removing or re-adapting a service needs `prisma dev` restarted. +- **An edit to the app's definition needs a restart.** The entry module is + imported once per process, and the converge runs in that process, so a + rebuild cannot see an edit to it. Dev hashes the entry module and each + service's declaring file when it starts; when one changes it says so and + pauses rebuilds until `prisma dev` is restarted (ADR-0064). An edit to + another file the entry imports, such as a sub-module's `module.ts`, is not + detected and takes effect at the next start. - **App-owned migrations are not run by `dev`** (by design — ADR-0022, spec § 4's `OrmMigration` line is for framework-run migrations only). An app that runs its own migrations (e.g. via `prisma db init`, like the diff --git a/docs/design/90-decisions/ADR-0007-deploy-drives-alchemy-through-a-generated-stack-file.md b/docs/design/90-decisions/ADR-0007-deploy-drives-alchemy-through-a-generated-stack-file.md index e0c6da95a..15b5498e6 100644 --- a/docs/design/90-decisions/ADR-0007-deploy-drives-alchemy-through-a-generated-stack-file.md +++ b/docs/design/90-decisions/ADR-0007-deploy-drives-alchemy-through-a-generated-stack-file.md @@ -1,6 +1,6 @@ # ADR-0007: Deploy drives Alchemy through a generated stack file -Superseded for `deploy` by [ADR-0064](ADR-0064-deploy-runs-alchemy-in-process.md): `deploy` runs Alchemy's plan and apply in the `prisma` process and writes no stack file. `destroy` and `prisma dev` still drive the Alchemy CLI through the generated file as described below. +Superseded for `deploy` and `prisma dev` by [ADR-0064](ADR-0064-deploy-runs-alchemy-in-process.md): both run Alchemy's plan and apply in the `prisma` process and write no stack file. `destroy` still drives the Alchemy CLI through the generated file as described below. Amended by [ADR-0050](ADR-0050-composer-runs-as-prisma-deploy-and-prisma-dev.md): the commands that generate and drive the stack file are `prisma deploy` and `prisma dev`; teardown is the `destroy` operation on `@prisma/composer/control`, which drives `alchemy destroy` against the same generated file. The `prisma-composer` binary is retired. The generated-file mechanism below is unchanged. diff --git a/docs/design/90-decisions/ADR-0038-containers-are-an-extension-descriptor.md b/docs/design/90-decisions/ADR-0038-containers-are-an-extension-descriptor.md index 4741ef7cd..f00a8ec3a 100644 --- a/docs/design/90-decisions/ADR-0038-containers-are-an-extension-descriptor.md +++ b/docs/design/90-decisions/ADR-0038-containers-are-an-extension-descriptor.md @@ -1,6 +1,6 @@ # ADR-0038: Containers are an extension descriptor -> Amended by [ADR-0064](ADR-0064-deploy-runs-alchemy-in-process.md): `deploy` runs Alchemy in the CLI's own process and hands each resolved container to `lower()` as a value (`LowerOptions.containers`). The environment-variable transport below remains for `destroy` and `prisma dev`, which still start an Alchemy child. +> Amended by [ADR-0064](ADR-0064-deploy-runs-alchemy-in-process.md): `deploy` and `prisma dev` run Alchemy in the CLI's own process and hand each resolved container to `lower()` as a value (`LowerOptions.containers`). The environment-variable transport below remains for `destroy`, which still starts an Alchemy child. > Amended (TML-3157): `ContainerInstance` gained an optional, framework-visible > `alchemyStage` — the deterministic Alchemy stage the container implies (the diff --git a/docs/design/90-decisions/ADR-0043-the-control-subpath-is-the-programmatic-deploy-surface.md b/docs/design/90-decisions/ADR-0043-the-control-subpath-is-the-programmatic-deploy-surface.md index aa9cbbac8..bef468220 100644 --- a/docs/design/90-decisions/ADR-0043-the-control-subpath-is-the-programmatic-deploy-surface.md +++ b/docs/design/90-decisions/ADR-0043-the-control-subpath-is-the-programmatic-deploy-surface.md @@ -1,6 +1,6 @@ # ADR-0043: `@prisma/composer/control` is the programmatic deploy surface -Amended by [ADR-0064](ADR-0064-deploy-runs-alchemy-in-process.md): `deploy` runs Alchemy in the calling process, so the report hook returns the `DeploymentSummary` as a value; `deploy` sets no `PRISMA_COMPOSER_DEPLOYMENT_RESULT_FILE`. `deploy` also takes an optional `signal` that interrupts the plan or apply. +Amended by [ADR-0064](ADR-0064-deploy-runs-alchemy-in-process.md): `deploy` runs Alchemy in the calling process, so the report hook returns the `DeploymentSummary` as a value; `deploy` sets no `PRISMA_COMPOSER_DEPLOYMENT_RESULT_FILE`. `deploy` also takes an optional `signal` that interrupts the plan or apply, and so does `dev`, whose signal interrupts a running converge (2026-11-02). Amended by [ADR-0049](ADR-0049-composers-configuration-is-the-composer-section-of-prisma-config.md): each operation takes a required `config: { value, file }`, and the effect-conflict diagnosis is retired. Amended by [ADR-0050](ADR-0050-composer-runs-as-prisma-deploy-and-prisma-dev.md): the `prisma-composer` binary is retired; the `prisma` CLI renders two of the four operations, as `prisma deploy` and `prisma dev`, and `destroy` and `log` are called from scripts. The surface below is otherwise unchanged. diff --git a/docs/design/90-decisions/ADR-0044-errors-are-structural-envelopes-with-dotted-namespace-codes.md b/docs/design/90-decisions/ADR-0044-errors-are-structural-envelopes-with-dotted-namespace-codes.md index 0b564b863..da7263683 100644 --- a/docs/design/90-decisions/ADR-0044-errors-are-structural-envelopes-with-dotted-namespace-codes.md +++ b/docs/design/90-decisions/ADR-0044-errors-are-structural-envelopes-with-dotted-namespace-codes.md @@ -1,6 +1,6 @@ # ADR-0044: Errors are structural envelopes with dotted namespace codes -Amended by [ADR-0064](ADR-0064-deploy-runs-alchemy-in-process.md): `deploy` starts no Alchemy child, so the child exit-status passthrough below applies only to `prisma dev` and `destroy`. A failed `deploy` plan or apply is `DEPLOY.ENGINE_FAILED` and exits 2. `DEPLOY.INTERRUPTED`, `DEPLOY.SERVICE_TOKEN_REQUIRED`, `DEPLOY.CONVERGE_RUNNING`, `DEPLOY.WORKER_THREAD`, `DEPS.CONVERGE_UNRESOLVABLE` and `DEPS.CONVERGE_CONTRACT_MISMATCH` are added. An interrupted `prisma deploy` reports `DEPLOY.INTERRUPTED` as a message and ends with the engine's `CLI.ABORTED`, exiting 130 (143 for SIGTERM). +Amended by [ADR-0064](ADR-0064-deploy-runs-alchemy-in-process.md): `deploy` and `prisma dev` start no Alchemy child, so the child exit-status passthrough below applies only to `destroy`. A failed first `prisma dev` converge is `DEV.CONVERGE_FAILED` and exits 2; `DEV.INTERRUPTED` and `DEV.STATE_WRITE_FAILED` are added and `DEV.STACK_WRITE_FAILED` is retired. A failed `deploy` plan or apply is `DEPLOY.ENGINE_FAILED` and exits 2. `DEPLOY.INTERRUPTED`, `DEPLOY.SERVICE_TOKEN_REQUIRED`, `DEPLOY.CONVERGE_RUNNING`, `DEPLOY.WORKER_THREAD`, `DEPS.CONVERGE_UNRESOLVABLE` and `DEPS.CONVERGE_CONTRACT_MISMATCH` are added. An interrupted `prisma deploy` reports `DEPLOY.INTERRUPTED` as a message and ends with the engine's `CLI.ABORTED`, exiting 130 (143 for SIGTERM). Amended 2026-10-23: the `CONFIG` registry gains ENTRY_MISSING, for a command or operation that has no entry module from the config or an override. @@ -106,20 +106,18 @@ Closed subcode registry: *(STAGE_INVALID, STAGE_UNVALIDATABLE and SCOPE_MISSING retired by [ADR-0059](ADR-0059-a-deployment-environment-is-called-an-environment.md))*. ENVIRONMENT_REQUIRED: a `destroy` call names no environment. - ENGINE_FAILED: Alchemy failed. `deploy` runs Alchemy in process and exits 2; `destroy` and `prisma dev` pass the Alchemy child's exit status through (see § Exit codes). + ENGINE_FAILED: Alchemy failed. `deploy` runs Alchemy in process and exits 2; `destroy` passes the Alchemy child's exit status through (see § Exit codes). INTERRUPTED: the abort signal stopped `deploy`'s plan or apply. `prisma deploy` exits 130 (143 for SIGTERM): it reports this code as a message event and ends with the engine's abort, `CLI.ABORTED`. SERVICE_TOKEN_REQUIRED: `deploy`'s credential is not a service token, such as a `prisma login` session, or there is none in the environment ([ADR-0064](ADR-0064-deploy-runs-alchemy-in-process.md)). Temporary: it is retired when `@prisma/cli-engine` gives a command the active access token (the in-process credentials need, `needs: { credentials: 'process' }`, asked of `@prisma/cli-engine`), so a session can deploy. CONVERGE_RUNNING: another in-process converge holds this process's environment and working directory. WORKER_THREAD: `deploy` was called from a worker thread with a `cwd` other than the process's working directory, which a worker cannot change. - `prisma dev` starts the Alchemy child the same way `deploy` does, so it - raises ALCHEMY_BIN_MISSING and NODE_MISSING under `DEPLOY` too. -- `DEV` — PLATFORM_UNSUPPORTED, TARGET_UNSUPPORTED, CONTAINER_FAILED, TEARDOWN_FAILED, PREFLIGHT_FAILED, EMULATOR_FAILED, SERVICE_START_FAILED, CONVERGE_FAILED, ATTACH_FAILED, STACK_WRITE_FAILED, EMULATORS_DIR_INVALID, EMULATOR_UNREACHABLE, SERVICE_READY_TIMEOUT_INVALID, NOT_STARTED_HERE, ADDRESS_UNKNOWN, NOT_RUNNING. EMULATOR_UNREACHABLE is raised by the emulator client when it cannot connect, so `log` raises it under `DEV` too; for a connection failure it takes the place of SERVICE_START_FAILED, ATTACH_FAILED and `LOG.ATTACH_FAILED`. `devUrl` also raises it when the emulator a node runs on is not running. NOT_STARTED_HERE (no `prisma dev` has recorded an app in the directory) is raised by `devUrl` and `devStop`; ADDRESS_UNKNOWN (the recorded app has no node at that address; the known addresses are listed) and NOT_RUNNING (the node exists but is not running; its state is named, and the reason and fix depend on it) by `devUrl` alone ([ADR-0053](ADR-0053-dev-url-dev-stop-and-emulator-list.md)). Both operations raise EMULATOR_FAILED when the local target fails without a code. `emulator list` fails only with EMULATORS_DIR_INVALID; an emulator it cannot list is reported in that emulator's entry with EMULATOR_UNREACHABLE or EMULATOR_FAILED. +- `DEV` — PLATFORM_UNSUPPORTED, TARGET_UNSUPPORTED, CONTAINER_FAILED, TEARDOWN_FAILED, PREFLIGHT_FAILED, EMULATOR_FAILED, SERVICE_START_FAILED, CONVERGE_FAILED, INTERRUPTED, ATTACH_FAILED, STATE_WRITE_FAILED, EMULATORS_DIR_INVALID, EMULATOR_UNREACHABLE, SERVICE_READY_TIMEOUT_INVALID, NOT_STARTED_HERE, ADDRESS_UNKNOWN, NOT_RUNNING. EMULATOR_UNREACHABLE is raised by the emulator client when it cannot connect, so `log` raises it under `DEV` too; for a connection failure it takes the place of SERVICE_START_FAILED, ATTACH_FAILED and `LOG.ATTACH_FAILED`. `devUrl` also raises it when the emulator a node runs on is not running. NOT_STARTED_HERE (no `prisma dev` has recorded an app in the directory) is raised by `devUrl` and `devStop`; ADDRESS_UNKNOWN (the recorded app has no node at that address; the known addresses are listed) and NOT_RUNNING (the node exists but is not running; its state is named, and the reason and fix depend on it) by `devUrl` alone ([ADR-0053](ADR-0053-dev-url-dev-stop-and-emulator-list.md)). CONVERGE_FAILED: the first converge failed; Alchemy's errors are in the message and in `meta.reasons`. INTERRUPTED: the abort signal stopped the first converge; `prisma dev` reports it as a message and exits 130. STATE_WRITE_FAILED: the environment's folder under `.prisma-composer/dev/` could not be written *(STACK_WRITE_FAILED retired by [ADR-0064](ADR-0064-deploy-runs-alchemy-in-process.md))*. Both operations raise EMULATOR_FAILED when the local target fails without a code. `emulator list` fails only with EMULATORS_DIR_INVALID; an emulator it cannot list is reported in that emulator's entry with EMULATOR_UNREACHABLE or EMULATOR_FAILED. - `LOG` — PLATFORM_UNSUPPORTED, ATTACH_FAILED, ADDRESS_UNKNOWN - `DEPS` — EXECUTOR_UNLOADABLE, EXTENSION_INCOMPATIBLE, CONVERGE_UNRESOLVABLE, CONVERGE_CONTRACT_MISMATCH *(EFFECT_VERSION_CONFLICT retired by [ADR-0049](ADR-0049-composers-configuration-is-the-composer-section-of-prisma-config.md))*. EXTENSION_INCOMPATIBLE: an installed extension is older than the Composer running the deploy or destroy, so the container it resolved, rebuilt through its own `deserialize`, does not name the environment that was asked for. - CONVERGE_UNRESOLVABLE: `deploy` cannot load `@prisma/composer/internal/converge` from the app's own install, or that install predates it ([ADR-0064](ADR-0064-deploy-runs-alchemy-in-process.md)). + CONVERGE_UNRESOLVABLE: `deploy` or `dev` cannot load `@prisma/composer/internal/converge` from the app's own install, or that install predates it ([ADR-0064](ADR-0064-deploy-runs-alchemy-in-process.md)). CONVERGE_CONTRACT_MISMATCH: the app's `@prisma/composer` speaks another converge contract than the `prisma` CLI's Composer; the message names both contract numbers and both package versions, and the fix says which side to upgrade. Different package versions with the same contract are accepted. Adding a subcode is an edit to this list; adding a namespace is an amendment to @@ -132,7 +130,7 @@ governs statuses the CLI itself generates: `0` OK, `1` internal error/bug ONLY, `2` expected failure (usage errors and structured failures alike), `3` user abort, `130`/`143` signals. -**Documented exception** (`destroy` and `prisma dev`; `deploy` runs Alchemy in process since [ADR-0064](ADR-0064-deploy-runs-alchemy-in-process.md)): when the spawned deploy engine (alchemy) exits +**Documented exception** (`destroy`; `deploy` and `prisma dev` run Alchemy in process since [ADR-0064](ADR-0064-deploy-runs-alchemy-in-process.md)): when the spawned deploy engine (alchemy) exits nonzero, the CLI prints the two reproduce-hint lines and passes the child's own exit status through unchanged — the child's status is the operator's signal and renumbering it would erase information. A passthrough status is the child's diff --git a/docs/design/90-decisions/ADR-0049-composers-configuration-is-the-composer-section-of-prisma-config.md b/docs/design/90-decisions/ADR-0049-composers-configuration-is-the-composer-section-of-prisma-config.md index fb413acf3..f14f4c0a4 100644 --- a/docs/design/90-decisions/ADR-0049-composers-configuration-is-the-composer-section-of-prisma-config.md +++ b/docs/design/90-decisions/ADR-0049-composers-configuration-is-the-composer-section-of-prisma-config.md @@ -1,6 +1,6 @@ # ADR-0049: Composer's configuration is the `composer` section of `prisma.config.ts` -Amended by [ADR-0064](ADR-0064-deploy-runs-alchemy-in-process.md): `deploy` no longer starts an Alchemy child or imports the declaring file in one, so the child-process reason below now holds for `destroy` and `prisma dev` only. The decision is unchanged: the section still comes from one file, and `file` is still returned. +Amended by [ADR-0064](ADR-0064-deploy-runs-alchemy-in-process.md): `deploy` and `prisma dev` no longer start an Alchemy child or import the declaring file in one, so the child-process reason below now holds for `destroy` only. `dev` still reads the section once and pauses on an edit to the declaring file, because it keeps the config it started with. The decision is unchanged: the section still comes from one file, and `file` is still returned. Amended by [ADR-0062](ADR-0062-environment-values-come-from-resolve-and-config-adapters.md): the section gains `config` (and later `secrets`), each an array of adapters. diff --git a/docs/design/90-decisions/ADR-0056-prisma-dev-runs-a-service-with-its-build-adapters-dev-server.md b/docs/design/90-decisions/ADR-0056-prisma-dev-runs-a-service-with-its-build-adapters-dev-server.md index 0dfbdbcc6..c15d893e9 100644 --- a/docs/design/90-decisions/ADR-0056-prisma-dev-runs-a-service-with-its-build-adapters-dev-server.md +++ b/docs/design/90-decisions/ADR-0056-prisma-dev-runs-a-service-with-its-build-adapters-dev-server.md @@ -1,5 +1,7 @@ # ADR-0056: `prisma dev` runs a service with its build adapter's dev server +Amended by [ADR-0064](ADR-0064-deploy-runs-alchemy-in-process.md) (2026-11-02): `prisma dev` converges in its own process and writes no dev stack file; `unassembled` is passed to `lower()` as a value. An edit to an authoring module no longer reconverges: the app is imported once, so dev reports the edit and pauses rebuilds until it is restarted. Edits to built output, and the serve step after every successful reconverge, are unchanged. + Amended 2026-10-23: `prisma dev` takes no positional entry; the entry is `entry` in the `composer` section of `prisma.config.ts`, or `--entry ` ([ADR-0050](ADR-0050-composer-runs-as-prisma-deploy-and-prisma-dev.md)). ## Decision diff --git a/docs/design/90-decisions/ADR-0062-environment-values-come-from-resolve-and-config-adapters.md b/docs/design/90-decisions/ADR-0062-environment-values-come-from-resolve-and-config-adapters.md index 1002665c4..f670de7de 100644 --- a/docs/design/90-decisions/ADR-0062-environment-values-come-from-resolve-and-config-adapters.md +++ b/docs/design/90-decisions/ADR-0062-environment-values-come-from-resolve-and-config-adapters.md @@ -1,6 +1,6 @@ # ADR-0062: A value that differs per environment is computed with `resolve()`; values from outside the code come through config adapters -Amended by [ADR-0064](ADR-0064-deploy-runs-alchemy-in-process.md): `deploy` runs Alchemy in its first process and passes the config pool to `lower()` as a value, so it starts no second process and writes no configuration file. What § Which commands run the adapters says about a second process now holds for `prisma dev` and `destroy` only. +Amended by [ADR-0064](ADR-0064-deploy-runs-alchemy-in-process.md): `deploy` runs Alchemy in its first process and passes the config pool to `lower()` as a value, so it starts no second process and writes no configuration file. What § Which commands run the adapters says about a second process now holds for `destroy` only: `prisma dev` converges in its first process too, and passes both pools to the local providers as values. ```ts // module.ts diff --git a/docs/design/90-decisions/ADR-0064-deploy-runs-alchemy-in-process.md b/docs/design/90-decisions/ADR-0064-deploy-runs-alchemy-in-process.md index 06c7fa144..751ec7e69 100644 --- a/docs/design/90-decisions/ADR-0064-deploy-runs-alchemy-in-process.md +++ b/docs/design/90-decisions/ADR-0064-deploy-runs-alchemy-in-process.md @@ -1,4 +1,4 @@ -# ADR-0064: `deploy` runs Alchemy in the `prisma` process +# ADR-0064: `deploy` and `prisma dev` run Alchemy in the `prisma` process ## Decision @@ -16,11 +16,11 @@ const plan = yield* Alchemist.plan({ stack, operation: 'deploy', stage, updateSt yield* Alchemist.apply(plan); ``` -This supersedes [ADR-0007](ADR-0007-deploy-drives-alchemy-through-a-generated-stack-file.md) for `deploy`. `destroy` and `prisma dev` keep the generated stack file and the Alchemy child until they move. +This supersedes [ADR-0007](ADR-0007-deploy-drives-alchemy-through-a-generated-stack-file.md) for `deploy` and, since the amendment below, for `prisma dev`. `destroy` keeps the generated stack file and the Alchemy child until it moves or is deleted. - **The app's install runs the converge.** The executor loads `lower` and `convergeInProcess` (together, the app converge) from `@prisma/composer/internal/converge`, resolved from the directory of the app's `prisma.config.ts`, as the old child path resolved Alchemy's bin. The extension descriptors, providers and state store come from that install through `prisma.config.ts`, and the `prisma` host pins its own Composer version ([ADR-0050](ADR-0050-composer-runs-as-prisma-deploy-and-prisma-dev.md)). Taking the converge from the host's copy would put two copies of `alchemy` and `effect` in one process whenever the two versions differ; taking it from the app's install keeps one. An app whose `@prisma/composer` is missing or predates the entries fails with `DEPS.CONVERGE_UNRESOLVABLE` before anything is created. -- **Host and app speak one converge contract.** The host's operation code calls the app's `lower` and converge and reads back their results, so both sides must agree on the shapes between them: `LowerOptions`, `ConvergeRequest`, `ConvergeOutcome`, `ConvergeEvent` and `DeploymentResult`. They need not run the same package version. The `prisma` host pins its Composer and ships on its own schedule (ADR-0050), and create-prisma pins the app's, so the two versions differ between releases, when an app keeps an older Composer, and on dev builds; an exact version match would refuse every deploy in those cases. So the app's install exports a `CONVERGE_CONTRACT` integer from `@prisma/composer/internal/converge-contract`, an entry that imports nothing. The host reads it before loading anything else and refuses a different number with `DEPS.CONVERGE_CONTRACT_MISMATCH`, which names both contract numbers and both package versions and says which side to upgrade. The number is bumped only when one of those shapes changes in a way the other side cannot read. -- **Values, not environment variables.** `lower()` takes the resolved containers and the preflight payloads as required values (`LowerOptions.containers`, `LowerOptions.preflight`), and each extension reads its own payload as `LowerContext.preflight`. Only the generated stack files for `destroy` and `prisma dev` read the environment transports, through one core helper, `transportedLowerInputs(config)`. This amends [ADR-0038](ADR-0038-containers-are-an-extension-descriptor.md) for `deploy`: no container crosses a process boundary. +- **Host and app speak one converge contract.** The host's operation code calls the app's `lower` and converge and reads back their results, so both sides must agree on the shapes between them: `LowerOptions`, `ConvergeRequest`, `ConvergeOutcome`, `ConvergeEvent` and `DeploymentResult`, and on which functions the entry exports. They need not run the same package version. The `prisma` host pins its Composer and ships on its own schedule (ADR-0050), and create-prisma pins the app's, so the two versions differ between releases, when an app keeps an older Composer, and on dev builds; an exact version match would refuse every deploy in those cases. So the app's install exports a `CONVERGE_CONTRACT` integer from `@prisma/composer/internal/converge-contract`, an entry that imports nothing. The host reads it before loading anything else and refuses a different number with `DEPS.CONVERGE_CONTRACT_MISMATCH`, which names both contract numbers and both package versions and says which side to upgrade. The number is bumped only when one of those shapes changes in a way the other side cannot read. +- **Values, not environment variables.** `lower()` takes the resolved containers and the preflight payloads as required values (`LowerOptions.containers`, `LowerOptions.preflight`), and each extension reads its own payload as `LowerContext.preflight`. Only the generated stack file for `destroy` reads the environment transports, through one core helper, `transportedLowerInputs(config)`. This amends [ADR-0038](ADR-0038-containers-are-an-extension-descriptor.md) for `deploy` and `prisma dev`: no container crosses a process boundary. - **The summary is a value.** The report hook runs inside the apply, in the same process, so `deploy` returns the `DeploymentSummary` directly. This amends [ADR-0043](ADR-0043-the-control-subpath-is-the-programmatic-deploy-surface.md): `PRISMA_COMPOSER_DEPLOYMENT_RESULT_FILE` and the failure-cause file are not used by `deploy`. - **Progress is engine step events.** Alchemy's typed events become steps under `deploy.apply`, one per resource, and its log lines become messages. Alchemy's console sink is replaced and `SigilRenderer` is not used, so human and JSON output keep one format and nothing but the engine writes to stdout. - **Failures are structured.** The converge returns the errors Alchemy failed with as typed reasons (tag and message), and the deploy maps them to `DEPLOY.ENGINE_FAILED`, with Composer's advice in place of the Alchemy CLI's and secrets removed. A converge that rejects instead of returning an outcome is `DEPLOY.ENGINE_FAILED` too. An interrupted one is `DEPLOY.INTERRUPTED`. Every message event, and an error raised while the stack is built, is redacted the same way. This amends [ADR-0044](ADR-0044-errors-are-structural-envelopes-with-dotted-namespace-codes.md): the child exit-status passthrough no longer applies to `deploy`; its failures exit 2, and Ctrl-C exits 130. @@ -29,6 +29,29 @@ This supersedes [ADR-0007](ADR-0007-deploy-drives-alchemy-through-a-generated-st - **One converge per process.** Because of that process state, a second converge while one runs is refused with `DEPLOY.CONVERGE_RUNNING`. The flag lives on `globalThis` under a `Symbol.for` key, so it holds across every install of the entry a host loads, such as two apps deployed from one script; and a call from a worker thread whose `cwd` is not the process's working directory, which a worker cannot change, with `DEPLOY.WORKER_THREAD`. - **Stdout stays the engine's.** Alchemy's log lines and events go to the caller as values, and Alchemy's non-interactive `Interaction` writes to stderr. The `prisma deploy` command, not the operation, sends `console`'s stdout methods to stderr while the operation runs, so a provider's `console.log` cannot corrupt JSON output; a host calling the operation keeps its own `console`, as ADR-0043 requires. A provider that writes to `process.stdout` directly is out of reach. +## Amendment: `prisma dev` (2026-11-02) + +`prisma dev`, and the `dev` operation, run every converge in the process too: the first one and each rebuild's. Dev writes no `.prisma-composer/dev//alchemy.run.ts` and starts no Alchemy child. + +```ts +const { lower, convergeInProcess, localTargetProviders, devState } = await import(/* the app's @prisma/composer/internal/converge */); +const stack = lower(root, config, { + name, environment, configPool, bundles, unassembled, containers, preflight, + providers: localTargetProviders(resolved, containers, devDir, pools), + state: devState(), +}); +await convergeInProcess({ operation: 'deploy', stack, stage: environment, cwd, env: {}, signal, onEvent }); +``` + +- **The same converge, with the local target.** Dev lowers against Composer's local target providers and Alchemy's local file store, not Alchemy's `dev: true` mode ([ADR-0041](ADR-0041-local-dev-runs-the-deploy-pipeline-against-local-providers.md)). Both functions come from the app's install, with `lower` and `convergeInProcess`, so the local providers share the app's copy of `alchemy` and `effect`. The entry now exports them, so `CONVERGE_CONTRACT` is 2. The converge sets no environment variables: dev needs no credential, and the local providers call no platform. +- **One converge at a time, each in its own scope.** Each converge closes its scope before it returns, so the state lease is held only while it runs. Rebuilds stay one at a time, with changes during a rebuild coalesced into one more, which also keeps dev within the one-converge-per-process rule above. +- **Progress through the dev event stream.** The operation emits `converge-resource` and `converge-message` events. `prisma dev` renders each resource as a step (`dev.converge.
`) and each log line as a message, and sends `console`'s stdout methods to stderr for the whole session. +- **Failures are structured.** A failed first converge is `DEV.CONVERGE_FAILED`, with the reasons in `meta.reasons`; an interrupted one is `DEV.INTERRUPTED`. A failed rebuild emits `converge-failed` with the cause and keeps the running app. Neither carries a stack file path or a reproduce command. The ADR-0044 child exit-status passthrough no longer applies to `prisma dev`: a failed first converge exits 2. +- **Ctrl-C.** The `dev` operation takes a `signal`. Ctrl-C during the first converge interrupts it; `prisma dev` reports `DEV.INTERRUPTED` as a message and exits 130. During a rebuild, stopping the session interrupts the converge and waits for it to end, and so for its lease to be released, before it stops the services. Ctrl-C while idle stops dev as before. +- **An edit to the app's definition asks for a restart.** Before, a rebuild re-ran the pipeline in the `prisma` process, which re-assembled every artifact but re-imported the entry from Node's or Bun's module cache, so its graph never changed. The Alchemy child then imported `prisma.config.ts` and the app afresh, so an edit to the topology took effect through the child, lowered against artifacts assembled from the old graph. In process there is only the graph imported at start, and neither runtime can drop a module from its cache. So dev hashes the files it already watches as authoring modules (the entry module and each service's `build.module` file) when it starts. When one of them changes, the session emits `definition-changed`, which `prisma dev` shows as a warning asking for a restart, and pauses rebuilds, as it does for an edit to `prisma.config.ts` (`config-changed`) or a change in what the adapters return (`values-changed`). An edit to an artifact takes effect as before. An edit to another file the entry imports (a sub-module's `module.ts`) is not detected; before, it took effect at the next rebuild, and now it takes effect at the next start. This amends [ADR-0056](ADR-0056-prisma-dev-runs-a-service-with-its-build-adapters-dev-server.md), where a topology edit reconverged. + +Rejected for the stale graph: importing the entry again under a new query string (it reloads only the entry, not the modules it imports, and keeps every old copy in memory); a resolve hook that tags the app's files with a generation (Node only, since Bun has no such hook, and it also keeps every old copy); and a child process or worker per converge (what this decision removes). + ## Reasoning ADR-0007 chose a generated file and the Alchemy CLI because Alchemy had no programmatic API, and because a file a user can rerun bisects a framework bug from an Alchemy bug. Alchemy now has one (`alchemy/Alchemist`, from commit `972f294`). With it, the generated file is a detour: Composer wrote TypeScript, found Alchemy's bin, spawned it, and passed containers, preflight payloads and the result through environment variables and files, only for the child to import the config again and rebuild what the parent already held. In process, the parent hands Alchemy the stack value and gets typed results and typed events back. @@ -37,8 +60,9 @@ The bisection argument weakens with the API. A failed deploy now reports Alchemy ## Consequences -- `deploy` no longer resolves Alchemy's bin or needs Node on PATH under Bun. `DEPLOY.ALCHEMY_BIN_MISSING`, `DEPLOY.NODE_MISSING` and `DEPLOY.STACK_WRITE_FAILED` remain for `destroy` and `prisma dev`. -- New codes: `DEPLOY.INTERRUPTED`, `DEPLOY.SERVICE_TOKEN_REQUIRED` (temporary), `DEPLOY.CONVERGE_RUNNING`, `DEPLOY.WORKER_THREAD`, `DEPS.CONVERGE_UNRESOLVABLE` and `DEPS.CONVERGE_CONTRACT_MISMATCH`. +- `deploy` and `prisma dev` no longer resolve Alchemy's bin or need Node on PATH under Bun. `DEPLOY.ALCHEMY_BIN_MISSING`, `DEPLOY.NODE_MISSING` and `DEPLOY.STACK_WRITE_FAILED` remain for `destroy`. +- New codes: `DEPLOY.INTERRUPTED`, `DEPLOY.SERVICE_TOKEN_REQUIRED` (temporary), `DEPLOY.CONVERGE_RUNNING`, `DEPLOY.WORKER_THREAD`, `DEPS.CONVERGE_UNRESOLVABLE` and `DEPS.CONVERGE_CONTRACT_MISMATCH`; for dev, `DEV.INTERRUPTED`, and `DEV.STATE_WRITE_FAILED` (the environment's folder could not be written) in place of `DEV.STACK_WRITE_FAILED`. +- `prisma dev` no longer declares `maySpawn`. `generate-dev-stack.ts`, the dev command's child spawn and settlement (`family/dev-converge-child.ts`), and `renderDevStackFile` on `@prisma/composer-cli/testing` are deleted. - The converge relies on two Alchemy modules outside `alchemy/Alchemist`: `alchemy/Util/LogRecord` (to read log records) and `alchemy/Interaction` (to send interaction output to stderr). - `prisma deploy` declares `needs: { credentials: true }` and no longer `maySpawn`. - Setting `process.env` and the working directory for the length of the apply is process-wide state, so one process runs one converge at a time. The upstream asks below remove the need. @@ -53,7 +77,7 @@ The bisection argument weakens with the API. A failed deploy now reports Alchemy ## Related -- Amends: [ADR-0007](ADR-0007-deploy-drives-alchemy-through-a-generated-stack-file.md) (superseded for `deploy`), [ADR-0038](ADR-0038-containers-are-an-extension-descriptor.md), [ADR-0043](ADR-0043-the-control-subpath-is-the-programmatic-deploy-surface.md), [ADR-0044](ADR-0044-errors-are-structural-envelopes-with-dotted-namespace-codes.md), [ADR-0049](ADR-0049-composers-configuration-is-the-composer-section-of-prisma-config.md) (its child-process reason narrows to `destroy` and `prisma dev`), [ADR-0062](ADR-0062-environment-values-come-from-resolve-and-config-adapters.md) (deploy has no second process). -- [ADR-0045](ADR-0045-deploy-state-lives-behind-the-platform-state-api.md), [ADR-0050](ADR-0050-composer-runs-as-prisma-deploy-and-prisma-dev.md) +- Amends: [ADR-0007](ADR-0007-deploy-drives-alchemy-through-a-generated-stack-file.md) (superseded for `deploy` and `prisma dev`), [ADR-0056](ADR-0056-prisma-dev-runs-a-service-with-its-build-adapters-dev-server.md) (an edit to the app's definition asks for a restart), [ADR-0038](ADR-0038-containers-are-an-extension-descriptor.md), [ADR-0043](ADR-0043-the-control-subpath-is-the-programmatic-deploy-surface.md), [ADR-0044](ADR-0044-errors-are-structural-envelopes-with-dotted-namespace-codes.md), [ADR-0049](ADR-0049-composers-configuration-is-the-composer-section-of-prisma-config.md) (its child-process reason narrows to `destroy`), [ADR-0062](ADR-0062-environment-values-come-from-resolve-and-config-adapters.md) (deploy and dev have no second process). +- [ADR-0041](ADR-0041-local-dev-runs-the-deploy-pipeline-against-local-providers.md), [ADR-0045](ADR-0045-deploy-state-lives-behind-the-platform-state-api.md), [ADR-0050](ADR-0050-composer-runs-as-prisma-deploy-and-prisma-dev.md) - [`../05-prisma-cloud/build-reporting.md`](../05-prisma-cloud/build-reporting.md) - [`../10-domains/deploy-cli.md`](../10-domains/deploy-cli.md) § The pipeline, step 9 diff --git a/docs/design/90-decisions/README.md b/docs/design/90-decisions/README.md index f7b312683..685fa3d64 100644 --- a/docs/design/90-decisions/README.md +++ b/docs/design/90-decisions/README.md @@ -28,7 +28,7 @@ _Earlier drafts (ADR-0001, ADR-0002) were retired as the high-level design settl - [ADR-0004](ADR-0004-paths-resolve-relative-to-the-authoring-file.md) — Paths resolve relative to the file that writes them; the build adapter carries the authoring module. - [ADR-0005](ADR-0005-users-build-the-framework-assembles.md) — Users build the app's code; the framework assembles the artifact by documented, deterministic steps (validate, wrap, each app-type's documented deploy step — e.g. Next's static/public copy). No guessing (arithmetic/depth-inference/discovery), no laundering (symlink = hard error); read the build tool's own manifest (Next's `relativeAppDir`), don't walk or compute. *(Superseded in part by ADR-0047: a symlink whose resolved target stays inside the bundle is preserved as a link; the no-guessing and no-dereferencing rules stand.)* - [ADR-0006](ADR-0006-every-node-is-named.md) — Every node is named; the root's name names the application. *(Amended by ADR-0050: the override is `prisma deploy --name`. Amended by ADR-0051: a node's name is its default provision ID, so changing any node's name changes its identity. Amended by ADR-0057: the root's name is the first segment of every address, so changing it changes every node's identity.)* -- [ADR-0007](ADR-0007-deploy-drives-alchemy-through-a-generated-stack-file.md) — Deploy drives Alchemy through a generated, inspectable stack file. *(Amended by ADR-0050: `prisma deploy` and `prisma dev` generate the file; teardown is the `destroy` operation.)* *(Superseded for `deploy` by ADR-0064: `deploy` runs Alchemy in process; `destroy` and `prisma dev` keep the file.)* +- [ADR-0007](ADR-0007-deploy-drives-alchemy-through-a-generated-stack-file.md) — Deploy drives Alchemy through a generated, inspectable stack file. *(Amended by ADR-0050: `prisma deploy` and `prisma dev` generate the file; teardown is the `destroy` operation.)* *(Superseded for `deploy` and `prisma dev` by ADR-0064: both run Alchemy in process; `destroy` keeps the file.)* - [ADR-0008](ADR-0008-wrapper-inlines-everything-except-runtime-builtins.md) — The boot wrapper inlines everything except runtime built-ins. - [ADR-0009](ADR-0009-deploy-state-is-hosted-in-the-workspace.md) — Deploy state is hosted in the workspace, not in local files. *(Superseded by ADR-0034: still hosted, now per-stage in the app's own Project.)* - [ADR-0010](ADR-0010-deploys-hold-a-session-advisory-lock.md) — Deploys hold a session advisory lock per stack and stage. *(Superseded by ADR-0045: the lease is server-side now; the fail-fast contention behavior survives.)* @@ -59,30 +59,30 @@ _Earlier drafts (ADR-0001, ADR-0002) were retired as the high-level design settl - [ADR-0035](ADR-0035-public-entrypoints-live-in-src-exports.md) — Public entrypoints live in `src/exports/` (one file per subpath; internals stay at the `src/` root); `@internal/tsdown-config` generates `package.json#exports` from object-named entries where safe, with two deliberate exceptions kept hand-maintained (the multi-pass `cron`/`storage`/`streams`, and the two published packages). Completes ADR-0028. - [ADR-0036](ADR-0036-the-rpc-kind-is-named-service-rpc.md) — The RPC kind is named **service RPC**: subpath `@prisma/composer/service-rpc`, unchanged call-site names (`rpc()`, `contract()`, `serve()`), kind brand stays `'rpc'`. Scope recorded in connection-contracts.md: edges internal to the application topology, agent-generatable by design — not an application API layer, not general distributed-systems infrastructure. - [ADR-0037](ADR-0037-service-rpc-calls-carry-an-idempotency-key.md) — The generated service RPC client carries an `Idempotency-Key` on every call — one per logical call, reused across a bounded retry — and the provider deduplicates on it: one call per key, replaying completed 2xx/4xx answers (never 5xx) from a bounded in-process store. A keyless request (a hand-rolled or older caller) is served once without deduplication rather than rejected. Retrying is permanent protocol behavior, not a platform workaround, and there is no per-method opt-in — a flag would be an unverifiable claim. Handlers may read the key via an optional third argument for their own durable exactly-once. -- [ADR-0038](ADR-0038-containers-are-an-extension-descriptor.md) — Container lifecycle (ensure/locate/remove) is an optional `container` descriptor on `ExtensionDescriptor`, the same pattern as `preflight`/`teardown`; the resolved instance is opaque to core and crosses the CLI parent→alchemy child boundary as one framework-named environment variable per extension, via the extension's own `serialize()`/`deserialize()`. `StateDescriptor` names its owning extension so core can hand it that extension's resolved container. Deletes the `crossDomainExceptions` entry that let the CLI import `@internal/lowering` directly — `0-framework` imports nothing again. *(Amended by ADR-0064: `deploy` hands containers and preflight payloads to `lower()` as values; the environment transport remains for `destroy` and `prisma dev`.)* +- [ADR-0038](ADR-0038-containers-are-an-extension-descriptor.md) — Container lifecycle (ensure/locate/remove) is an optional `container` descriptor on `ExtensionDescriptor`, the same pattern as `preflight`/`teardown`; the resolved instance is opaque to core and crosses the CLI parent→alchemy child boundary as one framework-named environment variable per extension, via the extension's own `serialize()`/`deserialize()`. `StateDescriptor` names its owning extension so core can hand it that extension's resolved container. Deletes the `crossDomainExceptions` entry that let the CLI import `@internal/lowering` directly — `0-framework` imports nothing again. *(Amended by ADR-0064: `deploy` and `prisma dev` hand containers and preflight payloads to `lower()` as values; the environment transport remains for `destroy`.)* - [ADR-0039](ADR-0039-a-compute-services-own-origin-is-a-target-resolved-property.md) — A compute service's own platform-assigned origin is a target-resolved property, read as the service's `origin()` — never a declared param, never operator config, never in `config()`. It rides ADR-0031's reserved provider-param channel as the first *service-derived* entry (`valueForService(provisioned, address)`, written for every compute service, exposing or not), sourced from the provisioned App's `appEndpointDomain` (ADR-0048) — made trustworthy pre-promote by the upstream PRO-200 fix. `envParam(…)` remains correct for operator-known origins (custom domains); narrows ADR-0032's `appOrigin` example accordingly. - [ADR-0040](ADR-0040-the-orm-binding-carries-the-url-and-a-lazy-client.md) — `postgres(contract)`'s dependency binding is `{ url, client }`: the raw connection string plus the typed client, constructed lazily and memoized on first `client` access — `hydrate` builds nothing. The contract remains the compatibility interface (hash check and deploy-time migration unchanged, ADR-0022); the binding becomes a strict superset of plain `rawPostgres()`'s `{ url }`, so an app that owns its database client still gets framework-run migrations. Contract validation cost and failure move from `load()` (where one bad input poisoned every input, unattributed) to the first `client` access. Cross-kind satisfaction (`'postgres'` satisfying `'postgres'`) rejected in its favor. *(Extended by ADR-0055: the binding also carries `connection` and `clientWith(options)`.)* - [ADR-0041](ADR-0041-local-dev-runs-the-deploy-pipeline-against-local-providers.md) — `prisma-composer dev` runs the **same deploy pipeline** (Load → assemble → lower → Alchemy converge) against local implementations of the same Alchemy resource types, declared on an optional `localTarget` field of `ExtensionDescriptor` (a lazy thunk resolving a `LocalTargetDescriptor`; subpaths `@prisma/composer/local-target` and `@prisma/composer-prisma-cloud/local-target` — "dev" names only the user-facing command/prefix/state dir) (providers, container, preflight, emulators, attach, teardown — **no** `nodes`/`provisions`, so the lowering cannot diverge; no `state` either — dev uses Alchemy's own `localState()` through `LowerOptions.state`). The target runs **emulators per node kind**: Compute and buckets are machine-global, multi-tenant daemons (the Compute emulator owns the service child processes — deployment PUTs, crash supervision, logs; buckets serve the S3 wire over plain files on disk), while Postgres the Postgres emulator hosts one `@prisma/dev` server per database instance. Providers provision instances by communicating with the emulators during converge, and the dev command is a view through `attach`; `ServiceKey`/`S3Credentials`/`PgWarm`/`OrmMigration` are shared verbatim. Credential-free by requirement. Rejects a local Management API (reimplements another team's server-side semantics, drifts silently) and per-kind dev descriptors (an open-set parallel seam). *(Amended by ADR-0050: the command is `prisma dev`; logs are the `log` operation, not a command. Amended 2026-10-23: the entry is the `composer` section's `entry` or `--entry`, not a positional. Amended by ADR-0052: emulator instances are keyed by emulator-generated ids, not the app name, Composer maps each app's nodes to them per directory, and `prisma dev` stops the app's database servers when it ends.)* *(Amended by ADR-0056: a service runs the dev server its build adapter declares in place of its artifact, as a command deployment on the Compute emulator.)* *(Amended by ADR-0062: local dev runs as a named environment, `dev` by default.)* - [ADR-0042](ADR-0042-service-input-is-one-standard-schema.md) — A compute service declares its entire incoming configuration — config and secrets together — as one Standard Schema (`input`), read back through one typed accessor; `params`/`secrets` and `config()`/`secrets()` are replaced. The framework never introspects the schema (validate-only, per the spec): the operator's binding is the traversable structure (sourcing: literals, `envParam`, `envSecret`), the schema is the black-box judge of legality (invoked at deploy over the resolved binding with secrets as opaque `SecretString` boxes, and again at boot), and secretness is a leaf *type* enforced by validation in both directions. The wire format is one self-describing JSON document row per service with `$secret` pointers to platform variables; an env-bound key whose variable is unset resolves to key-omitted and the schema arbitrates absence — subsuming optional secrets and conditional config (`stripeId` only when `stripeEnabled`) without a framework DSL. *(Extended by ADR-0061 to modules.)* *(Amended 2026-10-30: the forwardable secret need is removed; a module takes a secret as an input field.)* *(Amended 2026-10-30, sixth slice: `field()` and `secretString()` ship from `@prisma/composer/arktype` and `@prisma/composer/zod`; `secretString()` accepts a secret pointer.)* *(Amended 2026-10-30, seventh slice: the deploy checks a service's schema with pointers in place; a field bound to `serviceOrigin()` or a plain `generatedParam()` is wrapped with `field()`; `service.input()` has no pointers.)* - [ADR-0043](ADR-0043-the-control-subpath-is-the-programmatic-deploy-surface.md) — `@prisma/composer/control` is the programmatic deploy surface: typed `deploy`/`destroy`/`dev`/`log` operations (structured inputs/results, no argv/console/exit) implemented in `@internal/cli`'s `src/operations/` and re-exported per ADR-0035; the CLI is a thin renderer over them. The entry's static graph stays import-light — each operation lazily imports its executor, so importing the subpath executes nothing, and a tree that cannot load the deploy stack surfaces as a structured `pipeline` failure — and `PRISMA_COMPOSER_DEPLOYMENT_RESULT_FILE` carries the deploy result across the process boundary: the alchemy child's report hook writes a serializable `DeploymentSummary` to the named file, the operation reads it back best-effort (absent/malformed = undefined summary, never a failure). Distinct from an extension's ADR-0017 `/control` entry. *(Amended by ADR-0049: each operation takes a required `config: { value, file }`, and the effect-conflict diagnosis is retired.)* *(Amended by ADR-0050: the `prisma` CLI renders two of the four operations, as `prisma deploy` and `prisma dev`; `destroy` and `log` are called from scripts.)* *(Amended by ADR-0053: `devUrl` and `devStop` join the operations.)* *(Amended by ADR-0059: `deploy` and `destroy` take `environment`, which `destroy` requires.)* *(Amended 2026-10-28: `devUrl` and `devStop` act on the app and environment in their input; `dev` and `log` take `environment`.)* *(Amended by ADR-0064: `deploy` returns the summary as a value; it uses no result file.)* -- [ADR-0044](ADR-0044-errors-are-structural-envelopes-with-dotted-namespace-codes.md) — Errors are structural envelopes with dotted `NAMESPACE.SUBCODE` codes (the shared prisma/prisma foundation, duplicated pending extraction): structured at origin with why/fix splits, no catch-all codes, bugs carry no code (exit 1 + report hint), recognition is structural (`CliStructuredError.is()`), operation results ride the shared `Result` `ok` discriminator, expected failures exit 2 — with the alchemy child-status passthrough as the documented exception. *(Amended by ADR-0049: `DEPS.EFFECT_VERSION_CONFLICT` is retired. Amended 2026-10-23: `CONFIG.ENTRY_MISSING` is added. Amended by ADR-0059: the `DEPLOY` codes say environment instead of stage, and `DEPS.EXTENSION_INCOMPATIBLE` is added.)* *(Amended 2026-10-30: `CONFIG.ADAPTER_FAILED` and `CONFIG.ADAPTER_INVALID` are added.)* *(Amended by ADR-0064: the child exit-status passthrough no longer applies to `deploy`, whose failures exit 2; `DEPLOY.INTERRUPTED`, `DEPLOY.SERVICE_TOKEN_REQUIRED` (temporary), `DEPLOY.CONVERGE_RUNNING`, `DEPLOY.WORKER_THREAD`, `DEPS.CONVERGE_UNRESOLVABLE` and `DEPS.CONVERGE_CONTRACT_MISMATCH` are added; an interrupted `prisma deploy` ends as `CLI.ABORTED` with exit 130.)* +- [ADR-0044](ADR-0044-errors-are-structural-envelopes-with-dotted-namespace-codes.md) — Errors are structural envelopes with dotted `NAMESPACE.SUBCODE` codes (the shared prisma/prisma foundation, duplicated pending extraction): structured at origin with why/fix splits, no catch-all codes, bugs carry no code (exit 1 + report hint), recognition is structural (`CliStructuredError.is()`), operation results ride the shared `Result` `ok` discriminator, expected failures exit 2 — with the alchemy child-status passthrough as the documented exception. *(Amended by ADR-0049: `DEPS.EFFECT_VERSION_CONFLICT` is retired. Amended 2026-10-23: `CONFIG.ENTRY_MISSING` is added. Amended by ADR-0059: the `DEPLOY` codes say environment instead of stage, and `DEPS.EXTENSION_INCOMPATIBLE` is added.)* *(Amended 2026-10-30: `CONFIG.ADAPTER_FAILED` and `CONFIG.ADAPTER_INVALID` are added.)* *(Amended by ADR-0064: the child exit-status passthrough no longer applies to `deploy`, whose failures exit 2; `DEPLOY.INTERRUPTED`, `DEPLOY.SERVICE_TOKEN_REQUIRED` (temporary), `DEPLOY.CONVERGE_RUNNING`, `DEPLOY.WORKER_THREAD`, `DEPS.CONVERGE_UNRESOLVABLE` and `DEPS.CONVERGE_CONTRACT_MISMATCH` are added; an interrupted `prisma deploy` ends as `CLI.ABORTED` with exit 130. For `prisma dev` too the passthrough no longer applies; `DEV.INTERRUPTED` and `DEV.STATE_WRITE_FAILED` are added and `DEV.STACK_WRITE_FAILED` is retired.)* - [ADR-0045](ADR-0045-deploy-state-lives-behind-the-platform-state-api.md) — Deploy state lives behind the platform state API (the Management API implements Alchemy's stock `HttpStateApi` wire contract per Branch; composer's state layer is Alchemy's stock HTTP client), and deploys hold a server-side per-`(stack, stage)` lease (TTL 60s, heartbeated, released on exit; contention fails fast naming the holder; state operations without a live lease fail 409). Supersedes ADR-0010 (lock → lease) and the storage half of ADR-0034 (Branch scoping and lifetime stand; the visible per-stage database is gone); closes ADR-0012 as obsolete. No migration: legacy stages are refused until destroyed or deleted. - [ADR-0046](ADR-0046-the-orm-facade-is-a-peer-dependency.md) — `@prisma/composer-prisma-cloud` takes the Prisma ORM postgres facade (`@prisma/orm-postgres`) as a **peer** dependency at one exact version, not a regular dependency: Composer registers an extension pack against the application's copy of the target, and two copies of a shell in one tree means two codec/operation registries and two class identities — a value from one is rejected by the other, silently. As a peer, that combination fails at install instead. Every `@prisma/orm-*` spec in the workspace is one exact version and all name the same one (`scripts/lint-orm-pins.mjs`). `@prisma/orm-toolchain`, which Composer drives rather than extends, stays a regular dependency. Replaces ADR-0022's consequence bullet on how the ORM is installed. - [ADR-0047](ADR-0047-compute-assembly-preserves-safe-runtime-topology.md) — Compute assembly traces runtime files from the author-declared Node entry without rebundling app code, preserves only symlinks whose resolved targets remain inside the staged bundle, and installs narrowly runtime-gated bootstrap compatibility when a framework needs Node semantics. Each local mechanism is removed once the upstream Alchemy Compute provider owns the equivalent guarantee. Supersedes ADR-0005's blanket ban on symlinks and its assumption that directory output is already self-contained. - [ADR-0048](ADR-0048-prisma-cloud-resources-come-from-the-upstream-alchemy-provider.md) — The six Management-API resource families (project, database, connection, app, deployment, environment variable) are the upstream Prisma provider's (`@alchemy.run/prisma`, formerly `alchemy/Prisma`) classes, registered in Composer's `PrismaComposer` collection; Composer defines resources only where the upstream provider has no support yet (buckets) or no Management API exists behind them. Compute binds the low-level App/Deployment/EnvironmentVariable trio (the `COMPOSER_*_ORIGIN` self-edge and ADR-0005 rule out composite `Compute`); the env→deployment ordering edge rides the deployment's `app` prop, and a deployment is replaced exactly when its artifact or environment fingerprint changes — unchanged services are reused. Legacy rows in the platform state API (retired type-ids and shapes) migrate on read — the older retired SQL state stores stay destroy-only under ADR-0045 — branch-stage databases take generated physical names, and the platform's seeded `DATABASE_URL` is left system-managed. *(Amended by ADR-0057: resource IDs are the node's full address, or the address plus `.` and a role, each declaring its previous ID with `renamedFrom`.)* -- [ADR-0049](ADR-0049-composers-configuration-is-the-composer-section-of-prisma-config.md) — Composer's configuration is the `composer` section of `prisma.config.ts`, loaded by the Prisma CLI engine. The validator checks only the fields that identify extension and state descriptors, passes descriptors through as the file's own objects, rejects unknown keys, and takes the nearest declaring file's section whole so the generated stack imports exactly one file. The old `prisma-composer.config.ts` and the `configPath` pointer are refused with `CONFIG.SECTION_MISSING`, `CONFIG.FIELD_RETIRED` and `CONFIG.FILE_RETIRED`, never migrated. The validator returns the section with its declaring file, and the programmatic operations take the same `ComposerConfigSource` (`{ value, file }`) and run the same checks. The Alchemy child evaluates the whole `prisma.config.ts`. The effect pre-flight and `DEPS.EFFECT_VERSION_CONFLICT` are retired: a broken tree fails evaluation of `prisma.config.ts` with the engine's `CLI.CONFIG_UNREADABLE`. *(Amended: the alternatives section lists the rejected ways to keep a Composer-owned effect check. Amended 2026-10-23: the section takes an optional `entry`, resolved against the declaring file's directory; `--entry` or an operation's `entry` overrides it; neither gives `CONFIG.ENTRY_MISSING`.)* *(Amended by ADR-0062: the section gains `config` adapters.)* *(Amended 2026-10-30: the section takes `config` and `secrets` adapter lists, defaulting to `[environmentVariables()]`; the stack files read the pools again in the child.)* *(Amended by ADR-0064: the child-process reason for one declaring file now holds for `destroy` and `prisma dev`; the decision stands.)* +- [ADR-0049](ADR-0049-composers-configuration-is-the-composer-section-of-prisma-config.md) — Composer's configuration is the `composer` section of `prisma.config.ts`, loaded by the Prisma CLI engine. The validator checks only the fields that identify extension and state descriptors, passes descriptors through as the file's own objects, rejects unknown keys, and takes the nearest declaring file's section whole so the generated stack imports exactly one file. The old `prisma-composer.config.ts` and the `configPath` pointer are refused with `CONFIG.SECTION_MISSING`, `CONFIG.FIELD_RETIRED` and `CONFIG.FILE_RETIRED`, never migrated. The validator returns the section with its declaring file, and the programmatic operations take the same `ComposerConfigSource` (`{ value, file }`) and run the same checks. The Alchemy child evaluates the whole `prisma.config.ts`. The effect pre-flight and `DEPS.EFFECT_VERSION_CONFLICT` are retired: a broken tree fails evaluation of `prisma.config.ts` with the engine's `CLI.CONFIG_UNREADABLE`. *(Amended: the alternatives section lists the rejected ways to keep a Composer-owned effect check. Amended 2026-10-23: the section takes an optional `entry`, resolved against the declaring file's directory; `--entry` or an operation's `entry` overrides it; neither gives `CONFIG.ENTRY_MISSING`.)* *(Amended by ADR-0062: the section gains `config` adapters.)* *(Amended 2026-10-30: the section takes `config` and `secrets` adapter lists, defaulting to `[environmentVariables()]`; the stack files read the pools again in the child.)* *(Amended by ADR-0064: the child-process reason for one declaring file now holds for `destroy` only; the decision stands.)* - [ADR-0050](ADR-0050-composer-runs-as-prisma-deploy-and-prisma-dev.md) — Composer has no binary of its own: the `prisma` CLI mounts exactly two Composer commands, `prisma deploy` and `prisma dev`, mounted at the root as bare verbs on purpose rather than under a noun. `destroy` and `log` are operations on `@prisma/composer/control`, called from a script; a teardown script reads `PRISMA_SERVICE_TOKEN` and `PRISMA_WORKSPACE_ID`. Where teardown and logs belong in the `prisma` command tree is not decided; until it is, agents and users need a script for both. A `destroy` command was rejected: the CLI's destructive verb is `delete`, and no Composer command takes `--production`. Retires the `prisma-composer` binary; a CI lint keeps its name out of user-facing files. *(Amended by ADR-0053: the family also ships `dev url`, `dev stop` and `emulator list`, and `prisma dev` gains `--detach`. Amended 2026-10-23: neither command takes a positional entry; both take `--entry` over the config's `entry`. Amended 2026-10-27: the family also ships `deploy list`.)* *(Amended by ADR-0062: the flag is `--environment`, defaulting per command; `destroy` targets an environment by name.)* - [ADR-0051](ADR-0051-a-nodes-address-is-its-logical-id-on-the-platform.md) — A node's address is its identity on Prisma Cloud: Composer writes it, byte for byte, as the `logicalId` of the node's one platform row (Project, App, Database, Bucket) and as the node's `logicalId` in the application topology, so the platform joins topology nodes to rows, and one node across branches, by string equality. A provision ID defaults to the node's name, so changing a name changes identity. The value is never an Alchemy resource ID, a display name or a platform `id`; App, Database and Bucket rows get it as a prop on the upstream resource that creates them, and the Project from container resolution; no separate resource writes it. Changing an address changes the entity's identity; changing a display name changes nothing. *(Amended 2026-10-16: on the app's own Project and Branch, the row holding a node's `logicalId` is that node's row, so a deploy resuming an interrupted create uses it. Superseded by ADR-0057: addresses start with the app name, and resource IDs, env keys and `serviceOrigin()` are built from them; the identity rules here are carried forward.)* - [ADR-0052](ADR-0052-composer-maps-its-nodes-to-emulator-instances-per-directory.md) — The local emulators know nothing about Composer: the Compute, Postgres and bucket emulators key their instances by ids they generate and store labels they never read. Composer records per working directory, in `.prisma-composer/dev/emulator-instances.json`, which instance backs each node of each app run there, and nothing else maps Composer's names to instances, so two directories running one app get separate services, databases and buckets. An instance keeps its port until it is deleted, because the port is in the deployment's environment and its dependents' URLs and a warm start does not redeploy; a remembered port is requested, as a preference, only when an instance is re-created. When `prisma dev` ends it stops the app's services and database servers, keeping their data. Running a different app from a directory stops the previous app's instances and keeps them; `--fresh` deletes only the current app's. Amends ADR-0041's app-name local identity. *(Amended by ADR-0057: record keys are full addresses; an older record is rewritten on the first `prisma dev`. Amended 2026-10-28: each environment runs its own local copy, with its files in `.prisma-composer/dev//`; the record is keyed by app, environment and node and has no `current` app; running another app or environment stops nothing; `--fresh` deletes only that app's copy in that environment; earlier local state is moved into the `dev` environment.)* - [ADR-0053](ADR-0053-dev-url-dev-stop-and-emulator-list.md) — Composer's command family ships `dev url
` (where a node of the app `prisma dev` runs from this directory answers: a service's URL, a database's connection string, a bucket's S3 endpoint; the URL alone on stdout, piped too), `dev stop` (stops that app's services and database servers, data kept), `dev --detach` (returns once every service answers and leaves the app running until `dev stop`; the foreground stays the default) and `emulator list` (every local emulator and its instances, with labels; needs no project). `devUrl` and `devStop` are `@prisma/composer/control` operations that go through a new optional `current({ devDir })` on the local-target seam, so core never reads the target's record; `listEmulators` lives in the Prisma Cloud domain and `@prisma/composer-cli` adds its command. New codes: `DEV.NOT_STARTED_HERE`, `DEV.ADDRESS_UNKNOWN`, `DEV.NOT_RUNNING`. Amends ADR-0050 and ADR-0043. *(Amended by ADR-0056: `devUrl`'s result gains `runs`. Amended by ADR-0057: `dev url` takes the full address. Amended 2026-10-27: the family also ships `deploy list`, which lists every node an app has deployed on a stage with its platform ID and URL; `@prisma/composer-cli` adds it. Amended by ADR-0059: `deploy list` takes `--environment`. Amended 2026-10-28: `dev url` and `dev stop` take `--environment`, default `dev`, and never act on the app last run; `dev url` takes the app from the address, `dev stop` from the entry or `--name`; `emulator list` shows each instance's environment.)* - [ADR-0054](ADR-0054-raw-http-is-the-untyped-tier-of-service-connections.md) — `@prisma/composer/http` ships `rawHttp()` and `rawHttpContract` (kind `http`, compared by kind only): the untyped tier of service-to-service connections, as `rawPostgres()` is for data. The provider declares its HTTP port; there is no service key, retry or idempotency, because the far end is the app's own server. The auth module's `api` port becomes a raw HTTP port and the provider-named `auth-api` kind is removed, because contracts describe interfaces, not providers. A `Contract` declares its own communication `style`, which the application topology reads. The untyped `http({ name })` is removed. A more specific HTTP kind filling a raw slot is the intended design but is not built, because contract kinds have no parent kinds. - [ADR-0055](ADR-0055-the-postgres-binding-builds-clients-with-the-apps-options.md) — The `postgres(contract)` binding gains `clientWith(options)`, which builds a typed Prisma ORM client with the runtime's own options (less `poolOptions`), such as pgvector's runtime extension or `middleware`, and `connection`, the pool every client shares, for an app that builds its client itself. The dependency declaration takes no options: they are how the service consumes the database, not what it requires, and passing the runtime's option type through means a new option needs no Composer change. `client` is unchanged. Composer never discovers an extension package from the contract's pack ids; a missing one still fails when the client is built, with the runtime's error. Rejected: options on the declaration, extensions on the contract. Amends ADR-0022, extends ADR-0040. -- [ADR-0056](ADR-0056-prisma-dev-runs-a-service-with-its-build-adapters-dev-server.md) — Under `prisma dev`, a service runs the dev server its build adapter declares, in place of its built artifact: `nextjs()` declares `next dev`; `node()` declares none unless the author writes `dev: [...]` on it. No flag. The build descriptor gains `serve(input) → DevServer | undefined` (`cwd`, `argv({ port, host })`, `display`), called before assembly; such a service is not assembled and lowers without its `Deployment` (`LowerOptions.unassembled` + the service descriptor's `reserve`), so its rows, instance, port and dependents' URLs stay. The Compute emulator runs the dev server as a *command deployment* (`{ command: { cwd, argv }, env, listens }`) on the service's instance, with the deployment's materialized env re-stashed address-free by `stashRows`, the pure function `run()` now uses, plus `PORT`/`HOST`, over the shell's env; readiness, logs, crash supervision, restart on env change, `--detach`, `dev stop` and session resume are unchanged and the dev session owns no processes. The session streams dev servers' logs inline; `prisma dev` also watches the entry and every service's authoring module; `devUrl` gains `runs`. New codes `DEV.DEV_SERVER_UNAVAILABLE`, `DEV.DEV_SERVER_UNSUPPORTED`. Amends ADR-0041 and ADR-0053; extends ADR-0005. +- [ADR-0056](ADR-0056-prisma-dev-runs-a-service-with-its-build-adapters-dev-server.md) — Under `prisma dev`, a service runs the dev server its build adapter declares, in place of its built artifact: `nextjs()` declares `next dev`; `node()` declares none unless the author writes `dev: [...]` on it. No flag. The build descriptor gains `serve(input) → DevServer | undefined` (`cwd`, `argv({ port, host })`, `display`), called before assembly; such a service is not assembled and lowers without its `Deployment` (`LowerOptions.unassembled` + the service descriptor's `reserve`), so its rows, instance, port and dependents' URLs stay. The Compute emulator runs the dev server as a *command deployment* (`{ command: { cwd, argv }, env, listens }`) on the service's instance, with the deployment's materialized env re-stashed address-free by `stashRows`, the pure function `run()` now uses, plus `PORT`/`HOST`, over the shell's env; readiness, logs, crash supervision, restart on env change, `--detach`, `dev stop` and session resume are unchanged and the dev session owns no processes. The session streams dev servers' logs inline; `prisma dev` also watches the entry and every service's authoring module; `devUrl` gains `runs`. New codes `DEV.DEV_SERVER_UNAVAILABLE`, `DEV.DEV_SERVER_UNSUPPORTED`. Amends ADR-0041 and ADR-0053; extends ADR-0005. *(Amended by ADR-0064: an edit to an authoring module is reported and pauses rebuilds until dev restarts, instead of reconverging.)* - [ADR-0057](ADR-0057-a-nodes-address-is-its-full-path-from-the-app-name.md) — A node's address is the path of provision IDs starting at the root, whose address is the deployed app name (letters, digits, `-` and `_`; `shop`, `shop.catalog`, `shop.auth.api`), and it is the node's only identity. It is the `logicalId` of the node's platform row and topology node; Alchemy resource IDs are `
` for the main resource and `
.` for other pieces, which cannot collide because a module has children and no resources and a service or resource has resources and no children; `COMPOSER_` env keys are built from the full address with `-` turned into `_`; `serviceOrigin()` takes the full address. Every platform name Composer writes is the address, a label; the upgrade recreates Composer's Connections and bucket access keys with new credentials. Existing stages migrate on their next deploy: every resource declares `renamedFrom` its previous ID, and generated values, service keys and stream keys are renamed, not recreated; local dev's record is rewritten on first run. An older Composer deploying a migrated stage is not protected against. Rejected: unqualified child addresses, a second shorter "path" form, `-` suffixes on resource IDs, relative `serviceOrigin()`. Supersedes ADR-0051, carrying forward its identity rules. - [ADR-0058](ADR-0058-prisma-dev-reads-secrets-from-the-environment-and-never-invents-them.md) — Under `prisma dev` a secret or env-sourced param comes only from the run's environment variables: the shell, or a `.env` that `prisma.config.ts` loads (lead with `import 'dotenv/config'`; `process.loadEnvFile` fails under Bun). A required secret that is unset or empty stops dev with an error naming the variable, the service and both fixes; an optional one reaches the service as absent. Required or optional is decided by validating the input with the secret left out, never by reading the schema (ADR-0042). The dev preflight sends the alchemy process the absent names and an HMAC fingerprint of each pointed value on the existing preflight transport; the serializer leaves out only the names it lists, and the fingerprint fills the `:updatedAt` trigger member, so a changed value redeploys the services that read it. Each run takes values from its environment variables and never reuses one from an earlier run; a leftover `secrets.json` is deleted. Deploy is unchanged, so dev and deploy write different input documents when an optional secret is unset until deploy gets the same fix. A `.env` loaded by `prisma.config.ts` is also read by `prisma deploy`, which uploads missing secrets from it. Amends ADR-0041's secrets bullet. *(Amended 2026-10-30: dev reads the secrets and config pools; `dotenv()` is the documented loader.)* - [ADR-0059](ADR-0059-a-deployment-environment-is-called-an-environment.md) — A place the app runs is an **environment**: a target-neutral concept of Composer's core, used on every public surface (`--environment`, `destroy`'s target, guides, errors, skill), replacing "stage". Every environment has a name and none is special: `prisma deploy` defaults to `production`, `prisma dev` to `dev`. On Prisma Cloud, `production` is the Project's default Branch and any other environment is the Branch of that `gitName`; `dev` has no Branch. "Branch" is Prisma Cloud's term and its git-shaped workflow, and appears only where Composer reports what the target did. - [ADR-0060](ADR-0060-public-traffic-reaches-a-service-through-a-public-url.md) — Traffic from the internet reaches a service only through a **public URL** the author declares: `publicUrl({ name, url, target })`, a driver wired to the HTTP connection a service exposes, as `cron` is wired to what it calls. `url` is one of two explicit strategies, a named hostname or the prisma.build URL the platform gives every service, usually computed per environment with `resolve()`. One hostname per public URL. A reusable module decides what can be public by exposing connections; the app decides what is. Services stay publicly reachable underneath until the platform can address them privately. On Prisma Cloud a named hostname lowers to the platform's RoutingUrl record, covered by a domain ownership claim the workspace has verified; a prisma.build URL lowers to nothing, because every service keeps the platform's own hostname. *(Amended 2026-10-29: the public URL node is a resource by kind and a driver by role, with a slot wired at construction and `props`; see the note at its top.)* - [ADR-0061](ADR-0061-a-module-declares-its-input-as-a-compute-service-does.md) — A module declares its configuration as one Standard Schema, `input`, exactly as a compute service does; its body receives opaque per-key references it can only pass into children's inputs or derive from with `resolve()`, so the graph cannot branch on configuration. Module `params`/`paramNeed()` and `secrets`/`secret()` slots are removed. The root may declare `input`, filled by config adapters. *(Amended 2026-10-29: an input field is a `Value`; the module's schema validates its binding during Load's second pass; a truthiness check on a field is not caught, but always goes the same way.)* *(Amended 2026-10-29, third slice: `paramSource()` and `secretSource()` return values and take an options argument.)* *(Amended 2026-10-30: a root's input is the config pool.)* *(Amended 2026-10-30, fifth slice: the slots are removed; `ModuleNode`.)* *(Amended 2026-10-30, seventh slice: a module's schema checks each runtime value as its pointer; `paramSource()` loses `sentinel`.)* *(Amended 2026-10-30, eighth slice: only a plain field can be a `resolve()` dependency.)* -- [ADR-0062](ADR-0062-environment-values-come-from-resolve-and-config-adapters.md) — The graph's structure is the same whatever inputs Composer is given; only values differ per environment. A module body holds values (`Value`), opaque stand-ins for data known later: deploy-time values, whose data the deploy has by the end of Load, and runtime values (`RuntimeValue`, which is `Value>`), whose data only the platform holds. `resolve(dependencies, fn)` computes a value from deploy-time values only, such as `environment.name` or a plain input field; `fn` receives their real values as one tuple after the graph is built, returns plain data, and never sees a pointer. Config adapters in the `composer` section's `config` list produce the configuration, which is the root's input and what `envParam()` reads; secrets adapters in its `secrets` list produce the secrets, which fill missing `envSecret()` platform variables. Both lists default to `[environmentVariables()]`; `dotenv()` reads `.env` files through `dotenv-flow`. Secrets stay out of deploy files, state and output through pointers and redaction. Every environment has a name and none is special: `prisma deploy` defaults `--environment` to `production`, `prisma dev` to `dev`; on Prisma Cloud `production` is the Project's default Branch. *(Amended by ADR-0064: `deploy` runs Alchemy in its first process and writes no configuration file; the second process applies to `prisma dev` and `destroy`.)* +- [ADR-0062](ADR-0062-environment-values-come-from-resolve-and-config-adapters.md) — The graph's structure is the same whatever inputs Composer is given; only values differ per environment. A module body holds values (`Value`), opaque stand-ins for data known later: deploy-time values, whose data the deploy has by the end of Load, and runtime values (`RuntimeValue`, which is `Value>`), whose data only the platform holds. `resolve(dependencies, fn)` computes a value from deploy-time values only, such as `environment.name` or a plain input field; `fn` receives their real values as one tuple after the graph is built, returns plain data, and never sees a pointer. Config adapters in the `composer` section's `config` list produce the configuration, which is the root's input and what `envParam()` reads; secrets adapters in its `secrets` list produce the secrets, which fill missing `envSecret()` platform variables. Both lists default to `[environmentVariables()]`; `dotenv()` reads `.env` files through `dotenv-flow`. Secrets stay out of deploy files, state and output through pointers and redaction. Every environment has a name and none is special: `prisma deploy` defaults `--environment` to `production`, `prisma dev` to `dev`; on Prisma Cloud `production` is the Project's default Branch. *(Amended by ADR-0064: `deploy` and `prisma dev` run Alchemy in their first process and write no configuration file; the second process applies to `destroy` only.)* - [ADR-0063](ADR-0063-a-node-depends-on-an-interface-or-exposes-one.md) — The two ends of a connection are named by what the node does with them: a node **depends on** an interface (`deps`) or **exposes** one (`expose`); an interface is a `Contract` in code; Input and Output are retired as port words, because the dataflow and ports-and-adapters traditions give the same two ends opposite names; the module body's `inputs` becomes `deps`; the topology's `in`/`out` stay, defined as the dependency end and the exposed end. -- [ADR-0064](ADR-0064-deploy-runs-alchemy-in-process.md) — `prisma deploy` and the `deploy` operation run Alchemy's plan and apply in the `prisma` process through `alchemy/Alchemist`: no generated stack file, no Alchemy child. `lower()` takes containers, preflight payloads and the report hook as values, the summary comes back as a value, Alchemy's events become engine step events, failures are `DEPLOY.ENGINE_FAILED` or `DEPLOY.INTERRUPTED`, and Ctrl-C releases the state lease. The credential reaches Alchemy through `process.env`; a `prisma login` session needs `PRISMA_SERVICE_TOKEN` (`DEPLOY.SERVICE_TOKEN_REQUIRED`). Lowering and Alchemy load from the app's own install of `@prisma/composer`, which must speak the host's converge contract. Supersedes ADR-0007 for `deploy`; amends ADR-0038, ADR-0043, ADR-0044, ADR-0049 and ADR-0062. +- [ADR-0064](ADR-0064-deploy-runs-alchemy-in-process.md) — `prisma deploy` and `prisma dev` (and their operations) run Alchemy's plan and apply in the `prisma` process through `alchemy/Alchemist`: no generated stack file, no Alchemy child. `lower()` takes containers, preflight payloads and the report hook as values, the summary comes back as a value, Alchemy's events become engine step events, failures are `DEPLOY.ENGINE_FAILED` or `DEPLOY.INTERRUPTED`, and Ctrl-C releases the state lease. The credential reaches Alchemy through `process.env`; a `prisma login` session needs `PRISMA_SERVICE_TOKEN` (`DEPLOY.SERVICE_TOKEN_REQUIRED`). Lowering and Alchemy load from the app's own install of `@prisma/composer`, which must speak the host's converge contract. Dev converges once at start and once per rebuild against the local target, reports progress through its own events, and asks for a restart after an edit to the app's definition, because the app is imported once per process (`definition-changed`). Supersedes ADR-0007 for `deploy` and `prisma dev`; amends ADR-0038, ADR-0043, ADR-0044, ADR-0049, ADR-0056 and ADR-0062. diff --git a/docs/guides/deploying.md b/docs/guides/deploying.md index ae8563e09..a4dfcbcda 100644 --- a/docs/guides/deploying.md +++ b/docs/guides/deploying.md @@ -745,9 +745,9 @@ What to know before embedding it: - **`deploy` runs the engine in your process.** It returns `summary` as a value and reports the engine's progress through `onEvent`; it prints nothing. While it plans and applies it sets the service token and workspace id in `process.env` and changes to `cwd` when your process is elsewhere, then restores both, so a process runs one deploy at a time (`DEPLOY.CONVERGE_RUNNING`), and from a worker thread `cwd` must be the process's working directory (`DEPLOY.WORKER_THREAD`). Deploy uses the lowering and the converge from the app's own install of `@prisma/composer`, found beside its `prisma.config.ts`, which must speak the same converge contract as the Composer running the deploy (`DEPS.CONVERGE_CONTRACT_MISMATCH` otherwise); the package versions may differ. Pass `signal` to interrupt it. `destroy` still runs the engine as a child process whose output streams to your process's stdio. - **`dev` resolves to `{ ok: true, value: session }` or a failure** — never an exit code. The session is `{ endpoints, stop(), closed }`, with - progress (`ready`, `converge-failed`, `config-changed`, `watch-error`, …) delivered through + progress (`ready`, `converge-resource`, `converge-failed`, `config-changed`, `definition-changed`, `watch-error`, …) delivered through `onEvent`. The operation never installs signal handlers; wiring Ctrl-C to - `session.stop()` is yours. + `session.stop()` is yours, and passing `signal` interrupts a running converge. Like `deploy`, `dev` runs the engine in your process, one converge at a time. - **`log` resolves to `{ ok: true, value: { appName, services, lines } }` or a failure.** `lines` is an `AsyncIterable` ended by an `AbortSignal` you own (stopping early — `break`, `lines.return()` — also ends it cleanly). diff --git a/docs/guides/running-locally.md b/docs/guides/running-locally.md index 92a028909..c93c8f09f 100644 --- a/docs/guides/running-locally.md +++ b/docs/guides/running-locally.md @@ -49,7 +49,7 @@ orders.service: http://localhost:3003 `dev` prints `ready` only once every service accepts connections on its port, so a request to a printed URL right away connects. After a rebuild, `dev` waits the same way before it prints the URLs again. A service must listen on the port Composer gives it, unless it is declared with `compute({ ..., listens: false })`: such a service, like the cron module's scheduler, is running as soon as its process starts. -If a service exits before it listens, or does not listen within 30 seconds, `dev` fails instead of printing `ready`, and stops the app's other services. The message names the service, its port, and its log file, for example `service "my-app.gateway" exited (code 1) before it accepted connections on its port 3001. Its log: …/svc_….log`. On a first start it appears in the deploy engine's output, followed by `prisma dev`'s own error that the deploy engine exited with code 1 (`DEV.CONVERGE_FAILED` from the `dev` operation in `@prisma/composer/control`). On a warm start it is the message of the `DEV.SERVICE_START_FAILED` error. +If a service exits before it listens, or does not listen within 30 seconds, `dev` fails instead of printing `ready`, and stops the app's other services. The message names the service, its port, and its log file, for example `service "my-app.gateway" exited (code 1) before it accepted connections on its port 3001. Its log: …/svc_….log`. On a first start it is in the message of `prisma dev`'s error that the app could not be converged (`DEV.CONVERGE_FAILED` from the `dev` operation in `@prisma/composer/control`). On a warm start it is the message of the `DEV.SERVICE_START_FAILED` error. A service that needs longer to start, for example one that runs migrations first, can have more time: set `PRISMA_COMPOSER_SERVICE_READY_TIMEOUT_MS` to the limit in milliseconds (the default is `30000`). A value that is not a positive whole number fails with `DEV.SERVICE_READY_TIMEOUT_INVALID`. The compute emulator reads the variable when it starts, and a running emulator keeps the value it started with; `--fresh` does not restart it. To apply a new value, end the compute emulator's process (its PID is in `compute.json` in the emulators directory, see below), which stops the services of every app that uses it. The next `prisma dev` starts it again with the new value. @@ -66,6 +66,12 @@ and watches the file. When the file changes, `dev` prints that it changed and must be restarted, and it stops rebuilding: until you stop `dev` and start it again, a build change only repeats that notice. +`dev` also loads your app's definition once: the entry module and the +files your services are declared in, such as `module.ts` and each +`service.ts`. When one of them changes, `dev` prints which file changed and +that it must be restarted, and it stops rebuilding in the same way. An edit +to another file your entry imports takes effect the next time `dev` starts. + `Ctrl-C` stops your app's service processes and its local Postgres servers, and exits; so does `dev` ending on SIGTERM or a failure. The data stays, and so do the ports, so the next `prisma dev` is a warm start — same ports, same @@ -148,11 +154,10 @@ What else changes for such a service: adds `A dev server must listen on the port in its PORT variable.` A dev server that crashes later is restarted with backoff, as an artifact is. - **Edits to your source are the dev server's business.** `dev` does not - rebuild for them. It does watch your entry file and every service's - authoring module, so a change to `module.ts` or a `service.ts` reconverges - the app. When that changes what a dev server's environment holds, such as a - dependency's URL or a new input, the dev server is restarted: - `` shop.web: its environment changed; `next dev` restarted ``. + rebuild for them. A change to `module.ts` or a `service.ts` needs `dev` + restarted, as above. When a rebuild of another service changes what a dev + server's environment holds, such as a dependency's URL, the dev server is + restarted: `` shop.web: its environment changed; `next dev` restarted ``. - **It has no terminal.** Its output goes through the service's log, so a dev server that checks for a terminal drops its colours and keyboard shortcuts, and a line reaches your screen a moment after it is written. diff --git a/packages/0-framework/3-tooling/cli/src/__tests__/app-converge.test.ts b/packages/0-framework/3-tooling/cli/src/__tests__/app-converge.test.ts index 44726197e..188a2742c 100644 --- a/packages/0-framework/3-tooling/cli/src/__tests__/app-converge.test.ts +++ b/packages/0-framework/3-tooling/cli/src/__tests__/app-converge.test.ts @@ -7,6 +7,9 @@ import { CliStructuredError } from '@internal/foundation/errors'; import { HOST_COMPOSER_VERSION, loadAppConverge } from '../app-converge.ts'; import { CONVERGE_CONTRACT } from '../converge-contract.ts'; +const FULL_ENTRY = + "export const lower = () => {}; export const convergeInProcess = async () => ({ kind: 'applied' }); export const localTargetProviders = () => {}; export const devState = () => {};"; + const dirs: string[] = []; afterEach(() => { for (const dir of dirs.splice(0)) fs.rmSync(dir, { recursive: true, force: true }); @@ -69,12 +72,11 @@ async function failureOf(dir: string): Promise { } describe('loadAppConverge', () => { - test("loads lower and convergeInProcess from the app's own @prisma/composer", async () => { + test("loads lower, convergeInProcess and the local target's functions from the app's own @prisma/composer", async () => { const dir = appWith({ exports: { './internal/converge': './dist/internal-converge.mjs' }, files: { - 'dist/internal-converge.mjs': - "export const lower = () => {}; export const convergeInProcess = async () => ({ kind: 'applied' });", + 'dist/internal-converge.mjs': FULL_ENTRY, }, }); @@ -85,6 +87,8 @@ describe('loadAppConverge', () => { expect(engine.lower).toBe(stub.lower); expect(engine.converge).toBe(stub.convergeInProcess); + expect(engine.localTargetProviders).toBe(stub.localTargetProviders); + expect(engine.devState).toBe(stub.devState); }); test('an app without @prisma/composer is DEPS.CONVERGE_UNRESOLVABLE', async () => { @@ -110,14 +114,29 @@ describe('loadAppConverge', () => { }), ); - expect(failure.message).toContain('exports no lower and convergeInProcess'); + expect(failure.message).toContain( + 'exports no lower, convergeInProcess, localTargetProviders, devState', + ); + }); + + test("an entry that predates dev's exports is DEPS.CONVERGE_UNRESOLVABLE, naming what is missing", async () => { + const failure = await failureOf( + appWith({ + exports: { './internal/converge': './dist/internal-converge.mjs' }, + files: { + 'dist/internal-converge.mjs': + "export const lower = () => {}; export const convergeInProcess = async () => ({ kind: 'applied' });", + }, + }), + ); + + expect(failure.message).toContain('exports no localTargetProviders, devState'); }); const entry = { exports: { './internal/converge': './dist/internal-converge.mjs' }, files: { - 'dist/internal-converge.mjs': - "export const lower = () => {}; export const convergeInProcess = async () => ({ kind: 'applied' });", + 'dist/internal-converge.mjs': FULL_ENTRY, }, }; diff --git a/packages/0-framework/3-tooling/cli/src/__tests__/converge-in-process.test.ts b/packages/0-framework/3-tooling/cli/src/__tests__/converge-in-process.test.ts index 8c0e0a8d5..523df2df1 100644 --- a/packages/0-framework/3-tooling/cli/src/__tests__/converge-in-process.test.ts +++ b/packages/0-framework/3-tooling/cli/src/__tests__/converge-in-process.test.ts @@ -10,6 +10,7 @@ import * as path from 'node:path'; import { type Contract, module, resource } from '@internal/core'; import type { ExtensionDescriptor, PrismaAppConfig } from '@internal/core/config'; import { type DeploymentResult, type LowerOptions, lower } from '@internal/core/deploy'; +import { devState, localTargetProviders } from '@internal/core/local-target'; import * as Alchemist from 'alchemy/Alchemist'; import * as Interaction from 'alchemy/Interaction'; import * as State from 'alchemy/State'; @@ -427,7 +428,7 @@ describe('deploy with the real in-process converge', () => { }, { runAssembler: async () => ({ dir, entry: 'server.js' }), - converge: { lower, converge: convergeInProcess }, + converge: { lower, converge: convergeInProcess, localTargetProviders, devState }, credentials: { workspaceId: 'ws_1', client: {}, serviceToken: 'tok' }, }, ); diff --git a/packages/0-framework/3-tooling/cli/src/__tests__/fixtures/stack-import/stub-composer.ts b/packages/0-framework/3-tooling/cli/src/__tests__/fixtures/stack-import/stub-composer.ts index 6f6fcaa17..9d56a618a 100644 --- a/packages/0-framework/3-tooling/cli/src/__tests__/fixtures/stack-import/stub-composer.ts +++ b/packages/0-framework/3-tooling/cli/src/__tests__/fixtures/stack-import/stub-composer.ts @@ -14,7 +14,7 @@ plugin({ build.module( '@prisma/composer/deploy', js( - 'export const lower = (app, config, opts) => { console.log(JSON.stringify({ app: app.name, extensions: config.extensions.map((e) => e.id), state: config.state.extension, environment: opts.environment, configPool: opts.configPool, ...(globalThis.stubPools === undefined ? {} : { pools: globalThis.stubPools }) })); return {}; }; export const transportedLowerInputs = () => ({ containers: new Map(), preflight: new Map() });', + 'export const lower = (app, config, opts) => { console.log(JSON.stringify({ app: app.name, extensions: config.extensions.map((e) => e.id), state: config.state.extension, environment: opts.environment, configPool: opts.configPool, })); return {}; }; export const transportedLowerInputs = () => ({ containers: new Map(), preflight: new Map() });', ), ); build.module( @@ -27,11 +27,5 @@ plugin({ "export const readConfigPool = async (section, input) => ({ read: 'config', environment: input.environment, configFile: input.configFile }); export const readSecretsPool = async () => ({ read: 'secrets' });", ), ); - build.module( - '@prisma/composer/local-target', - js( - "export const devEnvironmentDir = (environment) => '.dev/' + environment; export const devState = () => ({}); export const localTargetProviders = (resolved, containers, devDir, pools) => { globalThis.stubPools = pools; return {}; }; export const resolveLocalTargets = async () => new Map();", - ), - ); }, }); diff --git a/packages/0-framework/3-tooling/cli/src/__tests__/stack-file-import.test.ts b/packages/0-framework/3-tooling/cli/src/__tests__/stack-file-import.test.ts index 911a9724b..a1df24f78 100644 --- a/packages/0-framework/3-tooling/cli/src/__tests__/stack-file-import.test.ts +++ b/packages/0-framework/3-tooling/cli/src/__tests__/stack-file-import.test.ts @@ -1,6 +1,6 @@ /** - * The generated stack files import the declaring prisma.config.ts and hand its - * `composer` export to lower(). These tests run the generated files in a child + * The generated stack file (destroy's) imports the declaring prisma.config.ts and hand its + * `composer` export to lower(). These tests run the generated file in a child * process against a real prisma.config.ts, with the heavy imports stubbed, so * the import itself is executed rather than string-matched. */ @@ -8,7 +8,6 @@ import { afterEach, describe, expect, test } from 'bun:test'; import { spawnSync } from 'node:child_process'; import * as fs from 'node:fs'; import * as path from 'node:path'; -import { writeDevStackFile } from '../dev/generate-dev-stack.ts'; import { writeStackFile } from '../generate-stack.ts'; const fixtureDir = path.join(import.meta.dir, 'fixtures', 'stack-import'); @@ -42,14 +41,14 @@ function runStack(stackFile: string) { return JSON.parse(child.stdout.trim()); } -describe('the generated stack files, evaluated', () => { +describe('the generated stack file, evaluated', () => { const configPool = { read: 'config', environment: 'staging', configFile: path.join(fixtureDir, 'prisma.config.ts'), }; - test('the deploy stack passes the composer export of prisma.config.ts, the environment, and the config pool it reads, to lower(), and reads no secrets', () => { + test('the stack passes the composer export of prisma.config.ts, the environment, and the config pool it reads, to lower(), and reads no secrets', () => { expect(runStack(writeStackFile(stackInput))).toEqual({ app: 'fixture-module', extensions: ['fixture-extension'], @@ -59,17 +58,6 @@ describe('the generated stack files, evaluated', () => { }); }); - test('the dev stack passes the composer export of prisma.config.ts, the environment, and the config pool it reads, to lower(), and both pools to the local providers', () => { - expect(runStack(writeDevStackFile(stackInput))).toEqual({ - app: 'fixture-module', - extensions: ['fixture-extension'], - state: 'fixture-extension', - environment: 'staging', - configPool, - pools: { config: configPool, secrets: { read: 'secrets' } }, - }); - }); - const noSectionFile = path.join(fixtureDir, '..', 'stack-import-no-section', 'prisma.config.ts'); /** @@ -77,17 +65,12 @@ describe('the generated stack files, evaluated', () => { * programmatic caller naming the wrong file, would otherwise fail inside * lower() with a TypeError. */ - for (const [stack, write] of [ - ['deploy', writeStackFile], - ['dev', writeDevStackFile], - ] as const) { - test(`the ${stack} stack refuses a prisma.config.ts that declares no composer section, naming it`, () => { - const child = spawnStack(write({ ...stackInput, configFile: noSectionFile })); - expect(child.status).not.toBe(0); - expect(child.stdout).toBe(''); - expect(child.stderr).toContain( - `${noSectionFile} declares no \`composer\` section. Pass the prisma.config.ts that declares it.`, - ); - }); - } + test('the stack refuses a prisma.config.ts that declares no composer section, naming it', () => { + const child = spawnStack(writeStackFile({ ...stackInput, configFile: noSectionFile })); + expect(child.status).not.toBe(0); + expect(child.stdout).toBe(''); + expect(child.stderr).toContain( + `${noSectionFile} declares no \`composer\` section. Pass the prisma.config.ts that declares it.`, + ); + }); }); diff --git a/packages/0-framework/3-tooling/cli/src/app-converge.ts b/packages/0-framework/3-tooling/cli/src/app-converge.ts index ae1a46ece..02dc62e24 100644 --- a/packages/0-framework/3-tooling/cli/src/app-converge.ts +++ b/packages/0-framework/3-tooling/cli/src/app-converge.ts @@ -1,5 +1,6 @@ /** - * Which copy of Composer's lowering and in-process converge runs `deploy`: + * Which copy of Composer's lowering and in-process converge runs `deploy` + * and `prisma dev`: * the one the app installs, found from the directory of the app's * `prisma.config.ts`, the file its extensions are imported from. The stack, * Alchemy and the providers then share one copy of `alchemy` and `effect` @@ -13,6 +14,7 @@ import { createRequire } from 'node:module'; import * as path from 'node:path'; import { pathToFileURL } from 'node:url'; import type { lower } from '@internal/core/deploy'; +import type { devState, localTargetProviders } from '@internal/core/local-target'; import { blindCast } from '@internal/foundation/casts'; import { CliStructuredError } from '@internal/foundation/errors'; import manifest from '../package.json' with { type: 'json' }; @@ -21,14 +23,17 @@ import type { Converge } from './converge-in-process.ts'; const ENTRY = '@prisma/composer/internal/converge'; const CONTRACT_ENTRY = '@prisma/composer/internal/converge-contract'; +const ENTRY_FUNCTIONS = ['lower', 'convergeInProcess', 'localTargetProviders', 'devState'] as const; /** The Composer version this host runs; every Composer package carries the same one. */ export const HOST_COMPOSER_VERSION: string = manifest.version; -/** The app install's `lower` and `convergeInProcess`. */ +/** The app install's `lower`, `convergeInProcess`, and the local target's providers and state store for `prisma dev`. */ export interface AppConverge { readonly lower: typeof lower; readonly converge: Converge; + readonly localTargetProviders: typeof localTargetProviders; + readonly devState: typeof devState; } function unresolvable(appDir: string, detail: string, cause?: unknown): CliStructuredError { @@ -36,7 +41,7 @@ function unresolvable(appDir: string, detail: string, cause?: unknown): CliStruc 'DEPS.CONVERGE_UNRESOLVABLE', `Could not load the converge from the app's @prisma/composer in "${appDir}": ${detail}`, { - why: "`prisma deploy` runs the lowering and Alchemy from the app's own install of @prisma/composer, so the app, its extensions and Alchemy share one copy of alchemy.", + why: "`prisma deploy` and `prisma dev` run the lowering and Alchemy from the app's own install of @prisma/composer, so the app, its extensions and Alchemy share one copy of alchemy.", fix: 'Install @prisma/composer in the app, at the same version as the Composer the `prisma` CLI runs.', ...(cause === undefined ? {} : { cause }), }, @@ -84,7 +89,7 @@ function contractMismatch( 'DEPS.CONVERGE_CONTRACT_MISMATCH', `The app's @prisma/composer ${appVersion} speaks converge contract ${app.contract}, and the \`prisma\` CLI's Composer ${host.version} speaks contract ${host.contract}.`, { - why: "Deploy calls the app's @prisma/composer from the CLI's Composer, and the two read each other's data only when they speak the same converge contract.", + why: "Deploy and dev call the app's @prisma/composer from the CLI's Composer, and the two read each other's data only when they speak the same converge contract.", fix: update, }, ); @@ -120,12 +125,18 @@ export async function loadAppConverge( if (appContract !== host.contract) { throw contractMismatch(host, { contract: appContract, version: installedVersion(require) }); } - const { lower: appLower, convergeInProcess } = await importFile(appDir, file); - if (typeof appLower !== 'function' || typeof convergeInProcess !== 'function') { - throw unresolvable(appDir, `${file} exports no lower and convergeInProcess.`); + const entry = await importFile(appDir, file); + const missing = ENTRY_FUNCTIONS.filter((name) => typeof entry[name] !== 'function'); + if (missing.length > 0) { + throw unresolvable(appDir, `${file} exports no ${missing.join(', ')}.`); } return blindCast< AppConverge, - "the typeof checks above prove both are functions, and the contract check proves the entry speaks this host's converge contract" - >({ lower: appLower, converge: convergeInProcess }); + "the typeof checks above prove each is a function, and the contract check proves the entry speaks this host's converge contract" + >({ + lower: entry['lower'], + converge: entry['convergeInProcess'], + localTargetProviders: entry['localTargetProviders'], + devState: entry['devState'], + }); } diff --git a/packages/0-framework/3-tooling/cli/src/converge-contract.ts b/packages/0-framework/3-tooling/cli/src/converge-contract.ts index 215b1c513..0425e1a4a 100644 --- a/packages/0-framework/3-tooling/cli/src/converge-contract.ts +++ b/packages/0-framework/3-tooling/cli/src/converge-contract.ts @@ -1,9 +1,10 @@ /** * The version of the shapes the `prisma` host and the app's * `@prisma/composer` pass each other through `internal/converge`: - * `LowerOptions`, `ConvergeRequest`, `ConvergeOutcome`, `ConvergeEvent` and - * `DeploymentResult`. Bump it only when one of them changes in a way the - * other side cannot read. Imports nothing, so a host can read it before - * loading the converge. + * `LowerOptions`, `ConvergeRequest`, `ConvergeOutcome`, `ConvergeEvent`, + * `DeploymentResult`, and the functions the entry exports (`lower`, + * `convergeInProcess`, `localTargetProviders`, `devState`). Bump it only + * when one of them changes in a way the other side cannot read. Imports + * nothing, so a host can read it before loading the converge. */ -export const CONVERGE_CONTRACT = 1; +export const CONVERGE_CONTRACT = 2; diff --git a/packages/0-framework/3-tooling/cli/src/dev/__tests__/generate-dev-stack.test.ts b/packages/0-framework/3-tooling/cli/src/dev/__tests__/generate-dev-stack.test.ts deleted file mode 100644 index 9a40f2c34..000000000 --- a/packages/0-framework/3-tooling/cli/src/dev/__tests__/generate-dev-stack.test.ts +++ /dev/null @@ -1,165 +0,0 @@ -import { describe, expect, test } from 'bun:test'; -import * as path from 'node:path'; -import { renderStackFile } from '../../generate-stack.ts'; -import { renderDevStackFile } from '../generate-dev-stack.ts'; - -describe('renderDevStackFile()', () => { - test('renders the config + app imports (relative), resolved dev providers, devState(), no report', () => { - const content = renderDevStackFile({ - entryPath: '/repo/app/module.ts', - cwd: '/repo/app', - configFile: '/repo/app/prisma.config.ts', - name: 'app', - environment: 'dev', - assembled: { - devServers: {}, - bundles: { - web: { dir: '/repo/app/modules/web/dist/bundle', entry: 'server.js' }, - }, - }, - }); - - expect(content).toContain( - "import { lower, transportedLowerInputs } from '@prisma/composer/deploy';", - ); - expect(content).not.toContain('deserializeContainers'); - expect(content).toContain( - "import { readConfigPool, readSecretsPool } from '@prisma/composer/internal/pools';", - ); - expect(content).toContain( - "import { devEnvironmentDir, devState, localTargetProviders, resolveLocalTargets } from '@prisma/composer/local-target';", - ); - // alchemy is Composer's dependency, not the app's: under pnpm the app - // cannot import it, so the stack file reaches it through Composer. - expect(content).not.toContain("from 'alchemy"); - expect(content).toContain('import prismaConfig from "../../../prisma.config.ts";'); - expect(content).toContain('const config = prismaConfig.composer;'); - expect(content).toContain('import app from "../../../module.ts";'); - // The one orchestration point (spec § 3 REVISED): containers - // deserialized, `localTarget` thunks resolved by top-level await, - // providers + state passed explicitly — and no `dev` flag, which no - // longer exists on LowerOptions. - expect(content).toContain('const transported = transportedLowerInputs(config);'); - expect(content).toContain('const resolved = await resolveLocalTargets(config);'); - expect(content).toContain('const devDir = path.join(process.cwd(), devEnvironmentDir("dev"));'); - expect(content).toContain('lower(app, config, {'); - expect(content).toContain('name: "app"'); - expect(content).toContain( - 'providers: localTargetProviders(resolved, transported.containers, devDir, pools)', - ); - expect(content).toContain('state: devState()'); - expect(content).not.toContain('dev: true'); - expect(content).toContain( - '"web": { dir: "/repo/app/modules/web/dist/bundle", entry: "server.js" }', - ); - // Dev prints its own front door — no report hook in the generated module. - expect(content).not.toContain('report:'); - expect(content).not.toContain('deploymentReport'); - }); - - test('the header comment names the command Composer starts, at the Alchemy stage named by the environment', () => { - const content = renderDevStackFile({ - entryPath: '/repo/app/module.ts', - cwd: '/repo/app', - configFile: '/repo/app/prisma.config.ts', - name: 'app', - environment: 'feature/x', - assembled: { devServers: {}, bundles: {} }, - }); - - expect(content).toContain( - ` deploy ${path.join('.prisma-composer', 'dev', 'feature%2Fx', 'alchemy.run.ts')} --yes --stage feature/x`, - ); - expect(content).toContain( - 'const devDir = path.join(process.cwd(), devEnvironmentDir("feature/x"));', - ); - expect(content).toContain('import app from "../../../module.ts";'); - }); -}); - -describe('renderDevStackFile() — the environment', () => { - test('passes the environment to lower(), so Load resolves values over environment.name', () => { - const content = renderDevStackFile({ - entryPath: '/repo/app/module.ts', - cwd: '/repo/app', - configFile: '/repo/app/prisma.config.ts', - name: 'app', - environment: 'test', - assembled: { devServers: {}, bundles: {} }, - }); - - expect(content).toContain(' name: "app",\n environment: "test",\n'); - }); - - test('reads both pools from the adapters, passes the config pool to lower() directly after the environment, and both pools to the local providers', () => { - const content = renderDevStackFile({ - entryPath: '/repo/app/module.ts', - cwd: '/repo/app', - configFile: '/repo/app/prisma.config.ts', - name: 'app', - environment: 'test', - assembled: { devServers: {}, bundles: {} }, - }); - - expect(content).toContain( - 'const poolInput = { environment: "test", configFile: "/repo/app/prisma.config.ts" };', - ); - expect(content).toContain( - 'const pools = { config: await readConfigPool(config, poolInput), secrets: await readSecretsPool(config, poolInput) };', - ); - expect(content).toContain(' environment: "test",\n configPool: pools.config,\n'); - expect(content).toContain( - 'localTargetProviders(resolved, transported.containers, devDir, pools)', - ); - }); - - test('no secret value in the process environment reaches either rendered stack file', () => { - process.env['S4_SECRET_SENTINEL'] = 'never-in-the-stack-file'; - try { - const input = { - entryPath: '/repo/app/module.ts', - cwd: '/repo/app', - configFile: '/repo/app/prisma.config.ts', - name: 'app', - environment: 'test', - assembled: { devServers: {}, bundles: {} }, - }; - - expect(renderDevStackFile(input)).not.toContain('never-in-the-stack-file'); - expect(renderStackFile(input)).not.toContain('never-in-the-stack-file'); - } finally { - delete process.env['S4_SECRET_SENTINEL']; - } - }); -}); - -describe('renderDevStackFile() — services with dev servers (ADR-0056)', () => { - const input = (devServers: Record) => ({ - entryPath: '/repo/app/module.ts', - cwd: '/repo/app', - configFile: '/repo/app/prisma.config.ts', - name: 'app', - environment: 'dev', - assembled: { - bundles: { api: { dir: '/repo/app/api/dist/bundle', entry: 'server.js' } }, - devServers: Object.fromEntries( - Object.entries(devServers).map(([address, s]) => [address, { ...s, argv: () => [] }]), - ), - }, - }); - - test('lists them as unassembled, beside the bundles of the assembled services', () => { - const content = renderDevStackFile( - input({ web: { cwd: '/repo/app/web', display: 'next dev' } }), - ); - - expect(content).toContain( - ' "api": { dir: "/repo/app/api/dist/bundle", entry: "server.js" },', - ); - expect(content).toContain(' unassembled: ["web"],'); - }); - - test('renders no unassembled list when no service runs a dev server', () => { - expect(renderDevStackFile(input({}))).not.toContain('unassembled'); - }); -}); diff --git a/packages/0-framework/3-tooling/cli/src/dev/generate-dev-stack.ts b/packages/0-framework/3-tooling/cli/src/dev/generate-dev-stack.ts deleted file mode 100644 index c78971fbe..000000000 --- a/packages/0-framework/3-tooling/cli/src/dev/generate-dev-stack.ts +++ /dev/null @@ -1,131 +0,0 @@ -/** - * Local-dev spec § 6 `generate-dev-stack.ts`: like generate-stack.ts but in - * the environment's folder, `.prisma-composer/dev//alchemy.run.ts`, - * at the Alchemy stage named by the environment, using - * Composer's `devState()` (Alchemy's file store) and no `report` — dev prints its own front - * door (run-dev.ts), so core's presentation-free lower() gets nothing to - * call back into. The generated module IS the one orchestration point where - * dev provenance is resolved (spec § 3 REVISED): it deserializes the - * containers, top-level-awaits `resolveLocalTargets(config)` (the lazy - * `localTarget` thunks, ADR-0041), and passes - * `providers: localTargetProviders(...)` + `state: devState()` explicitly - * — `lower()` learns nothing about dev. - */ -import * as fs from 'node:fs'; -import * as path from 'node:path'; -import type { AssembledServices } from '@internal/assemble'; -import { devEnvironmentDir } from '@internal/core/config'; -import { noComposerExportMessage } from '../composer-config.ts'; - -export interface DevStackFileInput { - /** Absolute path to the entry module (the app's default-exported node). */ - readonly entryPath: string; - /** The process's cwd — where `.prisma-composer/dev/` is written. */ - readonly cwd: string; - /** The environment the app runs as locally; its folder holds the stack file. */ - readonly environment: string; - /** Absolute path to the `prisma.config.ts` that declared the `composer` section — imported by relative path. */ - readonly configFile: string; - readonly name: string; - readonly assembled: AssembledServices; -} - -const DEV_GENERATED_FILE = 'alchemy.run.ts'; - -/** The stack file's path relative to the cwd. */ -export function devStackRelativePath(environment: string): string { - return path.join(devEnvironmentDir(environment), DEV_GENERATED_FILE); -} - -/** A relative import specifier from the stack file to `target` (posix separators). */ -function relativeImportSpecifier(generatedDir: string, target: string): string { - const rel = path.relative(generatedDir, target).split(path.sep).join('/'); - return rel.startsWith('.') ? rel : `./${rel}`; -} - -function quote(value: string): string { - return JSON.stringify(value); -} - -function renderBundle(bundle: { dir: string; entry: string }): string { - return `{ dir: ${quote(bundle.dir)}, entry: ${quote(bundle.entry)} }`; -} - -function renderOptions(input: DevStackFileInput): string { - const lines: string[] = []; - lines.push(` name: ${quote(input.name)},`); - lines.push(` environment: ${quote(input.environment)},`); - lines.push(' configPool: pools.config,'); - - lines.push(' bundles: {'); - for (const [id, bundle] of Object.entries(input.assembled.bundles)) { - lines.push(` ${quote(id)}: ${renderBundle(bundle)},`); - } - lines.push(' },'); - - // ADR-0056: services that run their build adapter's dev server are lowered - // without an artifact. - const unassembled = Object.keys(input.assembled.devServers); - if (unassembled.length > 0) { - lines.push(` unassembled: [${unassembled.map(quote).join(', ')}],`); - } - - lines.push(' ...transported,'); - lines.push(' providers: localTargetProviders(resolved, transported.containers, devDir, pools),'); - lines.push(' state: devState(),'); - - return lines.join('\n'); -} - -/** Renders the dev stack module's source (tests assert on it without touching disk). */ -export function renderDevStackFile(input: DevStackFileInput): string { - const generatedDir = path.join(input.cwd, devEnvironmentDir(input.environment)); - const appImport = relativeImportSpecifier(generatedDir, input.entryPath); - const configImport = relativeImportSpecifier(generatedDir, input.configFile); - // Kept out of the template so no line of this module's built output starts - // with `import`: check-family-static-graph.mjs reads such lines as the - // module's own imports, and this renderer ships in the testing entry. - const imports = [ - "import * as path from 'node:path';", - "import { readConfigPool, readSecretsPool } from '@prisma/composer/internal/pools';", - "import { lower, transportedLowerInputs } from '@prisma/composer/deploy';", - "import { devEnvironmentDir, devState, localTargetProviders, resolveLocalTargets } from '@prisma/composer/local-target';", - `import prismaConfig from ${quote(configImport)};`, - `import app from ${quote(appImport)};`, - ].join('\n'); - - return `// Generated by \`prisma dev\` — overwritten on every run; do not -// edit by hand. Independently runnable from ${quote(input.cwd)} by the -// command Composer starts, which a failed run prints: -// -// deploy ${devStackRelativePath(input.environment)} --yes --stage ${input.environment} -// -// bisects a CLI bug from an Alchemy bug (deploy-cli.md § Implementation decisions). -// It reads the config and secrets pools again here, from the same adapters, so an adapter must return the same values for the same environment every time. -${imports} - -const config = prismaConfig.composer; -if (config === undefined) { - throw new Error(${quote(noComposerExportMessage(input.configFile))}); -} - -const transported = transportedLowerInputs(config); -const resolved = await resolveLocalTargets(config); -const devDir = path.join(process.cwd(), devEnvironmentDir(${quote(input.environment)})); -const poolInput = { environment: ${quote(input.environment)}, configFile: ${quote(input.configFile)} }; -const pools = { config: await readConfigPool(config, poolInput), secrets: await readSecretsPool(config, poolInput) }; - -export default lower(app, config, { -${renderOptions(input)} -}); -`; -} - -/** Writes the dev stack file, returning its absolute path. */ -export function writeDevStackFile(input: DevStackFileInput): string { - const generatedDir = path.join(input.cwd, devEnvironmentDir(input.environment)); - fs.mkdirSync(generatedDir, { recursive: true }); - const filePath = path.join(generatedDir, DEV_GENERATED_FILE); - fs.writeFileSync(filePath, renderDevStackFile(input)); - return filePath; -} diff --git a/packages/0-framework/3-tooling/cli/src/exports/__tests__/control-import.test.ts b/packages/0-framework/3-tooling/cli/src/exports/__tests__/control-import.test.ts index 7a0aec3c5..79f40c380 100644 --- a/packages/0-framework/3-tooling/cli/src/exports/__tests__/control-import.test.ts +++ b/packages/0-framework/3-tooling/cli/src/exports/__tests__/control-import.test.ts @@ -28,7 +28,7 @@ describe('the ./control entry', () => { ' build.onLoad(\n' + ' {\n' + ' filter:\n' + - ' /(execute-deploy-destroy|execute-dev|execute-log|pipeline|run-alchemy|generate-stack|generate-dev-stack|watch|run-dev|run-log)\\.ts$/,\n' + + ' /(execute-deploy-destroy|execute-dev|execute-log|pipeline|run-alchemy|generate-stack|watch|run-dev|run-log)\\.ts$/,\n' + ' },\n' + ' (args) => {\n' + " throw new Error(`heavy module in the control entry's static graph: ${args.path}`);\n" + diff --git a/packages/0-framework/3-tooling/cli/src/exports/converge.ts b/packages/0-framework/3-tooling/cli/src/exports/converge.ts index a5f81a056..7e43d083c 100644 --- a/packages/0-framework/3-tooling/cli/src/exports/converge.ts +++ b/packages/0-framework/3-tooling/cli/src/exports/converge.ts @@ -1,5 +1,6 @@ /** The in-process converge the CLI loads from the app's install. Not public API: published as `@prisma/composer/internal/converge`, it may change in any release. */ export { lower } from '@internal/core/deploy'; +export { devState, localTargetProviders } from '@internal/core/local-target'; export type { Converge, ConvergeEvent, diff --git a/packages/0-framework/3-tooling/cli/src/exports/testing.ts b/packages/0-framework/3-tooling/cli/src/exports/testing.ts index 44c7936bb..3f15e093a 100644 --- a/packages/0-framework/3-tooling/cli/src/exports/testing.ts +++ b/packages/0-framework/3-tooling/cli/src/exports/testing.ts @@ -1,13 +1,9 @@ /** * Public surface (the `./testing` subpath, republished through - * `@prisma/composer-cli/testing`): the fixture-backed operations double, and - * the dev stack-file renderer for integration tests that drive Alchemy - * directly. Implementation lives in ../testing/operations-double.ts and - * ../dev/generate-dev-stack.ts. + * `@prisma/composer-cli/testing`): the fixture-backed operations double. + * Implementation lives in ../testing/operations-double.ts. */ -export type { DevStackFileInput } from '../dev/generate-dev-stack.ts'; -export { renderDevStackFile } from '../dev/generate-dev-stack.ts'; export type { OperationsDouble, OperationsDoubleCalls, diff --git a/packages/0-framework/3-tooling/cli/src/family/__tests__/dev-converge-child.test.ts b/packages/0-framework/3-tooling/cli/src/family/__tests__/dev-converge-child.test.ts deleted file mode 100644 index 5711a1958..000000000 --- a/packages/0-framework/3-tooling/cli/src/family/__tests__/dev-converge-child.test.ts +++ /dev/null @@ -1,116 +0,0 @@ -/** - * What the converge child writes when the parent's stdout is a pipe. - * - * In human format the engine starts the child with `output: "inherit"`, so a - * piped `prisma deploy` hands its own stdout pipe to Alchemy. That pipe must - * carry only the parent's data lines, so everything the child prints goes to - * stderr instead. - */ -import { afterEach, describe, expect, test } from 'bun:test'; -import { spawn } from 'node:child_process'; -import * as fs from 'node:fs'; -import * as os from 'node:os'; -import * as path from 'node:path'; -import type { ChildResult } from '@prisma/cli-engine'; -import { alchemyInvocation } from '../../run-alchemy.ts'; -import { type ConvergeContext, convergeSpawn } from '../dev-converge-child.ts'; - -const tmpDirs: string[] = []; - -afterEach(() => { - for (const dir of tmpDirs.splice(0)) fs.rmSync(dir, { recursive: true, force: true }); -}); - -/** An app directory whose `@prisma/composer` depends on an `alchemy` that runs `body`. */ -function appWithAlchemy(body: readonly string[]): string { - const dir = fs.realpathSync(fs.mkdtempSync(path.join(os.tmpdir(), 'composer-converge-spawn-'))); - tmpDirs.push(dir); - const nodeModules = path.join(dir, 'node_modules'); - fs.mkdirSync(path.join(nodeModules, '@prisma', 'composer'), { recursive: true }); - fs.writeFileSync( - path.join(nodeModules, '@prisma', 'composer', 'package.json'), - JSON.stringify({ name: '@prisma/composer', exports: { './package.json': './package.json' } }), - ); - fs.mkdirSync(path.join(nodeModules, 'alchemy', 'bin'), { recursive: true }); - fs.writeFileSync( - path.join(nodeModules, 'alchemy', 'package.json'), - JSON.stringify({ name: 'alchemy', type: 'commonjs', bin: { alchemy: './bin/cli.js' } }), - ); - fs.writeFileSync(path.join(nodeModules, 'alchemy', 'bin', 'cli.js'), body.join('\n')); - return dir; -} - -/** The `prisma` host's spawn in `inherit` mode, where the inherited stdout is a pipe. */ -function pipedInheritSpawn(captured: { stdout: string; stderr: string }): ConvergeContext { - return { - lastChild: () => undefined, - spawn: (options) => - new Promise((resolve, reject) => { - const child = spawn(options.command, [...(options.args ?? [])], { - cwd: options.cwd, - env: { ...process.env, ...options.env }, - stdio: ['ignore', 'pipe', 'pipe'], - }); - child.stdout.on('data', (chunk) => { - captured.stdout += String(chunk); - }); - child.stderr.on('data', (chunk) => { - captured.stderr += String(chunk); - }); - child.on('error', reject); - child.on('close', (exitCode, signal) => resolve({ exitCode, signal })); - }), - }; -} - -describe('convergeSpawn() with a piped stdout', () => { - test("the child's console output and stdout writes reach stderr, and stdout stays empty", async () => { - const cwd = appWithAlchemy([ - "console.log('Deploy · shop');", - "console.info('[web] created');", - "console.debug('debug detail');", - "process.stdout.write('Plan: 1 to create\\n');", - ]); - const captured = { stdout: '', stderr: '' }; - const run = convergeSpawn(pipedInheritSpawn(captured)); - - const outcome = await run( - alchemyInvocation({ - command: 'deploy', - stackFileRelativePath: '.prisma-composer/alchemy.run.ts', - cwd, - stage: 'test', - containerEnv: {}, - }), - ); - - expect(outcome.exitCode).toBe(0); - expect(captured).toEqual({ - stdout: '', - stderr: 'Deploy · shop\n[web] created\ndebug detail\nPlan: 1 to create\n', - }); - }, 15_000); - - test('the reproduce command stays the plain Alchemy command line', async () => { - const cwd = appWithAlchemy([]); - const run = convergeSpawn(pipedInheritSpawn({ stdout: '', stderr: '' })); - - const outcome = await run( - alchemyInvocation({ - command: 'deploy', - stackFileRelativePath: '.prisma-composer/alchemy.run.ts', - cwd, - stage: 'test', - containerEnv: {}, - }), - ); - - expect(outcome.commandLine?.args.slice(1)).toEqual([ - 'deploy', - '.prisma-composer/alchemy.run.ts', - '--yes', - '--stage', - 'test', - ]); - }, 15_000); -}); diff --git a/packages/0-framework/3-tooling/cli/src/family/__tests__/dev-converge.test.ts b/packages/0-framework/3-tooling/cli/src/family/__tests__/dev-converge.test.ts new file mode 100644 index 000000000..433d5694d --- /dev/null +++ b/packages/0-framework/3-tooling/cli/src/family/__tests__/dev-converge.test.ts @@ -0,0 +1,178 @@ +/** How `prisma dev` renders its in-process converges, and how Ctrl-C during the first one settles. */ +import { describe, expect, test } from 'bun:test'; +import { CliStructuredError } from '@internal/foundation/errors'; +import { notOk } from '@internal/foundation/result'; +import type { EngineEvent } from '@prisma/cli-engine'; +import { createTestCli } from '@prisma/cli-engine/testing'; +import type { DevEvent } from '../../operations/dev.ts'; +import { createOperationsDouble } from '../../testing/operations-double.ts'; +import { type ComposerOperations, createComposerFamily } from '../family.ts'; +import { commandAt } from './command-at.ts'; +import { validComposerSection } from './fixtures/composer-section.ts'; + +function cliWith(operations: ComposerOperations) { + const family = createComposerFamily({ operations }); + return createTestCli({ + commandFamilies: [family], + commands: { deploy: commandAt(family, 'deploy'), dev: commandAt(family, 'dev') }, + config: { composer: validComposerSection() }, + }); +} + +/** Runs `dev` against a session double that sends `devEvents`, and stops it once the last of them is reported. */ +async function runWithEvents( + devEvents: readonly DevEvent[], + last: (event: EngineEvent) => boolean, +) { + const double = createOperationsDouble({ + devEndpoints: [{ address: 'shop.web', url: 'http://localhost:3000' }], + devEvents, + }); + const interrupt = new AbortController(); + const result = await cliWith(double.operations).run(['dev', '--entry', 'src/service.ts'], { + isTty: { stdout: true, stderr: true }, + cwd: '/repo/shop', + abort: interrupt.signal, + onEvent: (event: EngineEvent) => { + if (last(event)) interrupt.abort(); + }, + }); + return { result, double }; +} + +describe('dev — converging in process', () => { + test("hands the run's signal to the operation and starts no child", async () => { + const { result, double } = await runWithEvents( + [], + (event) => event.kind === 'status' && event.status === 'ready', + ); + + expect(double.calls.dev[0]?.signal).toBeInstanceOf(AbortSignal); + expect(double.calls.deps.dev[0]?.alchemy).toBeUndefined(); + expect(commandAt(createComposerFamily({ operations: double.operations }), 'dev').maySpawn).toBe( + false, + ); + expect(result.exitCode).toBe(130); + }); + + test('each resource is a step, a failed one with its message, and log lines are messages', async () => { + const { result } = await runWithEvents( + [ + { + kind: 'converge-resource', + fqn: 'shop.web', + resourceType: 'Local.Compute', + status: 'updating', + phase: 'running', + }, + { + kind: 'converge-resource', + fqn: 'shop.web', + resourceType: 'Local.Compute', + status: 'updated', + phase: 'done', + }, + { + kind: 'converge-resource', + fqn: 'shop.db', + resourceType: 'Local.Postgres', + status: 'fail', + phase: 'failed', + message: 'port in use', + }, + { kind: 'converge-message', severity: 'info', text: 'Plan: 1 to update' }, + ], + (event) => event.kind === 'message' && event.text === 'Plan: 1 to update', + ); + + expect(result.events).toContainEqual({ + kind: 'step-started', + step: 'updating shop.web', + id: 'dev.converge.shop.web', + data: { resourceType: 'Local.Compute' }, + }); + expect(result.events).toContainEqual({ + kind: 'step-finished', + step: 'updated shop.web', + id: 'dev.converge.shop.web', + outcome: 'ok', + }); + expect(result.events).toContainEqual({ + kind: 'step-finished', + step: 'fail shop.db', + id: 'dev.converge.shop.db', + outcome: 'failed', + }); + expect(result.events).toContainEqual({ + kind: 'message', + severity: 'warn', + text: 'shop.db: port in use', + }); + expect( + result.events.filter( + (event) => event.kind === 'step-started' && event.id === 'dev.converge.shop.web', + ), + ).toHaveLength(1); + }); + + test('a failed rebuild is a warning with the cause, and no reproduce command', async () => { + const text = + 'Converge failed; the running app is untouched and dev is still watching.\nProviderError: the provider refused'; + const { result } = await runWithEvents( + [ + { + kind: 'converge-failed', + message: 'ProviderError: the provider refused', + reasons: [{ tag: 'ProviderError', message: 'the provider refused' }], + }, + ], + (event) => event.kind === 'message' && event.text === text, + ); + + expect(result.events).toContainEqual({ + kind: 'message', + severity: 'warn', + text, + data: { reasons: [{ tag: 'ProviderError', message: 'the provider refused' }] }, + }); + expect(result.events.some((event) => event.kind === 'remediation')).toBe(false); + }); + + test('an edit to the app definition renders as a warning that asks for a restart', async () => { + const text = + "/repo/shop/src/service.ts changed. dev loaded the app's definition when it started and cannot load it again; restart dev to apply it. Rebuilds are paused until then."; + const { result } = await runWithEvents( + [{ kind: 'definition-changed', files: ['/repo/shop/src/service.ts'] }], + (event) => event.kind === 'message' && event.text === text, + ); + + expect(result.events).toContainEqual({ kind: 'message', severity: 'warn', text }); + }); + + test('Ctrl-C during the first converge settles 130, reporting the interruption, not an error', async () => { + const double = createOperationsDouble({}); + const interrupt = new AbortController(); + const operations: ComposerOperations = { + ...double.operations, + dev: async () => { + interrupt.abort(); + return notOk( + new CliStructuredError('DEV.INTERRUPTED', 'dev was interrupted while it converged.', { + fix: 'Run dev again to finish.', + }), + ); + }, + }; + + const result = await cliWith(operations).run(['dev', '--entry', 'src/service.ts', '--json'], { + abort: interrupt.signal, + }); + + expect(result.exitCode).toBe(130); + expect(result.events).toContainEqual({ + kind: 'message', + severity: 'warn', + text: 'dev was interrupted while it converged. Run dev again to finish.', + }); + }); +}); diff --git a/packages/0-framework/3-tooling/cli/src/family/__tests__/dev-emulators-dir.test.ts b/packages/0-framework/3-tooling/cli/src/family/__tests__/dev-emulators-dir.test.ts index b6e7ca6d7..5227b1158 100644 --- a/packages/0-framework/3-tooling/cli/src/family/__tests__/dev-emulators-dir.test.ts +++ b/packages/0-framework/3-tooling/cli/src/family/__tests__/dev-emulators-dir.test.ts @@ -15,6 +15,7 @@ import type { LocalTargetDescriptor } from '@internal/core/local-target'; import { CliStructuredError } from '@internal/foundation/errors'; import { createTestCli } from '@prisma/cli-engine/testing'; import * as Layer from 'effect/Layer'; +import type { AppConverge } from '../../app-converge.ts'; import { devWithDeps } from '../../operations/dev.ts'; import { createComposerFamily, realOperations } from '../family.ts'; import { commandAt } from './command-at.ts'; @@ -110,6 +111,13 @@ function configWhoseEmulatorsThrow(): PrismaAppConfig { }; } +const notReached: AppConverge = { + lower: unused, + converge: unused, + localTargetProviders: unused, + devState: unused, +}; + const fakeAssembler = async (node: ServiceNode) => ({ dir: path.join(path.dirname(fileURLToPath(node.build.module)), 'dist', 'bundle'), entry: 'server.js', @@ -123,7 +131,8 @@ describe.skipIf(process.platform === 'win32')( const family = createComposerFamily({ operations: { ...realOperations, - dev: (input, deps) => devWithDeps(input, { ...deps, runAssembler: fakeAssembler }), + dev: (input, deps) => + devWithDeps(input, { ...deps, runAssembler: fakeAssembler, converge: notReached }), }, }); const cli = createTestCli({ diff --git a/packages/0-framework/3-tooling/cli/src/family/__tests__/fake-child.test.ts b/packages/0-framework/3-tooling/cli/src/family/__tests__/fake-child.test.ts deleted file mode 100644 index 611a305af..000000000 --- a/packages/0-framework/3-tooling/cli/src/family/__tests__/fake-child.test.ts +++ /dev/null @@ -1,83 +0,0 @@ -/** - * The fake-child fixture is itself a contract: D3's spawn tests state a - * child's behavior on argv and trust it. Each scripted behavior is proven - * here against a real spawned process, so a test that fails over the fixture - * is failing over the code under test, not the fixture. - */ -import { describe, expect, test } from 'bun:test'; -import { spawn, spawnSync } from 'node:child_process'; -import * as path from 'node:path'; - -const FIXTURE = path.join(import.meta.dir, 'fixtures', 'fake-child.mjs'); - -function run(args: readonly string[], env?: Record) { - return spawnSync(process.execPath, [FIXTURE, ...args], { - encoding: 'utf-8', - ...(env === undefined ? {} : { env }), - }); -} - -describe('the fake child', () => { - test('exits with the scripted code — including the non-zero ones spawn tests must pass through', () => { - for (const code of [0, 1, 2, 3]) { - expect(run(['--exit', String(code)]).status).toBe(code); - } - }); - - test('writes scripted stdout and stderr, unframed', () => { - const result = run(['--stdout', 'plain child output', '--stderr', 'child warning']); - expect(result.stdout).toBe('plain child output\n'); - expect(result.stderr).toBe('child warning\n'); - }); - - test('reports env-key presence, never the value', () => { - const set = run(['--report-env', 'PRISMA_SERVICE_TOKEN'], { - PRISMA_SERVICE_TOKEN: 'secret-token-material', - }); - expect(set.stdout).toBe('env:PRISMA_SERVICE_TOKEN=set\n'); - expect(set.stdout).not.toContain('secret-token-material'); - expect(run(['--report-env', 'PRISMA_SERVICE_TOKEN'], {}).stdout).toBe( - 'env:PRISMA_SERVICE_TOKEN=unset\n', - ); - }); - - test('a lingering child dies by the signal by default — a signal-killed child, not an exit code', async () => { - const child = spawn(process.execPath, [FIXTURE, '--linger']); - await new Promise((resolve) => setTimeout(resolve, 150)); - child.kill('SIGTERM'); - const [code, signal] = await new Promise<[number | null, string | null]>((resolve) => { - child.on('exit', (exitCode, exitSignal) => resolve([exitCode, exitSignal])); - }); - expect(code).toBeNull(); - expect(signal).toBe('SIGTERM'); - }); - - test.skipIf(process.platform === 'win32')( - 'a lingering child scripted to report a signal names it and exits 0', - async () => { - const child = spawn(process.execPath, [FIXTURE, '--linger', '--on-signal', 'report']); - const chunks: string[] = []; - child.stdout.on('data', (chunk: Buffer) => chunks.push(chunk.toString())); - await new Promise((resolve) => setTimeout(resolve, 150)); - child.kill('SIGINT'); - const code = await new Promise((resolve) => { - child.on('close', (exitCode) => resolve(exitCode)); - }); - expect(code).toBe(0); - expect(chunks.join('')).toBe('signal:SIGINT\n'); - }, - ); - - test.skipIf(process.platform === 'win32')( - 'a lingering child scripted to ignore signals survives them — the escalation-ladder case', - async () => { - const child = spawn(process.execPath, [FIXTURE, '--linger', '--on-signal', 'ignore']); - await new Promise((resolve) => setTimeout(resolve, 150)); - child.kill('SIGTERM'); - await new Promise((resolve) => setTimeout(resolve, 150)); - expect(child.exitCode).toBeNull(); - child.kill('SIGKILL'); - await new Promise((resolve) => child.on('exit', resolve)); - }, - ); -}); diff --git a/packages/0-framework/3-tooling/cli/src/family/__tests__/fixtures/fake-child.mjs b/packages/0-framework/3-tooling/cli/src/family/__tests__/fixtures/fake-child.mjs deleted file mode 100644 index 704709903..000000000 --- a/packages/0-framework/3-tooling/cli/src/family/__tests__/fixtures/fake-child.mjs +++ /dev/null @@ -1,59 +0,0 @@ -#!/usr/bin/env node -// The scripted fake child (contract R-S3-5): a real program for spawn tests -// to run instead of alchemy. Its whole behavior is declared on argv, so a -// test states the child it needs and asserts what came back: -// -// --exit exit with code n (default 0) -// --stdout write text + newline to stdout, unframed -// --stderr write text + newline to stderr -// --report-env write "env:=set" or "env:=unset" to stdout — -// key presence only, never the value (the credential -// tests assert env KEYS, never token material) -// --linger stay alive until a signal arrives -// --on-signal with --linger: "exit" (default) dies by the signal's -// default disposition; "ignore" swallows SIGINT/SIGTERM -// and keeps lingering (the escalation-ladder case); -// "report" writes "signal:" to stdout, then exits 0 -// -// Plain JS on purpose: it is spawned as a real operating-system child, so it -// must run with no transpilation under whatever runtime the suite spawns. - -const args = process.argv.slice(2); - -function argValue(flag) { - const index = args.indexOf(flag); - return index === -1 ? undefined : args[index + 1]; -} - -const stdout = argValue('--stdout'); -if (stdout !== undefined) process.stdout.write(`${stdout}\n`); - -const stderr = argValue('--stderr'); -if (stderr !== undefined) process.stderr.write(`${stderr}\n`); - -const envKey = argValue('--report-env'); -if (envKey !== undefined) { - process.stdout.write(`env:${envKey}=${envKey in process.env ? 'set' : 'unset'}\n`); -} - -if (args.includes('--linger')) { - const onSignal = argValue('--on-signal') ?? 'exit'; - if (onSignal === 'ignore') { - process.on('SIGINT', () => {}); - process.on('SIGTERM', () => {}); - } else if (onSignal === 'report') { - // Exiting from the write callback, not after the call: stdout is a pipe - // here, and an exit issued while the write is still pending discards the - // report the reader is waiting for. - const report = (name) => { - process.stdout.write(`signal:${name}\n`, () => process.exit(0)); - }; - process.on('SIGINT', () => report('SIGINT')); - process.on('SIGTERM', () => report('SIGTERM')); - } - // With mode "exit" no handler is installed: the signal's default - // disposition kills the process, which is what a signal-killed child is. - setInterval(() => {}, 1 << 30); -} else { - process.exitCode = Number(argValue('--exit') ?? '0'); -} diff --git a/packages/0-framework/3-tooling/cli/src/family/commands/dev.ts b/packages/0-framework/3-tooling/cli/src/family/commands/dev.ts index 17ab7159c..51722ede8 100644 --- a/packages/0-framework/3-tooling/cli/src/family/commands/dev.ts +++ b/packages/0-framework/3-tooling/cli/src/family/commands/dev.ts @@ -3,10 +3,11 @@ * user interrupts it. * * The watch loop, the emulators and the live attachments stay the operation's - * state. What the handler owns is the boundary: the converge child goes out - * through `ctx.spawn`, so the terminal reaches alchemy natively and the engine - * owns signal policy; the operation's events become engine events; and the - * session settles on how it actually ended. + * state. What the handler owns is the boundary: the operation's events become + * engine events, Ctrl-C reaches a running converge through `ctx.signal`, and + * the session settles on how it actually ended. Every converge runs in this + * process, so `console`'s stdout methods write to stderr for the whole + * session, as they do for `deploy`. * * Windows is refused inside the operation itself (DEV.PLATFORM_UNSUPPORTED), * which is why nothing here reads the platform. @@ -14,15 +15,15 @@ * `dev` is credential-free: everything it starts runs on this machine. */ import * as path from 'node:path'; -import type { ChildResult, EngineEvent } from '@prisma/cli-engine'; +import type { EngineEvent } from '@prisma/cli-engine'; import { defineSessionCommand, flag } from '@prisma/cli-engine'; -import { ok } from '@prisma/cli-engine/protocol'; +import { notOk, ok } from '@prisma/cli-engine/protocol'; import type { DevEvent, DevInput } from '../../operations/dev.ts'; import { DEV_ENVIRONMENT, type ServiceEndpoint } from '../../operations/shared.ts'; -import type { AlchemyInvocation, AlchemyOutcome, RunAlchemy } from '../../run-alchemy.ts'; -import { convergeSpawn, settleConvergeFailure } from '../dev-converge-child.ts'; +import { withConsoleOnStderr } from '../console-to-stderr.ts'; import type { ComposerOperations } from '../family.ts'; import { composerSection } from '../section.ts'; +import { toEngineError } from '../translate-error.ts'; const STOP_STEP = "Stopping the app's services — emulators and data stay up"; @@ -34,43 +35,6 @@ function stopRequested(signal: AbortSignal): Promise { }); } -/** - * One converge at a time. `ctx.spawn` refuses a second live child, and the - * watch loop can well fire a rebuild while the previous converge is still - * running — so a request that arrives during a live converge joins the single - * queued follow-up instead of starting its own. That extends the coalescing - * the watcher already does across a burst of edits to cover the converge - * itself, and it is sound because every dev converge is the SAME invocation: - * one stack file at one fixed path, rewritten in place before each request. - */ -function coalescedConverge(run: RunAlchemy): RunAlchemy { - let live: Promise | undefined; - let queued: Promise | undefined; - - const start = (invocation: AlchemyInvocation): Promise => { - const outcome = run(invocation); - const settled = outcome.then( - () => undefined, - () => undefined, - ); - live = settled; - void settled.then(() => { - if (live === settled) live = undefined; - }); - return outcome; - }; - - return (invocation) => { - const running = live; - if (running === undefined) return start(invocation); - queued ??= running.then(() => { - queued = undefined; - return start(invocation); - }); - return queued; - }; -} - /** The front door, ordered by address depth (fewest dots first) then lexicographically. */ function frontDoorOrder(endpoints: readonly ServiceEndpoint[]): readonly ServiceEndpoint[] { return [...endpoints].sort((a, b) => { @@ -86,10 +50,10 @@ function frontDoorOrder(endpoints: readonly ServiceEndpoint[]): readonly Service */ function reportDevEvent( report: (event: EngineEvent) => void, - lastChild: () => ChildResult | undefined, cwd: string, ): (event: DevEvent) => void { let stopFailed = false; + const openResources = new Set(); return (event) => { switch (event.kind) { @@ -165,33 +129,48 @@ function reportDevEvent( report({ kind: 'message', severity: 'warn', text: `Watch error: ${event.message}` }); return; - case 'converge-failed': { - const child = lastChild(); - // A signal-killed converge is the user shutting the session down, not - // a converge that failed — the engine replays that signal into - // ctx.signal the moment the child ends, and the teardown below is what - // reports it. - if (child !== undefined && child.signal !== null) return; + case 'definition-changed': report({ kind: 'message', severity: 'warn', - text: - 'Converge failed — the running app is untouched; still watching.\n' + - `Generated stack file: ${event.stackFilePath}\n` + - `Run \`${event.reproduceCommand}\` from ${event.cwd} to reproduce this directly.`, + text: `${event.files.join(', ')} changed. dev loaded the app's definition when it started and cannot load it again; restart dev to apply it. Rebuilds are paused until then.`, }); + return; + + case 'converge-resource': { + const id = `dev.converge.${event.fqn}`; + const step = `${event.status} ${event.fqn}`; + if (!openResources.has(id)) { + openResources.add(id); + report({ kind: 'step-started', step, id, data: { resourceType: event.resourceType } }); + } + if (event.phase === 'running') return; + openResources.delete(id); report({ - kind: 'remediation', - action: { - kind: 'run-command', - label: `Run the converge directly from ${event.cwd} to reproduce this`, - command: event.reproduceCommand, - reason: `Generated stack file: ${event.stackFilePath}`, - }, + kind: 'step-finished', + step, + id, + outcome: event.phase === 'done' ? 'ok' : 'failed', }); + if (event.message !== undefined) { + report({ kind: 'message', severity: 'warn', text: `${event.fqn}: ${event.message}` }); + } return; } + case 'converge-message': + report({ kind: 'message', severity: event.severity, text: event.text }); + return; + + case 'converge-failed': + report({ + kind: 'message', + severity: 'warn', + text: `Converge failed; the running app is untouched and dev is still watching.\n${event.message}`, + data: { reasons: event.reasons }, + }); + return; + case 'stopping': report({ kind: 'step-started', step: STOP_STEP, id: 'dev-stop' }); return; @@ -243,7 +222,9 @@ export const createDevCommand = (operations: ComposerOperations) => "Bring up the application whose root node is the entry module's default export, entirely on " + 'this machine.', description: - 'Runs credential-free and watches the app for changes, reconverging on every edit. ' + + 'Runs credential-free and watches the app for changes, reconverging when a ' + + "service's build output changes. An edit to the entry module or to a file a service " + + 'is declared in needs a restart of dev, which says so. ' + "A service whose build adapter declares a dev server (Next.js's `next dev`, or the " + '`dev` command written on node()) runs that instead of its build, and its output is ' + 'shown here, prefixed with its address. ' + @@ -302,49 +283,57 @@ export const createDevCommand = (operations: ComposerOperations) => }, }, needs: { config: composerSection }, - maySpawn: true, - handler: async (args, ctx) => { - const alchemy = convergeSpawn(ctx); - const result = await operations.dev( - { - entry: args.flags.entry, - config: ctx.config, - name: args.flags.name, - environment: args.flags.environment, - fresh: args.flags.fresh, - detach: args.flags.detach, - cwd: ctx.cwd, - onEvent: reportDevEvent(ctx.report, ctx.lastChild, ctx.cwd), - }, - { alchemy: coalescedConverge(alchemy) }, - ); + handler: (args, ctx) => + withConsoleOnStderr(async () => { + const result = await operations.dev( + { + entry: args.flags.entry, + config: ctx.config, + name: args.flags.name, + environment: args.flags.environment, + fresh: args.flags.fresh, + detach: args.flags.detach, + cwd: ctx.cwd, + onEvent: reportDevEvent(ctx.report, ctx.cwd), + signal: ctx.signal, + }, + {}, + ); - // Nothing is live yet, so the ending is the converge's — settled by the - // same decision deploy and destroy settle by, which is what keeps the - // three from drifting. A signal-killed converge needs no branch: the - // engine settles the run from its own record of the signal. - if (!result.ok) return settleConvergeFailure(result.failure, ctx); + // Ctrl-C during the first converge: the engine settles 130 from its + // own record of the signal, so the interruption is only reported. + if (!result.ok) { + if (result.failure.code === 'DEV.INTERRUPTED' && ctx.signal.aborted) { + ctx.report({ + kind: 'message', + severity: 'warn', + text: `${result.failure.message} ${result.failure.fix ?? ''}`.trim(), + }); + return ok(undefined); + } + return notOk(toEngineError(result.failure)); + } - if (args.flags.detach) { - if (ctx.signal.aborted) { - await stopDetachedApp( - operations, - { - entry: args.flags.entry, - name: args.flags.name, - environment: args.flags.environment, - }, - ctx, - ); + if (args.flags.detach) { + if (ctx.signal.aborted) { + await stopDetachedApp( + operations, + { + entry: args.flags.entry, + name: args.flags.name, + environment: args.flags.environment, + }, + ctx, + ); + } + return ok(undefined); } - return ok(undefined); - } - await stopRequested(ctx.signal); - await result.value.stop(); - // Ctrl-C settles 130 because the engine records the signal that fired and - // settles at 128 + it, including for a handler that cleaned up and - // returned. Reporting the termination is not this handler's to do. - return ok(undefined); - }, + await stopRequested(ctx.signal); + await result.value.stop(); + // Ctrl-C settles 130 because the engine records the signal that fired and + // settles at 128 + it, including for a handler that cleaned up and + // returned. Reporting the termination is not this handler's to do. + return ok(undefined); + }), }); diff --git a/packages/0-framework/3-tooling/cli/src/family/dev-converge-child.ts b/packages/0-framework/3-tooling/cli/src/family/dev-converge-child.ts deleted file mode 100644 index fe2a54db5..000000000 --- a/packages/0-framework/3-tooling/cli/src/family/dev-converge-child.ts +++ /dev/null @@ -1,94 +0,0 @@ -/** - * The `dev` command's converge child: the spawn adapter backed by the - * engine's `ctx.spawn`, and the settlement rules for how the child ended. - * `deploy` converges in this process and settles as an ordinary command. - */ - -import type { CliStructuredError as ComposerError } from '@internal/foundation/errors'; -import type { ChildResult, ChildStatusSettlement } from '@prisma/cli-engine'; -import { exitWithChildStatus } from '@prisma/cli-engine'; -import type { CliStructuredError, NextAction, Result } from '@prisma/cli-engine/protocol'; -import { notOk, ok } from '@prisma/cli-engine/protocol'; -import { executionDiagnostics } from '../operations/shared.ts'; -import { alchemyCommandLine, type RunAlchemy } from '../run-alchemy.ts'; -import { toEngineError } from './translate-error.ts'; - -/** The subset of the engine context this module needs, so the helpers stay - * testable without building a whole context. */ -export interface ConvergeContext { - readonly spawn: (options: { - readonly command: string; - readonly args?: readonly string[]; - readonly cwd?: string; - readonly env?: Readonly>; - }) => Promise; - /** How the run's most recent completed child ended, or undefined when none - * ran — the operation failed before reaching the converge, or the spawn - * never launched. The engine keeps this record; composer does not. */ - readonly lastChild: () => ChildResult | undefined; -} - -/** - * Node's `--import` of a module that sends the child's stdout to stderr when - * stdout is not a terminal. In human format the engine hands a piped run's - * stdout to the child, and that pipe carries only the parent's data lines. - */ -const STDOUT_TO_STDERR = [ - '--import', - `data:text/javascript,${encodeURIComponent( - 'if (process.stdout.isTTY !== true) process.stdout.write = process.stderr.write.bind(process.stderr);', - )}`, -] as const; - -/** The adapter the operations call, backed by the engine's `ctx.spawn` so the - * terminal reaches alchemy natively and the engine owns signal policy. */ -export function convergeSpawn(ctx: ConvergeContext): RunAlchemy { - return async (invocation) => { - const commandLine = alchemyCommandLine(invocation); - const result = await ctx.spawn({ - command: commandLine.command, - args: [...STDOUT_TO_STDERR, ...commandLine.args], - cwd: commandLine.cwd, - env: commandLine.env, - }); - return { ...result, commandLine }; - }; -} - -/** - * The reproduce hint a failed converge carries. Deliberately absent for an - * aborted one: the user stopped it, so there is nothing to reproduce. - */ -export function reproduceHint(failure: ComposerError): readonly NextAction[] { - const diagnostics = executionDiagnostics(failure); - if (diagnostics === undefined) return []; - return [ - { - kind: 'run-command', - label: `Run the converge directly from ${diagnostics.cwd} to reproduce this`, - command: diagnostics.reproduceCommand, - reason: `Generated stack file: ${diagnostics.stackFilePath}`, - }, - ]; -} - -/** - * How a FAILED `dev` converge settles. - * - * A failure that reached the child exits with the child's status verbatim and - * carries the reproduce hint; a failure that never reached the child is an - * ordinary structured error and gets the normal envelope. Nothing here reads - * `signal`: `exitWithChildStatus` settles from the engine's own record of the - * child, and a signal-killed one overrules it with 128 + the signal and no - * reproduce hint. - */ -export function settleConvergeFailure( - failure: ComposerError, - ctx: ConvergeContext, -): Result { - const child = ctx.lastChild(); - if (child !== undefined && child.exitCode !== 0) { - return ok(exitWithChildStatus({ nextActions: reproduceHint(failure) })); - } - return notOk(toEngineError(failure)); -} diff --git a/packages/0-framework/3-tooling/cli/src/operations/__tests__/operations.test.ts b/packages/0-framework/3-tooling/cli/src/operations/__tests__/operations.test.ts index c26b04f29..acfafe675 100644 --- a/packages/0-framework/3-tooling/cli/src/operations/__tests__/operations.test.ts +++ b/packages/0-framework/3-tooling/cli/src/operations/__tests__/operations.test.ts @@ -20,11 +20,13 @@ import type { PrismaAppConfig, RunOutcome, } from '@internal/core/config'; -import { lower } from '@internal/core/deploy'; -import type { - LocalTargetAttachment, - LocalTargetCurrentApp, - LocalTargetDescriptor, +import { type LowerOptions, lower } from '@internal/core/deploy'; +import { + devState, + type LocalTargetAttachment, + type LocalTargetCurrentApp, + type LocalTargetDescriptor, + localTargetProviders, } from '@internal/core/local-target'; import { CliStructuredError } from '@internal/foundation/errors'; import * as Layer from 'effect/Layer'; @@ -202,10 +204,14 @@ const fakeAssembler = async (node: ServiceNode) => ({ }); const APPLIED: ConvergeOutcome = { kind: 'applied' }; +const FAILED: ConvergeOutcome = { + kind: 'failed', + reasons: [{ tag: 'ProviderError', message: 'the provider refused' }], +}; /** Core's own `lower` with a stand-in for Alchemy's plan and apply: deploy tests never run the real Alchemy here. */ function appConverge(converge: Converge): AppConverge { - return { lower, converge }; + return { lower, converge, localTargetProviders, devState }; } // Deploy needs a service token; a caller passing no credentials leaves it to the environment. @@ -903,7 +909,7 @@ describe('deploy() credentials, app converge and redaction', () => { ); fs.writeFileSync( path.join(pkg, 'converge.mjs'), - "export const lower = () => ({}); export const convergeInProcess = async (request) => ({ kind: 'applied', cwd: request.cwd });", + "export const lower = () => ({}); export const convergeInProcess = async (request) => ({ kind: 'applied', cwd: request.cwd }); export const localTargetProviders = () => ({}); export const devState = () => ({});", ); const result = await silently(() => @@ -967,10 +973,10 @@ describe('deploy() credentials, app converge and redaction', () => { { credentials, converge: { + ...appConverge(applied), lower: () => { throw new Error(`state store refused ${token}`); }, - converge: applied, }, }, stateBroken, @@ -1828,11 +1834,16 @@ describe.skipIf(process.platform === 'win32')('dev()', () => { logs: async function* () {}, }; - async function childEnvOfDev( - preflight: LocalTargetDescriptor['preflight'], - ): Promise>> { + /** Runs a detached dev session and returns what it lowered and converged. */ + async function convergedDev(preflight: LocalTargetDescriptor['preflight']): Promise<{ + readonly lowered: LowerOptions[]; + readonly requests: ConvergeRequest[]; + readonly spawned: number; + }> { const app = makeAppDir('hello-dev'); - let childEnv: Readonly> = {}; + const lowered: LowerOptions[] = []; + const requests: ConvergeRequest[] = []; + let spawned = 0; const result = await silently(() => devWithDeps( { @@ -1843,44 +1854,50 @@ describe.skipIf(process.platform === 'win32')('dev()', () => { }, { runAssembler: fakeAssembler, - alchemy: async (invocation) => { - childEnv = invocation.env; + alchemy: async () => { + spawned += 1; return { exitCode: 0, signal: null }; }, + converge: { + ...appConverge(async (request) => { + requests.push(request); + return APPLIED; + }), + lower: (root, config, opts) => { + lowered.push(opts); + return lower(root, config, opts); + }, + }, }, ), ); expect(result.ok).toBe(true); - return childEnv; + return { lowered, requests, spawned }; } - test("the dev preflight's payload reaches the alchemy process", async () => { - const env = await childEnvOfDev(() => Promise.resolve('dev:{"absentSecrets":[]}')); + test("the dev preflight's payload reaches lower() as a value, and the converge sets no environment variable", async () => { + const { lowered, requests } = await convergedDev(() => + Promise.resolve('dev:{"absentSecrets":[]}'), + ); - expect(env['PRISMA_COMPOSER_PREFLIGHT_FIXTURE_EXTENSION']).toBe('dev:{"absentSecrets":[]}'); + expect(lowered).toHaveLength(1); + expect(lowered[0]?.preflight).toEqual( + new Map([['fixture-extension', 'dev:{"absentSecrets":[]}']]), + ); + expect(lowered[0]?.containers.get('fixture-extension')?.serialize()).toBe('dev'); + expect(requests.map((request) => request.env)).toEqual([{}]); }, 15_000); - test('a stale payload in the shell is blanked for every configured extension', async () => { - const saved = { - own: process.env['PRISMA_COMPOSER_PREFLIGHT_FIXTURE_EXTENSION'], - build: process.env['PRISMA_COMPOSER_PREFLIGHT_FIXTURE_BUILD'], - }; - process.env['PRISMA_COMPOSER_PREFLIGHT_FIXTURE_EXTENSION'] = 'stale'; - process.env['PRISMA_COMPOSER_PREFLIGHT_FIXTURE_BUILD'] = 'stale'; - try { - const env = await childEnvOfDev(() => Promise.resolve(undefined)); + test('dev lowers against the local providers and state store, and converges as a deploy in this process, starting no Alchemy child', async () => { + const { lowered, requests, spawned } = await convergedDev(() => Promise.resolve(undefined)); - expect(env['PRISMA_COMPOSER_PREFLIGHT_FIXTURE_EXTENSION']).toBe(''); - expect(env['PRISMA_COMPOSER_PREFLIGHT_FIXTURE_BUILD']).toBe(''); - } finally { - for (const [key, name] of [ - ['own', 'PRISMA_COMPOSER_PREFLIGHT_FIXTURE_EXTENSION'], - ['build', 'PRISMA_COMPOSER_PREFLIGHT_FIXTURE_BUILD'], - ] as const) { - if (saved[key] === undefined) delete process.env[name]; - else process.env[name] = saved[key]; - } - } + expect(spawned).toBe(0); + expect(lowered[0]?.providers).toBeDefined(); + expect(lowered[0]?.state).toBeDefined(); + expect(lowered[0]?.environment).toBe('dev'); + expect(requests.map(({ operation, stage }) => ({ operation, stage }))).toEqual([ + { operation: 'deploy', stage: 'dev' }, + ]); }, 15_000); test('a throw after services start (endpoint merge) is a pipeline failure, and the started services are stopped again', async () => { @@ -1905,7 +1922,7 @@ describe.skipIf(process.platform === 'win32')('dev()', () => { }, { runAssembler: fakeAssembler, - alchemy: async () => ({ exitCode: 0, signal: null }), + converge: appConverge(async () => APPLIED), }, ), ); @@ -1931,10 +1948,10 @@ describe.skipIf(process.platform === 'win32')('dev()', () => { }, { runAssembler: fakeAssembler, - alchemy: async (input) => { + converge: appConverge(async (input) => { stages.push(input.stage); - return { exitCode: 1, signal: null }; - }, + return FAILED; + }), }, ), ); @@ -1966,7 +1983,7 @@ describe.skipIf(process.platform === 'win32')('dev()', () => { fresh: true, cwd: app.dir, }, - { runAssembler: fakeAssembler, alchemy: async () => ({ exitCode: 0, signal: null }) }, + { runAssembler: fakeAssembler, converge: appConverge(async () => APPLIED) }, ), ); @@ -1980,10 +1997,10 @@ describe.skipIf(process.platform === 'win32')('dev()', () => { expect(environments).toEqual(['ensure:Dev']); }, 15_000); - test("--environment runs that environment's copy: its own folder, stack file and Alchemy stage, and --fresh tears down only it", async () => { + test("--environment runs that environment's copy: its own folder and Alchemy stage, and --fresh tears down only it; no stack file is written", async () => { const app = makeAppDir('hello-dev'); const environments: string[] = []; - const runs: { stage: string; stack: string }[] = []; + const runs: { stage: string; cwd: string }[] = []; await silently(() => devWithDeps( @@ -1996,10 +2013,10 @@ describe.skipIf(process.platform === 'win32')('dev()', () => { }, { runAssembler: fakeAssembler, - alchemy: async (input) => { - runs.push({ stage: input.stage, stack: input.stackFileRelativePath }); - return { exitCode: 1, signal: null }; - }, + converge: appConverge(async (request) => { + runs.push({ stage: request.stage, cwd: request.cwd }); + return FAILED; + }), }, ), ); @@ -2012,13 +2029,9 @@ describe.skipIf(process.platform === 'win32')('dev()', () => { `emulators:${devDir}`, `attach:${devDir}`, ]); - expect(runs).toEqual([ - { - stage: 'feature/x', - stack: path.join('.prisma-composer', 'dev', 'feature%2Fx', 'alchemy.run.ts'), - }, - ]); - expect(fs.existsSync(path.join(devDir, 'alchemy.run.ts'))).toBe(true); + expect(runs).toEqual([{ stage: 'feature/x', cwd: app.dir }]); + expect(fs.existsSync(path.join(devDir, 'alchemy.run.ts'))).toBe(false); + expect(fs.readdirSync(devDir)).toEqual(['environment-name']); }, 15_000); test.each([ @@ -2049,10 +2062,10 @@ describe.skipIf(process.platform === 'win32')('dev()', () => { }, { runAssembler: fakeAssembler, - alchemy: async () => { + converge: appConverge(async () => { converged = true; - return { exitCode: 0, signal: null }; - }, + return APPLIED; + }), }, ), ); @@ -2092,7 +2105,7 @@ describe.skipIf(process.platform === 'win32')('dev()', () => { }, { runAssembler: fakeAssembler, - alchemy: async () => ({ exitCode: 1, signal: null }), + converge: appConverge(async () => FAILED), }, ), ); @@ -2103,7 +2116,7 @@ describe.skipIf(process.platform === 'win32')('dev()', () => { expect(stops).toBe(1); }, 15_000); - test('a converge that cannot be spawned still stops the app services', async () => { + test('a converge that throws still stops the app services', async () => { const app = makeAppDir('hello-dev'); let stops = 0; const attachment: LocalTargetAttachment = { @@ -2125,9 +2138,9 @@ describe.skipIf(process.platform === 'win32')('dev()', () => { }, { runAssembler: fakeAssembler, - alchemy: async () => { - throw new Error('spawn failed'); - }, + converge: appConverge(async () => { + throw new Error('outcome.reasons is not iterable'); + }), }, ), ); @@ -2163,7 +2176,7 @@ describe.skipIf(process.platform === 'win32')('dev()', () => { }, { runAssembler: fakeAssembler, - alchemy: async () => ({ exitCode: 0, signal: null }), + converge: appConverge(async () => APPLIED), }, ), ); @@ -2176,9 +2189,9 @@ describe.skipIf(process.platform === 'win32')('dev()', () => { }, 15_000); /** - * The session keeps the config it started with, while each rebuild's child - * would import prisma.config.ts from disk. So an edit is reported, and - * rebuilds stop until the user restarts dev, rather than mixing the two. + * The session keeps the config it started with. So an edit is reported, and + * rebuilds stop until the user restarts dev, rather than converging a + * config that is not the one on disk. */ test('an edit to prisma.config.ts is reported, and rebuilds stop until dev restarts', async () => { const app = makeAppDir('hello-dev'); @@ -2212,10 +2225,10 @@ describe.skipIf(process.platform === 'win32')('dev()', () => { ...(await fakeAssembler(node)), watch: [buildOutput], }), - alchemy: async () => { + converge: appConverge(async () => { converges += 1; - return { exitCode: 0, signal: null }; - }, + return APPLIED; + }), }, ); if (!start.ok) throw new Error('expected a started session'); @@ -2240,8 +2253,9 @@ describe.skipIf(process.platform === 'win32')('dev()', () => { /** * A rebuild re-assembles every service's artifact directory, and the - * converge child reads those directories. A second rebuild starting while - * the first child is still running rewrites them underneath it. + * converge reads those directories. A second rebuild starting while the + * first converge is still running rewrites them underneath it, and one + * process runs one converge at a time. */ test('a build change during a rebuild waits for it, and later changes coalesce into one rebuild', async () => { const app = makeAppDir('hello-dev'); @@ -2274,14 +2288,14 @@ describe.skipIf(process.platform === 'win32')('dev()', () => { ...(await fakeAssembler(node)), watch: [path.join(app.dir, 'built.txt')], }), - alchemy: async () => { + converge: appConverge(async () => { converges += 1; running += 1; mostRunning = Math.max(mostRunning, running); if (converges > 1) await new Promise((resolve) => releases.push(resolve)); running -= 1; - return { exitCode: 0, signal: null }; - }, + return APPLIED; + }), watch: (targets, onChange) => { if (!targets.some((t) => t.paths.includes(path.join(app.dir, 'prisma.config.ts')))) { buildChange = onChange; @@ -2310,6 +2324,282 @@ describe.skipIf(process.platform === 'win32')('dev()', () => { }); }, 15_000); + /** A started dev session whose build watch the test fires by hand. */ + async function devWithBuildWatch( + app: { dir: string; entryPath: string }, + converge: Converge, + events: DevEvent[], + input: { readonly signal?: AbortSignal } = {}, + ) { + let change: () => void = () => undefined; + let stops = 0; + const attachment: LocalTargetAttachment = { + startServices: () => Promise.resolve(), + stopApp: () => { + stops += 1; + return Promise.resolve(); + }, + endpoints: () => Promise.resolve([]), + logs: async function* () {}, + }; + const start = await devWithDeps( + { + config: composerConfig(devConfigWith(attachment)), + entry: app.entryPath, + cwd: app.dir, + onEvent: (event) => void events.push(event), + ...input, + }, + { + runAssembler: fakeAssembler, + converge: appConverge(converge), + watch: (targets, onChange) => { + if (targets.some((target) => target.paths.includes(app.entryPath))) change = onChange; + return { ready: Promise.resolve(), stop: () => undefined }; + }, + }, + ); + return { start, change: () => change(), stops: () => stops }; + } + + const waitUntil = async (done: () => boolean) => { + for (let waited = 0; !done() && waited < 5_000; waited += 10) { + await new Promise((resolve) => setTimeout(resolve, 10)); + } + }; + + test("a rebuild's converge progress reaches the event stream", async () => { + const app = makeAppDir('hello-dev'); + const events: DevEvent[] = []; + let converges = 0; + + await silently(async () => { + const { start, change } = await devWithBuildWatch( + app, + async (request) => { + converges += 1; + request.onEvent?.({ + kind: 'resource', + fqn: 'hello-dev.app', + resourceType: 'Local.Compute', + status: 'pending', + phase: 'pending', + }); + request.onEvent?.({ + kind: 'resource', + fqn: 'hello-dev.app', + resourceType: 'Local.Compute', + status: 'updated', + phase: 'done', + }); + request.onEvent?.({ kind: 'log', severity: 'info', text: 'applied' }); + return APPLIED; + }, + events, + ); + if (!start.ok) throw start.failure; + change(); + await waitUntil( + () => converges === 2 && events.filter((e) => e.kind === 'ready').length === 2, + ); + await start.value.stop(); + }); + + const progress = events.filter( + (event) => event.kind === 'converge-resource' || event.kind === 'converge-message', + ); + expect(progress).toEqual([ + { + kind: 'converge-resource', + fqn: 'hello-dev.app', + resourceType: 'Local.Compute', + status: 'updated', + phase: 'done', + }, + { kind: 'converge-message', severity: 'info', text: 'applied' }, + { + kind: 'converge-resource', + fqn: 'hello-dev.app', + resourceType: 'Local.Compute', + status: 'updated', + phase: 'done', + }, + { kind: 'converge-message', severity: 'info', text: 'applied' }, + ]); + }, 15_000); + + test('a failed rebuild is reported with its cause, keeps the app running, and the next change converges again', async () => { + const app = makeAppDir('hello-dev'); + const events: DevEvent[] = []; + let converges = 0; + + let stopsBeforeStop = -1; + await silently(async () => { + const { start, change, stops } = await devWithBuildWatch( + app, + async () => { + converges += 1; + return converges === 2 ? FAILED : APPLIED; + }, + events, + ); + if (!start.ok) throw start.failure; + change(); + await waitUntil(() => events.some((event) => event.kind === 'converge-failed')); + change(); + await waitUntil(() => events.filter((event) => event.kind === 'ready').length === 2); + stopsBeforeStop = stops(); + await start.value.stop(); + }); + + expect(events.filter((event) => event.kind === 'converge-failed')).toEqual([ + { + kind: 'converge-failed', + message: 'ProviderError: the provider refused', + reasons: [{ tag: 'ProviderError', message: 'the provider refused' }], + }, + ]); + expect(converges).toBe(3); + expect(stopsBeforeStop).toBe(0); + }, 15_000); + + test('stopping the session interrupts the running converge and waits for it before stopping the services', async () => { + const app = makeAppDir('hello-dev'); + const events: DevEvent[] = []; + const order: string[] = []; + let converges = 0; + + await silently(async () => { + const { start, change } = await devWithBuildWatch( + app, + async (request) => { + converges += 1; + if (converges === 1) return APPLIED; + order.push('converging'); + await new Promise((resolve) => + request.signal?.addEventListener('abort', () => resolve(), { once: true }), + ); + await new Promise((resolve) => setTimeout(resolve, 50)); + order.push('lease released'); + return { kind: 'interrupted' }; + }, + events, + ); + if (!start.ok) throw start.failure; + change(); + await waitUntil(() => order.includes('converging')); + await start.value.stop(); + order.push('stopped'); + }); + + expect(order).toEqual(['converging', 'lease released', 'stopped']); + expect(events.some((event) => event.kind === 'converge-failed')).toBe(false); + expect(events.at(-1)).toEqual({ kind: 'stopped' }); + }, 15_000); + + test('Ctrl-C during the first converge is DEV.INTERRUPTED, and the started services are stopped', async () => { + const app = makeAppDir('hello-dev'); + const interrupt = new AbortController(); + let stops = 0; + + const result = await silently(async () => { + const { start, stops: stopCount } = await devWithBuildWatch( + app, + async (request) => { + interrupt.abort(); + return request.signal?.aborted === true ? { kind: 'interrupted' } : APPLIED; + }, + [], + { signal: interrupt.signal }, + ); + stops = stopCount(); + return start; + }); + + if (result.ok) throw new Error('expected an interruption'); + expect(result.failure.code).toBe('DEV.INTERRUPTED'); + expect(stops).toBe(1); + }, 15_000); + + test('a failed first converge carries its reasons, without a stack file or reproduce command', async () => { + const app = makeAppDir('hello-dev'); + + const result = await silently(async () => { + const { start } = await devWithBuildWatch(app, async () => FAILED, []); + return start; + }); + + if (result.ok) throw new Error('expected a failure'); + expect(result.failure.code).toBe('DEV.CONVERGE_FAILED'); + expect(result.failure.message).toBe( + 'The app could not be converged.\nProviderError: the provider refused', + ); + expect(result.failure.meta).toEqual({ + reasons: [{ tag: 'ProviderError', message: 'the provider refused' }], + }); + expect( + fs.existsSync(path.join(app.dir, '.prisma-composer', 'dev', 'dev', 'alchemy.run.ts')), + ).toBe(false); + }, 15_000); + + /** + * The app is imported once, in this process, and Node and Bun cache the + * import. A rebuild therefore cannot see an edit to a file that defines the + * app, so the session says so and pauses instead of converging the old one. + */ + test('an edit to a file that defines the app is reported, and rebuilds stop until dev restarts', async () => { + const app = makeAppDir('hello-dev'); + const events: DevEvent[] = []; + let converges = 0; + + await silently(async () => { + const { start, change } = await devWithBuildWatch( + app, + async () => { + converges += 1; + return APPLIED; + }, + events, + ); + if (!start.ok) throw start.failure; + fs.appendFileSync(app.entryPath, '// a topology edit\n'); + change(); + change(); + await new Promise((resolve) => setTimeout(resolve, 200)); + await start.value.stop(); + }); + + expect(events.filter((event) => event.kind === 'definition-changed')).toEqual([ + { kind: 'definition-changed', files: [app.entryPath] }, + ]); + expect(converges).toBe(1); + }, 15_000); + + test('a save that leaves the definition files unchanged still rebuilds', async () => { + const app = makeAppDir('hello-dev'); + const events: DevEvent[] = []; + let converges = 0; + + await silently(async () => { + const { start, change } = await devWithBuildWatch( + app, + async () => { + converges += 1; + return APPLIED; + }, + events, + ); + if (!start.ok) throw start.failure; + fs.writeFileSync(app.entryPath, fs.readFileSync(app.entryPath)); + change(); + await waitUntil(() => converges === 2); + await start.value.stop(); + }); + + expect(events.some((event) => event.kind === 'definition-changed')).toBe(false); + expect(converges).toBe(2); + }, 15_000); + describe('the prisma.config.ts watch', () => { const quietAttachment: LocalTargetAttachment = { startServices: () => Promise.resolve(), @@ -2345,10 +2635,10 @@ describe.skipIf(process.platform === 'win32')('dev()', () => { ...(await fakeAssembler(node)), watch: [path.join(app.dir, 'built.txt')], }), - alchemy: async () => { + converge: appConverge(async () => { converges += 1; - return { exitCode: 0, signal: null }; - }, + return APPLIED; + }), watch: (targets, onChange, onError) => { watchers.push({ paths: targets.flatMap((t) => t.paths), onChange, onError }); return { ready: Promise.resolve(), stop: () => undefined }; @@ -2444,7 +2734,7 @@ describe.skipIf(process.platform === 'win32')('dev()', () => { }, { runAssembler: fakeAssembler, - alchemy: async () => ({ exitCode: 0, signal: null }), + converge: appConverge(async () => APPLIED), }, ); if (!start.ok) throw new Error('expected a started session'); @@ -2479,7 +2769,7 @@ describe.skipIf(process.platform === 'win32')('dev()', () => { }, { runAssembler: fakeAssembler, - alchemy: async () => ({ exitCode: 0, signal: null }), + converge: appConverge(async () => APPLIED), }, ); if (!start.ok) throw new Error('expected a started session'); @@ -2532,7 +2822,7 @@ describe.skipIf(process.platform === 'win32')('dev()', () => { }, { runAssembler: fakeAssembler, - alchemy: async () => ({ exitCode: 0, signal: null }), + converge: appConverge(async () => APPLIED), watch: () => { watches += 1; throw new Error('a detached session must not watch'); @@ -2571,7 +2861,7 @@ describe.skipIf(process.platform === 'win32')('dev()', () => { }, { runAssembler: fakeAssembler, - alchemy: async () => ({ exitCode: 0, signal: null }), + converge: appConverge(async () => APPLIED), }, ); if (!start.ok) throw new Error('expected a started session'); @@ -2603,17 +2893,17 @@ describe.skipIf(process.platform === 'win32')('dev()', () => { }, { runAssembler: fakeAssembler, - alchemy: async () => { + converge: appConverge(async () => { convergeRan = true; - return { exitCode: 0, signal: null }; - }, + return APPLIED; + }), }, ), ); expect(result.ok).toBe(false); if (result.ok) throw new Error('unreachable'); - expect(result.failure.code).toBe('DEV.STACK_WRITE_FAILED'); + expect(result.failure.code).toBe('DEV.STATE_WRITE_FAILED'); expect(convergeRan).toBe(false); }, 15_000); }); @@ -3337,7 +3627,7 @@ describe.skipIf(process.platform === 'win32')('dev() — dev servers (ADR-0056)' }, }; const events: DevEvent[] = []; - let stackFile = ''; + const lowered: LowerOptions[] = []; await silently(async () => { const start = await devWithDeps( @@ -3350,13 +3640,15 @@ describe.skipIf(process.platform === 'win32')('dev() — dev servers (ADR-0056)' { runAssembler: () => Promise.reject(new Error('a service with a dev server is not assembled')), - alchemy: async () => { - order.push('converge'); - stackFile = fs.readFileSync( - path.join(app.dir, '.prisma-composer', 'dev', 'dev', 'alchemy.run.ts'), - 'utf8', - ); - return { exitCode: 0, signal: null }; + converge: { + ...appConverge(async () => { + order.push('converge'); + return APPLIED; + }), + lower: (root, config, opts) => { + lowered.push(opts); + return lower(root, config, opts); + }, }, }, ); @@ -3375,7 +3667,7 @@ describe.skipIf(process.platform === 'win32')('dev() — dev servers (ADR-0056)' cwd: '/app/web', }); expect(order).toEqual(['converge', 'startServices', 'serve', 'endpoints']); - expect(stackFile).toContain('unassembled: ["hello-dev.app"],'); + expect(lowered.map((opts) => opts.unassembled)).toEqual([['hello-dev.app']]); expect(served).toEqual([ { address: 'hello-dev.app', @@ -3391,7 +3683,7 @@ describe.skipIf(process.platform === 'win32')('dev() — dev servers (ADR-0056)' }); }, 15_000); - test('an edit to the entry module reconverges and serves again; a restart is reported', async () => { + test('a rebuild converges and serves again; a restart is reported', async () => { const app = makeAppDir('hello-dev'); let serves = 0; const attachment: LocalTargetAttachment = { @@ -3406,6 +3698,7 @@ describe.skipIf(process.platform === 'win32')('dev() — dev servers (ADR-0056)' }; const events: DevEvent[] = []; let converges = 0; + let change: () => void = () => undefined; await silently(async () => { const start = await devWithDeps( @@ -3417,15 +3710,19 @@ describe.skipIf(process.platform === 'win32')('dev() — dev servers (ADR-0056)' }, { runAssembler: fakeAssembler, - alchemy: async () => { + converge: appConverge(async () => { converges += 1; - return { exitCode: 0, signal: null }; + return APPLIED; + }), + watch: (targets, onChange) => { + if (targets.some((target) => target.paths.includes(app.entryPath))) change = onChange; + return { ready: Promise.resolve(), stop: () => undefined }; }, }, ); if (!start.ok) throw new Error(`expected a started session: ${start.failure.message}`); - fs.appendFileSync(app.entryPath, '// a topology edit\n'); + change(); await until(() => events.some((event) => event.kind === 'dev-server-restarted')); await start.value.stop(); }); @@ -3455,10 +3752,10 @@ describe.skipIf(process.platform === 'win32')('dev() — dev servers (ADR-0056)' }, { runAssembler: fakeAssembler, - alchemy: async () => { + converge: appConverge(async () => { converges += 1; - return { exitCode: 0, signal: null }; - }, + return APPLIED; + }), }, ), ); @@ -3485,10 +3782,10 @@ describe.skipIf(process.platform === 'win32')('dev() — dev servers (ADR-0056)' }, { runAssembler: fakeAssembler, - alchemy: async () => { + converge: appConverge(async () => { converges += 1; - return { exitCode: 0, signal: null }; - }, + return APPLIED; + }), }, ), ); @@ -3527,7 +3824,7 @@ describe.skipIf(process.platform === 'win32')('dev() — dev servers (ADR-0056)' entry: app.entryPath, cwd: app.dir, }, - { runAssembler: fakeAssembler, alchemy: async () => ({ exitCode: 0, signal: null }) }, + { runAssembler: fakeAssembler, converge: appConverge(async () => APPLIED) }, ), ); @@ -3835,7 +4132,7 @@ describe.skipIf(process.platform === 'win32')( const result = await silently(() => devWithDeps( { config: composerConfig(config), entry: app.entryPath, cwd: app.dir, detach: true }, - { runAssembler: fakeAssembler, alchemy: async () => ({ exitCode: 0, signal: null }) }, + { runAssembler: fakeAssembler, converge: appConverge(async () => APPLIED) }, ), ); @@ -3848,7 +4145,6 @@ describe.skipIf(process.platform === 'win32')( const app = makeAppDir('hello-dev'); const buildOutput = path.join(app.dir, 'built.txt'); fs.writeFileSync(buildOutput, 'v1'); - const stackFile = path.join(app.dir, '.prisma-composer', 'dev', 'dev', 'alchemy.run.ts'); const events: DevEvent[] = []; let converges = 0; let assembles = 0; @@ -3876,14 +4172,13 @@ describe.skipIf(process.platform === 'win32')( assembles += 1; return { ...(await fakeAssembler(node)), watch: [buildOutput] }; }, - alchemy: async () => { + converge: appConverge(async () => { converges += 1; - return { exitCode: 0, signal: null }; - }, + return APPLIED; + }), }, ); if (!start.ok) throw new Error('expected a started session'); - fs.rmSync(stackFile); value = 'v2'; fs.writeFileSync(buildOutput, 'v2'); @@ -3899,7 +4194,6 @@ describe.skipIf(process.platform === 'win32')( }); expect(valuesChanges()).toEqual([{ kind: 'values-changed' }]); - expect(fs.existsSync(stackFile)).toBe(false); expect(converges).toBe(1); }, 20_000); }, diff --git a/packages/0-framework/3-tooling/cli/src/operations/dev.ts b/packages/0-framework/3-tooling/cli/src/operations/dev.ts index d420e2ef8..398fec047 100644 --- a/packages/0-framework/3-tooling/cli/src/operations/dev.ts +++ b/packages/0-framework/3-tooling/cli/src/operations/dev.ts @@ -25,12 +25,29 @@ export type DevEvent = | { readonly kind: 'config-changed'; readonly file: string } /** What the `config` or `secrets` adapters return changed since the session started. The session keeps the values it started with and stops rebuilding until it is restarted. */ | { readonly kind: 'values-changed' } - /** The app keeps running, still watching. */ + /** A file that defines the app (the entry module, or a file a service is declared in) changed. The session keeps the app it loaded at start and stops rebuilding until it is restarted. */ + | { readonly kind: 'definition-changed'; readonly files: readonly string[] } + /** One resource's progress in a converge: the first one, or a rebuild's. */ + | { + readonly kind: 'converge-resource'; + readonly fqn: string; + readonly resourceType: string; + /** Alchemy's own status word, such as `creating` or `updated`. */ + readonly status: string; + readonly phase: 'running' | 'done' | 'failed'; + readonly message?: string | undefined; + } + /** A line Alchemy logged during a converge. */ + | { + readonly kind: 'converge-message'; + readonly severity: 'info' | 'warn' | 'verbose'; + readonly text: string; + } + /** A rebuild's converge failed. The app keeps running, still watching. */ | { readonly kind: 'converge-failed'; - readonly stackFilePath: string; - readonly reproduceCommand: string; - readonly cwd: string; + readonly message: string; + readonly reasons: readonly { readonly tag: string; readonly message: string }[]; } /** Once per service that runs its build adapter's dev server, before the first converge (ADR-0056). `cwd` is absolute. */ | { @@ -68,15 +85,16 @@ export interface DevInput { readonly detach?: boolean | undefined; readonly cwd?: string | undefined; readonly onEvent?: ((event: DevEvent) => void) | undefined; + /** Interrupts a running converge. The first converge then fails with `DEV.INTERRUPTED`; a rebuild's ends quietly. It does not stop the session: `stop()` does. */ + readonly signal?: AbortSignal | undefined; } -/** A running dev session. The operation NEVER touches process signal handlers — - * the host owns signals (and must evict alchemy's import-time SIGINT/SIGTERM - * listeners before installing its own; see run-dev.ts). */ +/** A running dev session. The operation NEVER touches process signal handlers: the host owns signals. */ export interface DevSession { /** The initial front door, already merged across attachments. */ readonly endpoints: readonly ServiceEndpoint[]; /** Stop the watch loop and the app's services (emulators and data stay up). + * A converge still running is interrupted and awaited first, so its state lease is released. * Idempotent; emits 'stopping'/'stopped'; resolves `closed`. A no-op for a detached session. */ stop(): Promise; /** Settles when the session has fully stopped (via stop()); already settled for a detached session. */ diff --git a/packages/0-framework/3-tooling/cli/src/operations/execute-deploy-destroy.ts b/packages/0-framework/3-tooling/cli/src/operations/execute-deploy-destroy.ts index c30a43647..6e44afe1c 100644 --- a/packages/0-framework/3-tooling/cli/src/operations/execute-deploy-destroy.ts +++ b/packages/0-framework/3-tooling/cli/src/operations/execute-deploy-destroy.ts @@ -20,12 +20,7 @@ import { CliStructuredError, InternalError } from '@internal/foundation/errors'; import { notOk, ok, okVoid, type Result } from '@internal/foundation/result'; import { type AppConverge, loadAppConverge } from '../app-converge.ts'; import type { ComposerConfigSource } from '../composer-config.ts'; -import type { - ConvergeEvent, - ConvergeFailureReason, - ConvergeOutcome, - LoweredStack, -} from '../converge-in-process.ts'; +import type { ConvergeEvent, ConvergeOutcome, LoweredStack } from '../converge-in-process.ts'; import { capFailureCause, convergeRedactor, @@ -61,6 +56,7 @@ import { } from './deploy.ts'; import type { DestroyEvent, DestroyInput } from './destroy.ts'; import { + convergeReasonsText, type ExtensionId, type OperationDeps, requireContainerForEnvironment, @@ -863,16 +859,6 @@ function reportConvergeEvent( } } -function reasonsText(reasons: readonly ConvergeFailureReason[]): string { - return reasons - .map(({ tag, message }) => - tag === 'Error' || tag === 'Unknown' || message.startsWith(tag) - ? message - : `${tag}: ${message}`, - ) - .join('\n'); -} - /** The structured failure a converge that did not apply settles as. */ function convergeFailure( outcome: Exclude, @@ -903,7 +889,9 @@ function convergeFailure( }, ); case 'failed': { - const cause = capFailureCause(redact(withComposerAdvice(reasonsText(outcome.reasons)))); + const cause = capFailureCause( + redact(withComposerAdvice(convergeReasonsText(outcome.reasons))), + ); return new CliStructuredError( 'DEPLOY.ENGINE_FAILED', `Alchemy could not deploy the app.\n${cause}`, diff --git a/packages/0-framework/3-tooling/cli/src/operations/execute-dev.ts b/packages/0-framework/3-tooling/cli/src/operations/execute-dev.ts index 91b5eae3e..fa3c6f6dd 100644 --- a/packages/0-framework/3-tooling/cli/src/operations/execute-dev.ts +++ b/packages/0-framework/3-tooling/cli/src/operations/execute-dev.ts @@ -1,9 +1,10 @@ /** - * The dev executor — run-dev.ts's pipeline (local-dev spec § 6) with console - * and signal handling removed: events out through `onEvent`, lifetime owned by - * the returned DevSession. The operation NEVER touches process signal - * handlers — the host does (see run-dev.ts). Reached only by lazy import - * from dev.ts — this module's static graph transitively loads alchemy's + * The dev executor (local-dev spec § 6): events out through `onEvent`, + * lifetime owned by the returned DevSession. Every converge lowers the app + * against the local target and runs Alchemy's plan and apply in this process, + * through the app's own install (`loadAppConverge`, ADR-0064). The operation + * NEVER touches process signal handlers; the host does. Reached only by lazy + * import from dev.ts: this module's static graph transitively loads alchemy's * provider tree, so the control entry must never import it statically. */ import { createHash } from 'node:crypto'; @@ -12,31 +13,27 @@ import * as path from 'node:path'; import { isDeepStrictEqual } from 'node:util'; import type { Graph, ServiceNode } from '@internal/core'; import type { ConfigPools, ContainerInstance, PrismaAppConfig } from '@internal/core/config'; -import { containerEnv, preflightEnv, preflightEnvVarName } from '@internal/core/config'; import type { DevServer } from '@internal/core/deploy'; import type { LocalTargetAttachment, LocalTargetDescriptor } from '@internal/core/local-target'; import { devEnvironmentDir, resolveLocalTargets } from '@internal/core/local-target'; -import { CliStructuredError } from '@internal/foundation/errors'; +import { CliStructuredError, InternalError } from '@internal/foundation/errors'; import { notOk, ok, type Result } from '@internal/foundation/result'; -import { devStackRelativePath, writeDevStackFile } from '../dev/generate-dev-stack.ts'; +import { type AppConverge, loadAppConverge } from '../app-converge.ts'; +import type { ConvergeEvent, ConvergeOutcome } from '../converge-in-process.ts'; +import { convergeRedactor, poolSecrets, withComposerAdvice } from '../deployment-summary.ts'; import { authoringModuleTargets, startWatch, type WatchHandle, watchTargetsFrom, } from '../dev/watch.ts'; -import { type PipelineDeps, runPipeline } from '../pipeline.ts'; -import { - type AlchemyOutcome, - alchemyInvocation, - reproduceCommand, - spawnAlchemy, -} from '../run-alchemy.ts'; +import { type PipelineDeps, type PipelineResult, runPipeline } from '../pipeline.ts'; import { validateEnvironmentName } from '../validate-environment.ts'; import type { DevEvent, DevInput, DevSession } from './dev.ts'; import { claimDevEnvironmentName, requireDevEnvironmentName } from './dev-environment-name.ts'; import { withEmulatorRetry } from './emulator-retry.ts'; import { + convergeReasonsText, DEV_ENVIRONMENT, type ExtensionId, type OperationDeps, @@ -123,7 +120,6 @@ function assertDevServersSupported( } } -/** Runs the full dev pipeline; resolves to a running session or a structured failure. */ /** The file's content hash; undefined when it cannot be read, which counts as a change. */ function contentHash(file: string): string | undefined { try { @@ -139,6 +135,46 @@ function samePools(started: ConfigPools, now: ConfigPools): boolean { ); } +/** What a converge that did not apply failed with, as text; undefined for an interruption. */ +function convergeFailureText( + outcome: Exclude, + redact: (text: string) => string, +): string | undefined { + switch (outcome.kind) { + case 'interrupted': + return undefined; + case 'refused': + return outcome.reason === 'already-running' + ? 'Another converge is already running in this process.' + : 'dev cannot converge in a worker thread when its cwd is not the process working directory.'; + case 'failed': + return redact(withComposerAdvice(convergeReasonsText(outcome.reasons))); + } + return 'The converge ended with an outcome this Composer does not know.'; +} + +function failureReasonsOf( + outcome: ConvergeOutcome, + redact: (text: string) => string, +): readonly { readonly tag: string; readonly message: string }[] { + return outcome.kind === 'failed' + ? outcome.reasons.map(({ tag, message }) => ({ tag, message: redact(message) })) + : []; +} + +function thrownOutcome(error: unknown): ConvergeOutcome { + return { + kind: 'failed', + reasons: [ + { + tag: error instanceof Error ? error.name : 'Unknown', + message: error instanceof Error ? error.message : String(error), + }, + ], + }; +} + +/** Runs the full dev pipeline; resolves to a running session or a structured failure. */ export async function executeDev( input: DevInput, deps: OperationDeps, @@ -159,8 +195,11 @@ export async function executeDev( let pipeline: Awaited>; let resolved: ReadonlyMap; + let appConverge: AppConverge; + // The files that define the app, hashed as they were when it was imported. + let startedDefinitions: ReadonlyMap; const containers = new Map(); - let preflightTransportEnv: Record = {}; + const preflightPayloads = new Map(); // Each participating extension's view of the running app, made before the // first converge so a target that cannot run a dev server fails first. const attachmentsById = new Map(); @@ -196,6 +235,16 @@ export async function executeDev( ); const { graph, name } = pipeline; const config = pipeline.configSource.value; + startedDefinitions = new Map( + authoringModuleTargets(pipeline.entryModule.path, graph) + .flatMap((target) => target.paths) + .map((file) => [file, contentHash(file)]), + ); + + // Lowering and Alchemy run from the app's own install, found before + // anything local is created. + appConverge = + deps.converge ?? (await loadAppConverge(path.dirname(pipeline.configSource.file))); // Dev-capability check — resolve every non-build-only extension's lazy // `localTarget` thunk ONCE (ADR-0041's lazy reference); its pinned error @@ -231,7 +280,6 @@ export async function executeDev( } // Preflight — always (dev has no deploy/destroy split). - const preflightPayloads = new Map(); for (const [id, dev] of resolved) { if (dev.preflight === undefined) continue; try { @@ -247,16 +295,6 @@ export async function executeDev( throw toStructured('DEV.PREFLIGHT_FAILED', error); } } - try { - preflightTransportEnv = preflightEnv(preflightPayloads); - } catch (error) { - throw toStructured('DEV.PREFLIGHT_FAILED', error); - } - // Blank every transport var this run did not fill, so a stale one in the - // shell cannot pass for this run's payload. - for (const extension of config.extensions) { - preflightTransportEnv[preflightEnvVarName(extension.id)] ??= ''; - } // Emulators — ensure the daemons this topology's node kinds need, and their per-node state. for (const [id, dev] of resolved) { @@ -290,49 +328,75 @@ export async function executeDev( throw error; } - const invocation = alchemyInvocation({ - command: 'deploy', - stackFileRelativePath: devStackRelativePath(environment), - cwd, - stage: environment, - containerEnv: containerEnv(containers), - preflightEnv: preflightTransportEnv, - }); + const config = pipeline.configSource.value; + const redact = convergeRedactor( + process.env, + poolSecrets(pipeline.pools, config.secrets !== undefined), + ); + const reportConverge = (event: ConvergeEvent): void => { + if (event.kind === 'log') { + emit({ + kind: 'converge-message', + severity: event.severity, + text: redact(withComposerAdvice(event.text)), + }); + return; + } + if (event.phase === 'pending') return; + emit({ + kind: 'converge-resource', + fqn: event.fqn, + resourceType: event.resourceType, + status: event.status, + phase: event.phase, + ...(event.message === undefined + ? {} + : { message: redact(withComposerAdvice(event.message)) }), + }); + }; - const converge = async (): Promise<{ outcome: AlchemyOutcome; stackPath: string }> => { - let stackPath: string; + /** + * Lowers `current` against the local target and plans and applies it in + * this process. The plan's scope, and the state lease it holds, close + * before this returns. + */ + const converge = async ( + current: PipelineResult, + signal: AbortSignal | undefined, + ): Promise => { try { claimDevEnvironmentName(devDir, environment); - stackPath = writeDevStackFile({ - entryPath: pipeline.entryModule.path, - cwd, - environment, - configFile: pipeline.configSource.file, - name: pipeline.name, - assembled: pipeline.assembled, - }); } catch (error) { - // Stack-file write I/O is structured at the catch that knows (rule 6). - throw toStructured('DEV.STACK_WRITE_FAILED', error); + throw toStructured('DEV.STATE_WRITE_FAILED', error); + } + const root = current.graph.root.node; + if (root.kind !== 'module') { + throw new InternalError('runPipeline admitted a root that is not a module'); } - let outcome: AlchemyOutcome; try { - outcome = await (deps.alchemy ?? spawnAlchemy)(invocation); - } catch (error) { - if (CliStructuredError.is(error)) throw error; - throw new CliStructuredError('DEV.CONVERGE_FAILED', failureMessage(error), { - cause: error, - meta: { - diagnostics: { - exitCode: undefined, - stackFilePath: stackPath, - reproduceCommand: reproduceCommand(invocation), - cwd, - }, - }, + const stack = appConverge.lower(root, config, { + name: current.name, + environment, + configPool: current.pools.config, + bundles: current.assembled.bundles, + unassembled: Object.keys(current.assembled.devServers), + containers, + preflight: preflightPayloads, + providers: appConverge.localTargetProviders(resolved, containers, devDir, current.pools), + state: appConverge.devState(), + }); + return await appConverge.converge({ + operation: 'deploy', + stack, + stage: environment, + cwd, + env: {}, + signal, + onEvent: reportConverge, }); + } catch (error) { + return thrownOutcome(error); } - return { outcome, stackPath }; }; // A failed first converge can leave some services started; the session never began, so stop them. @@ -378,41 +442,32 @@ export async function executeDev( } }; - // Write the dev stack file and converge. Inside the try: a stray - // `.prisma-composer` FILE, a full disk, or a spawn that throws must come - // back as a failure result, not a rejection out of dev(). - let first: { outcome: AlchemyOutcome; stackPath: string }; + // Inside the try: a stray `.prisma-composer` FILE must come back as a + // failure result, not a rejection out of dev(). + let first: ConvergeOutcome; try { - first = await converge(); + first = await converge(pipeline, input.signal); } catch (error) { await stopStartedServices(); if (CliStructuredError.is(error)) return notOk(error); throw error; } - const firstStatus = first.outcome.signal !== null ? null : (first.outcome.exitCode ?? 1); - if (firstStatus !== 0) { + if (first.kind !== 'applied') { await stopStartedServices(); + const text = convergeFailureText(first, redact); return notOk( - new CliStructuredError( - 'DEV.CONVERGE_FAILED', - first.outcome.signal !== null - ? `alchemy deploy was interrupted by ${first.outcome.signal}.` - : `alchemy deploy exited with status ${String(firstStatus)}.`, - { - meta: { - ...(first.outcome.signal !== null - ? { signal: first.outcome.signal } - : { exitCode: firstStatus }), - diagnostics: { - exitCode: firstStatus ?? undefined, - ...(first.outcome.signal !== null ? { signal: first.outcome.signal } : {}), - stackFilePath: first.stackPath, - reproduceCommand: reproduceCommand(invocation, first.outcome), - cwd, + text === undefined + ? new CliStructuredError('DEV.INTERRUPTED', 'dev was interrupted while it converged.', { + why: 'It stopped while Alchemy planned or applied the app. Resources it had already changed stay as they are.', + fix: 'Run dev again to finish.', + }) + : new CliStructuredError( + 'DEV.CONVERGE_FAILED', + `The app could not be converged.\n${text}`, + { + meta: { reasons: failureReasonsOf(first, redact) }, }, - }, - }, - ), + ), ); } @@ -429,6 +484,12 @@ export async function executeDev( let configWatch: WatchHandle | undefined; // Ends the inline following of dev servers' output when the session stops. const following = new AbortController(); + // Interrupts a rebuild's converge when the session stops. + const stopController = new AbortController(); + const convergeSignal = + input.signal === undefined + ? stopController.signal + : AbortSignal.any([input.signal, stopController.signal]); try { for (const attachment of attachments) { // Recorded BEFORE the call: startServices can start some services and @@ -480,13 +541,12 @@ export async function executeDev( // Watch loop until the session is stopped: rebuild → re-assemble → // re-converge; a converge failure keeps the running app and keeps watching. - // Authoring modules are watched too, so an app whose services all run dev - // servers still reconverges on a topology edit (ADR-0056). + // Authoring modules are watched too: the app was imported once, in this + // process, so an edit to one cannot reach the converge, and the session + // reports it and pauses instead (ADR-0064). const bundleWatch = watchTargetsFrom(pipeline.assembled.bundles); - const targets = [ - ...bundleWatch.targets, - ...authoringModuleTargets(pipeline.entryModule.path, pipeline.graph), - ]; + const definitionTargets = authoringModuleTargets(pipeline.entryModule.path, pipeline.graph); + const targets = [...bundleWatch.targets, ...definitionTargets]; for (const address of bundleWatch.unwatchable) { emit({ kind: 'unwatchable', address }); } @@ -501,15 +561,25 @@ export async function executeDev( emit({ kind: 'watch-error', message: failureMessage(error) }); const configFile = pipeline.configSource.file; const startedConfig = contentHash(configFile); + const definitionFiles = [...startedDefinitions.keys()]; let configChanged = false; let valuesChanged = false; + let definitionChanged = false; // Read on every event, not only from the config watcher: a build watch // on a directory that contains the file can fire first. const paused = (): boolean => { - if (valuesChanged) return true; - if (!configChanged && contentHash(configFile) === startedConfig) return false; - configChanged = true; - emit({ kind: 'config-changed', file: configFile }); + if (valuesChanged || definitionChanged) return true; + if (configChanged || contentHash(configFile) !== startedConfig) { + configChanged = true; + emit({ kind: 'config-changed', file: configFile }); + return true; + } + const changed = definitionFiles.filter( + (file) => contentHash(file) !== startedDefinitions.get(file), + ); + if (changed.length === 0) return false; + definitionChanged = true; + emit({ kind: 'definition-changed', files: changed }); return true; }; configWatch = watchFiles( @@ -536,22 +606,16 @@ export async function executeDev( valuesChanged = true; return; } - const stackPath = writeDevStackFile({ - entryPath: rePipeline.entryModule.path, - cwd, - environment, - configFile: rePipeline.configSource.file, - name: rePipeline.name, - assembled: rePipeline.assembled, - }); - const outcome = await (deps.alchemy ?? spawnAlchemy)(invocation); - if (outcome.signal !== null || outcome.exitCode !== 0) { - emit({ - kind: 'converge-failed', - stackFilePath: stackPath, - reproduceCommand: reproduceCommand(invocation, outcome), - cwd, - }); + const outcome = await converge(rePipeline, convergeSignal); + if (outcome.kind !== 'applied') { + const message = convergeFailureText(outcome, redact); + if (message !== undefined) { + emit({ + kind: 'converge-failed', + message, + reasons: failureReasonsOf(outcome, redact), + }); + } return; } // A converge can move what a dev server's environment holds (a @@ -571,27 +635,28 @@ export async function executeDev( } }; // One rebuild at a time: a rebuild rewrites every service's artifact - // directory, which the previous rebuild's converge child may still be - // reading. Changes during a rebuild coalesce into one more after it. - let rebuilding = false; + // directory, which the previous rebuild's converge may still be reading, + // and one process runs one converge at a time. Changes during a rebuild + // coalesce into one more after it. + let stopping = false; + let rebuilding: Promise | undefined; let changedDuringRebuild = false; const rebuildUntilSettled = async (): Promise => { - rebuilding = true; do { changedDuringRebuild = false; await rebuild(); - } while (changedDuringRebuild && !paused()); - rebuilding = false; + } while (changedDuringRebuild && !stopping && !paused()); + rebuilding = undefined; }; watch = watchFiles( targets, () => { - if (paused()) return; - if (rebuilding) { + if (stopping || paused()) return; + if (rebuilding !== undefined) { changedDuringRebuild = true; return; } - void rebuildUntilSettled(); + rebuilding = rebuildUntilSettled(); }, onWatchError, ); @@ -601,7 +666,6 @@ export async function executeDev( const startedConfigWatch = configWatch; await Promise.all([watch.ready, configWatch.ready]); - let stopping = false; let resolveClosed: () => void = () => undefined; const closed = new Promise((resolve) => { resolveClosed = resolve; @@ -611,10 +675,14 @@ export async function executeDev( if (!stopping) { stopping = true; emit({ kind: 'stopping' }); + stopController.abort(); following.abort(); startedWatch.stop(); startedConfigWatch.stop(); void (async () => { + // A converge still running is interrupted above; its scope closes, + // releasing the state lease, before the services stop. + await rebuilding; // A service that refuses to stop is surfaced, not swallowed — // teardown continues, `stopped` still fires, `closed` still settles. for (const attachment of attachments) { diff --git a/packages/0-framework/3-tooling/cli/src/operations/shared.ts b/packages/0-framework/3-tooling/cli/src/operations/shared.ts index 2b9ffbb9b..1ddd4ca61 100644 --- a/packages/0-framework/3-tooling/cli/src/operations/shared.ts +++ b/packages/0-framework/3-tooling/cli/src/operations/shared.ts @@ -13,6 +13,7 @@ import type { import { blindCast } from '@internal/foundation/casts'; import { CliStructuredError } from '@internal/foundation/errors'; import type { AppConverge } from '../app-converge.ts'; +import type { ConvergeFailureReason } from '../converge-in-process.ts'; import type { startWatch } from '../dev/watch.ts'; import type { RunAlchemy } from '../run-alchemy.ts'; @@ -33,19 +34,15 @@ export interface ServiceEndpoint { /** * The operations' in-package injection seam — lets the CLI's own tests drive - * them without a real wrapper build or alchemy process. + * them without a real wrapper build, Alchemy converge or alchemy process. * Threaded through the *WithDeps variants, never part of the published * surface: the fields mirror internal types. */ export interface OperationDeps { readonly runAssembler?: RunAssembler | undefined; - /** - * Starts the converge child for `destroy` and `dev`. The CLI passes one - * backed by the engine's `ctx.spawn`, so the terminal reaches the child - * natively and the engine owns signal policy; hosts get the default `spawnAlchemy`. - */ + /** Starts the converge child for `destroy`; defaults to `spawnAlchemy`. Tests substitute it. */ readonly alchemy?: RunAlchemy | undefined; - /** Lowers and converges `deploy`'s stack; defaults to the app's own install (`loadAppConverge`). Tests substitute it. */ + /** Lowers and converges the stack for `deploy` and `dev`; defaults to the app's own install (`loadAppConverge`). Tests substitute it. */ readonly converge?: AppConverge | undefined; /** Starts a file watch for `dev`; defaults to chokidar-backed `startWatch`. Tests substitute it to drive changes directly. */ readonly watch?: typeof startWatch | undefined; @@ -194,3 +191,14 @@ export function requireContainerForEnvironment( }, ); } + +/** A failed converge's reasons as text, one per line, without a tag the message already starts with. */ +export function convergeReasonsText(reasons: readonly ConvergeFailureReason[]): string { + return reasons + .map(({ tag, message }) => + tag === 'Error' || tag === 'Unknown' || message.startsWith(tag) + ? message + : `${tag}: ${message}`, + ) + .join('\n'); +} diff --git a/packages/0-framework/3-tooling/cli/src/run-alchemy.ts b/packages/0-framework/3-tooling/cli/src/run-alchemy.ts index c8c1a2aed..f19865790 100644 --- a/packages/0-framework/3-tooling/cli/src/run-alchemy.ts +++ b/packages/0-framework/3-tooling/cli/src/run-alchemy.ts @@ -7,11 +7,9 @@ * `node_modules/.bin` link, which pnpm creates only for an app's direct * dependencies. * - * This module composes the invocation; it does not decide how the child is - * started. Under the CLI the engine starts it (`ctx.spawn`), which is what - * makes Ctrl-C reach the child natively and keeps signal policy in one place. - * `spawnAlchemy` is the default for programmatic hosts driving - * `@prisma/composer/control`, which have no engine to borrow. + * Only `destroy` still runs Alchemy in a child; `deploy` and `dev` converge + * in-process (ADR-0064). This module composes the invocation; `spawnAlchemy` + * starts it, and tests substitute their own runner. */ import * as fs from 'node:fs'; import * as path from 'node:path'; @@ -152,8 +150,7 @@ export interface AlchemyOutcome { readonly commandLine?: AlchemyCommandLine; } -/** Starts the converge and resolves when it ends. The CLI supplies one backed - * by `ctx.spawn`; hosts get `spawnAlchemy`. */ +/** Starts the converge and resolves when it ends; `spawnAlchemy` by default. */ export type RunAlchemy = (invocation: AlchemyInvocation) => Promise; export interface AlchemyInvocationInput { @@ -162,7 +159,7 @@ export interface AlchemyInvocationInput { readonly cwd: string; readonly stage: string; readonly containerEnv: Readonly>; - /** What each extension's deploy or dev preflight handed back, serialized — one env var per extension (core's preflight-transport naming). Absent for destroy, which runs no preflight. Content-blind, like `containerEnv`. */ + /** Each extension's preflight transport variable, one per extension (core's preflight-transport naming). Destroy runs no preflight and blanks every one, so a stale value in the shell cannot pass for a payload. Content-blind, like `containerEnv`. */ readonly preflightEnv?: Readonly>; /** Extra additions beyond the containers — the deployment-result pointer. */ readonly env?: Readonly> | undefined; diff --git a/packages/1-prisma-cloud/0-lowering/lowering/src/state/empty-scope.ts b/packages/1-prisma-cloud/0-lowering/lowering/src/state/empty-scope.ts index bf2dc10b5..2a29fb07e 100644 --- a/packages/1-prisma-cloud/0-lowering/lowering/src/state/empty-scope.ts +++ b/packages/1-prisma-cloud/0-lowering/lowering/src/state/empty-scope.ts @@ -80,8 +80,8 @@ const listBranchResources = ( * logicalId, but the rest of the state could not be rebuilt from them * (ADR-0057), so fail once, up front. A genuinely fresh deploy sees an empty * Branch and passes. Connections are not counted: they are children of databases, which - * are. Local dev never reaches this — the dev stack pins - * `state: localState()` (generate-dev-stack.ts). + * are. Local dev never reaches this: it lowers with + * `state: devState()`, Alchemy's local file store (execute-dev.ts). */ export const failOnEmptyScopeWithLiveResources = ( projectId: string, diff --git a/packages/9-public/composer-cli/src/exports/testing.ts b/packages/9-public/composer-cli/src/exports/testing.ts index 79855c068..827d4c8de 100644 --- a/packages/9-public/composer-cli/src/exports/testing.ts +++ b/packages/9-public/composer-cli/src/exports/testing.ts @@ -1,8 +1,6 @@ /** * Public surface (the `./testing` subpath): `createOperationsDouble`, the * fixture-backed double of the operations the command family calls, for hosts that test how they - * mount the command family without running real deploys; and - * `renderDevStackFile`, the dev stack file `dev` writes, for tests that drive - * Alchemy directly. + * mount the command family without running real deploys. */ export * from '@internal/cli/testing'; diff --git a/skills/prisma-composer-core-concepts/SKILL.md b/skills/prisma-composer-core-concepts/SKILL.md index 4d5635743..a8478dc68 100644 --- a/skills/prisma-composer-core-concepts/SKILL.md +++ b/skills/prisma-composer-core-concepts/SKILL.md @@ -507,12 +507,14 @@ environment as production, so deploy and destroy refuse it with **The engine underneath is alchemy.** Convergence is executed by [alchemy](https://alchemy.run), a third-party infrastructure-as-code engine that arrives as an ordinary, exactly-pinned npm dependency of `@prisma/composer` (a preview build of 2.0.0-beta.84, commit `972f294`, at this library version). Your code never imports or configures it; consult alchemy's own docs for the engine itself. What matters operationally: -`prisma deploy` runs alchemy inside the `prisma` process through alchemy's -programmatic API: no generated file and no child process. Its progress prints -as one step per resource, and a failure is `DEPLOY.ENGINE_FAILED` carrying the +`prisma deploy` and `prisma dev` run alchemy inside the `prisma` process through alchemy's +programmatic API: no generated file and no child process. Their progress prints +as one step per resource. A failed deploy is `DEPLOY.ENGINE_FAILED` carrying the engine's own errors (credentials redacted, capped at 1000 characters), or -`DEPLOY.INTERRUPTED` after Ctrl-C, which releases the environment's lease. -For `prisma dev` and `destroy`, Composer runs the alchemy package installed beside the app's +`DEPLOY.INTERRUPTED` after Ctrl-C, which releases the environment's lease. A failed +first `dev` converge is `DEV.CONVERGE_FAILED` with the engine's errors; a failed rebuild +is a warning and the running app is kept. +For `destroy`, Composer runs the alchemy package installed beside the app's `@prisma/composer` and starts it with Node (the first `node` on PATH when `prisma` runs under Bun; `DEPLOY.NODE_MISSING` if none), so the app needs no direct `alchemy` dependency and no `.bin` link. No global Alchemy installation is @@ -520,12 +522,11 @@ needed. 1. The destroy operation writes the pipeline's results to a generated, gitignored stack file at `.prisma-composer/alchemy.run.ts`, then runs the alchemy CLI - against it as a child process; `dev` does the same at - `.prisma-composer/dev//alchemy.run.ts` with local providers. The file + against it as a child process. The file carries the computed values as literals but reads credentials via `fromEnv()`, so nothing sensitive lands on disk, and it is regenerated every run: output, not configuration, never edited. -2. Failures are bisectable through that file. A failing destroy or dev converge names its +2. Failures are bisectable through that file. A failing destroy names its path and prints a reproduce command: Node running the bin of the `alchemy` Composer depends on, with `destroy .prisma-composer/alchemy.run.ts --yes --stage ` (Alchemy's own flag; on Prisma Cloud its value is the @@ -600,9 +601,9 @@ that surprise: environment variables; Composer's values win. `dev` prints ``
: runs `` in `` before the app comes up and streams the dev server's output prefixed `[
]` (other services' logs are not - streamed). Source edits are the dev server's own hot reload; `dev` watches - the entry and every service's authoring module, reconverges on a topology - edit, and restarts a dev server only when its environment variables changed + streamed). Source edits are the dev server's own hot reload; an edit to the + entry or a service's authoring module needs `dev` restarted (it says so), + and `dev` restarts a dev server only when its environment variables changed (``
: its environment changed; `` restarted``). It has no terminal (no colours or key shortcuts). The standalone build is not run locally, so a standalone-only bug surfaces at `prisma deploy`. Next.js @@ -613,7 +614,7 @@ that surprise: in its PORT variable.`; `DEV.DEV_SERVER_UNAVAILABLE` (e.g. `next` not installed in `appDir`); `DEV.DEV_SERVER_UNSUPPORTED` (the service's package is too old; the message names it). - `ready` prints only once every service accepts connections on its port, except a service declared `compute({ ..., listens: false })` (one that never listens, like the cron scheduler), which counts as running once its process starts. A service that exits before it listens, or does not listen within 30 seconds, fails `dev` (and stops the app's other services) with a message naming the service, its port, and its log file: on a first start it is in the deploy engine's output, before the error that the deploy engine exited (`DEV.CONVERGE_FAILED` from the `dev` operation); on a warm start it is the `DEV.SERVICE_START_FAILED` message. `PRISMA_COMPOSER_SERVICE_READY_TIMEOUT_MS` raises the 30-second limit (milliseconds); the compute emulator reads it when it starts, so a running emulator keeps its old value until its process ends. An emulator that does not accept connections fails `dev` and `log` with `DEV.EMULATOR_UNREACHABLE` after a few seconds of retries; nothing else is retried. + `ready` prints only once every service accepts connections on its port, except a service declared `compute({ ..., listens: false })` (one that never listens, like the cron scheduler), which counts as running once its process starts. A service that exits before it listens, or does not listen within 30 seconds, fails `dev` (and stops the app's other services) with a message naming the service, its port, and its log file: on a first start it is in the message of the error that the app could not be converged (`DEV.CONVERGE_FAILED` from the `dev` operation); on a warm start it is the `DEV.SERVICE_START_FAILED` message. `PRISMA_COMPOSER_SERVICE_READY_TIMEOUT_MS` raises the 30-second limit (milliseconds); the compute emulator reads it when it starts, so a running emulator keeps its old value until its process ends. An emulator that does not accept connections fails `dev` and `log` with `DEV.EMULATOR_UNREACHABLE` after a few seconds of retries; nothing else is retried. 2. Ctrl-C (or `dev` ending on SIGTERM or a failure) stops the app's service processes and its local Postgres servers and keeps their data and ports: the next `dev` is a warm start. Starting clean, wiping this app's local instances and data in this environment first, is an explicit opt-in flag (`--fresh`). On a fresh start, each service gets the lowest free port from 3000 up, in dependency order: a service before the services that call it, otherwise provision order. A warm start keeps the ports services already have. Services, databases and buckets belong to the directory `dev` runs in: two checkouts of one app (two git worktrees) run separate processes on separate ports, with separate databases and buckets, and never restart or write into each other's. `.prisma-composer/dev//emulator-instances.json` records which local instance backs each node. A service or database keeps its port in its directory across runs, even while stopped. If a local instance is gone (a Composer upgrade discarded an emulator's state), the next `dev` re-creates it, empty, and asks for the old port, and a Prisma ORM database's (`postgres()`) committed migrations run again (a `rawPostgres()` database stays empty); when another directory has taken the port, the instance gets the lowest free one and `dev` prints `[dev] service "
" now uses port : its previous port is in use` (`database "
"` for a database). `dev` runs as the environment `dev`; `--environment ` runs a complete, separate local copy with its own services, databases, buckets and ports, beside any other. Running a different app or environment stops nothing. For tests that must not touch working data, use `prisma dev --environment test --fresh --detach`, and pass the same `--environment` to `dev url` and `dev stop`. `--fresh` wipes only the app it starts, in its environment. Names that differ only in case share one folder on a case-insensitive filesystem (macOS), so a name that differs from the one the folder records is refused with `DEV.ENVIRONMENT_NAME_MISMATCH`. Everything local to one copy lives in `.prisma-composer/dev//`; the first `dev`, `dev url`, `dev stop` or `log` after upgrading from a version with one copy per directory moves the existing local state into `dev`, keeping its data. An older `@prisma/composer-prisma-cloud` is refused with `DEPS.EXTENSION_INCOMPATIBLE`. A stopped instance keeps its port reserved and a database its data, so a deleted checkout's instances keep theirs; there is no command to remove them yet. 3. `dev` does not print the logs of services that run their artifact. The @@ -654,8 +655,9 @@ that surprise: app, including one an earlier `--detach` left running (a failed redeploy has already replaced that service's process), and exits non-zero: fix the build and run `prisma dev --detach` again. -4. `dev` reads `prisma.config.ts` once, at start, and watches it: after an - edit it says so and pauses rebuilds until you restart `dev`. +4. `dev` reads `prisma.config.ts` and loads the app's definition (the entry + and each service's authoring module) once, at start, and watches them: + after an edit it says so and pauses rebuilds until you restart `dev`. 5. `dev` reads secrets only from the `secrets` adapters. Put them where a `secrets` adapter reads them: the shell by default, or `.env.dev` with `secrets: [dotenv(), environmentVariables()]`. A required secret that is unset or empty stops `dev` with an error naming it; an optional one reaches the service as absent. A changed value takes effect on the next `prisma dev`. A `.env` that only the framework loads (Next.js) does not count: Composer reads secrets before the dev server starts. 6. The emulators are machine-wide daemons shared by every `dev`, registered under `~/.prisma-composer/emulators`. Set `PRISMA_COMPOSER_EMULATORS_DIR` diff --git a/test/integration/test/local-dev-store.integration.ts b/test/integration/test/local-dev-store.integration.ts index ff2b89b06..5ed1de1c3 100644 --- a/test/integration/test/local-dev-store.integration.ts +++ b/test/integration/test/local-dev-store.integration.ts @@ -26,22 +26,17 @@ * after a cold start restarted catalog's RPC consumers too, even though * nothing about them had changed. * - * Attempt 1 drives the CLI's own real chokidar-based watch loop: the + * Both attempts drive the CLI's own real chokidar-based watch loop: the * catalog build adapter declares `Bundle.watch: [runnable.source]` — * the built artifact path itself (build.ts) — so touching * `modules/catalog/dist/server.mjs` is exactly the file session 1's own * running `prisma dev` process is already watching. The script * only touches the file and polls the compute emulator for the new pid * — it never calls assemble or alchemy itself; the running process's - * own debounce, re-assemble, and re-converge are what's under test. - * Attempt 2 is the direct-converge variant kept as a fallback assertion: - * the artifact is touched, the watch loop's own rebuild for that touch - * is awaited (the two would otherwise rewrite and read the same - * artifact directories at once), then the SAME dev stack file is - * re-converged directly with the real `alchemy` binary (matching - * local-dev.integration.ts's own pattern) — this re-hashes catalog's now-different bundle bytes and - * PUTs a fresh deployment for every service, letting the emulator's own - * (untouched) hash/env diffing decide which one(s) actually restart. + * own debounce, re-assemble, and in-process re-converge are what's + * under test. The converge PUTs a fresh deployment for every service, + * letting the emulator's own hash/env diffing decide which one(s) + * actually restart. * Session 1's services are NEVER stopped in between (SIGINT is * app-scoped and unconditionally restarts every service on the next * redeploy — S3's own pinned "a stopped service always starts on @@ -73,8 +68,6 @@ import { type ChildProcess, spawn, spawnSync } from 'node:child_process'; import * as fs from 'node:fs'; import * as net from 'node:net'; import * as path from 'node:path'; -import { containerEnv } from '@prisma/composer/config'; -import { nodeBuild } from '@prisma/composer/node/control'; import { prismaCloud } from '@prisma/composer-prisma-cloud/control'; import { emulatorRegistryRoot } from '@prisma/composer-prisma-cloud/local-target'; import { prismaBinDir } from './spawn-prisma.ts'; @@ -417,35 +410,6 @@ const APP_NAME = 'store'; const catalogDist = path.join(storeDir, 'modules', 'catalog', 'dist', 'server.mjs'); const DEV_STACK_FILE = path.join(storeDir, '.prisma-composer', 'dev', 'dev', 'alchemy.run.ts'); -/** Walks up from `startDir` looking for `node_modules/.bin/alchemy` — same as local-dev.integration.ts's own helper. */ -function alchemyBin(startDir: string): string { - let dir = startDir; - for (;;) { - const candidate = path.join(dir, 'node_modules', '.bin', 'alchemy'); - if (fs.existsSync(candidate)) return candidate; - const parent = path.dirname(dir); - if (parent === dir) - throw new Error(`could not find node_modules/.bin/alchemy above ${startDir}`); - dir = parent; - } -} - -/** - * Touches ONLY catalog's built artifact (append a harmless comment — its - * content hash moves, nothing else's does), re-runs the node build - * adapter's OWN `assemble()` for catalog specifically (it copies the built - * runnable into `.prisma-composer/artifacts/catalog.service/bundle/` — - * touching the source alone does not move anything alchemy actually reads; - * the copy must be refreshed), then re-converges the SAME dev stack file - * session 1's own `prisma dev` already wrote, directly with the - * real `alchemy` binary — no CLI re-invocation, no SIGINT, session 1's - * services stay running throughout. The stack file's `bundles` map needs no - * edit: `assemble()`'s output directory is a deterministic function of the - * address, so it already points at the freshly-copied bytes. Re-hashing - * catalog's now-different bundle and PUTting a fresh deployment for every - * service lets the emulator's own (untouched) hash/env diffing decide which - * one(s) restart. Returns every service's pid AFTER the converge. - */ const REAL_WATCH_TIMEOUT_MS = 30_000; /** @@ -489,62 +453,6 @@ async function touchCatalogAndAwaitRealWatchLoop( }, 15_000); } -async function rebuildCatalogAndReconverge( - session: DevSession, -): Promise> { - // The touch also fires session 1's own watch loop, which re-assembles every - // service's artifact directory. Converging directly while it does races - // that rewrite (seen as "no main.js/main.mjs found in bundle dir - // .../artifacts/storefront", and as an unrelated service restarting), so - // the direct converge starts only after the loop's own front-door reprint. - const beforeLineCount = readLog(session.logPath).split('\n').length; - fs.appendFileSync(catalogDist, `\n// proving-script touch ${Date.now()}\n`); - await waitForAsync(async () => { - const newLines = readLog(session.logPath).split('\n').slice(beforeLineCount); - return parseFrontDoor(newLines.join('\n')); - }, REAL_WATCH_TIMEOUT_MS); - - const catalogServiceModule = path.join(storeDir, 'modules', 'catalog', 'src', 'service.ts'); - const buildDescriptor = nodeBuild().nodes['node']; - assert( - buildDescriptor !== undefined && buildDescriptor.kind === 'build', - 'nodeBuild() must declare a "node" build descriptor', - ); - await buildDescriptor.assemble({ - build: { - extension: '@prisma/composer/node', - type: 'node', - module: `file://${catalogServiceModule}`, - entry: '../dist/server.mjs', - }, - address: 'store.catalog.service', - cwd: storeDir, - }); - - const descriptor = prismaCloud(); - if (descriptor.localTarget === undefined) throw new Error('no localTarget descriptor'); - const dev = await descriptor.localTarget(); - const container = await dev.container.ensure({ appName: APP_NAME, environment: 'dev' }); - const envVars = containerEnv(new Map([[descriptor.id, container]])); - - const result = spawnSync( - alchemyBin(storeDir), - ['deploy', path.relative(storeDir, DEV_STACK_FILE), '--yes', '--stage', 'dev'], - { cwd: storeDir, stdio: 'inherit', env: { ...process.env, ...envVars } }, - ); - assert( - result.status === 0, - `re-converge (alchemy deploy) failed with status ${String(result.status)}`, - ); - - // The pid the emulator reports settles a moment after the converge - // completes (the restarted child still needs to actually start). - return waitForAsync(async () => { - const pids = await pidsByAddress(APP_NAME); - return pids['store.catalog.service'] !== undefined ? pids : undefined; - }, 15_000); -} - async function main(): Promise { console.log('local dev (S5 proving): examples/store via the real prisma dev'); @@ -567,6 +475,10 @@ async function main(): Promise { 15_000, ); assertEqual(health.status, 200, 'the storefront HTTP round-trip'); + assert( + !fs.existsSync(DEV_STACK_FILE), + 'prisma dev converges in process and writes no alchemy.run.ts', + ); console.log('[proving] PASS criterion 1: credential-free bring-up + HTTP round-trip'); const firstPortsByAddress = Object.fromEntries( @@ -595,23 +507,11 @@ async function main(): Promise { assert(activeSession !== undefined, 'session 1 must be running before criterion 2'); let pidsBefore = pidsBaseline; - const RECONVERGE_STRATEGIES = [ - { - label: 'real watch loop', - run: (before: Record) => - touchCatalogAndAwaitRealWatchLoop(activeSession, before), - }, - { - label: 'direct converge (fallback)', - run: () => rebuildCatalogAndReconverge(activeSession), - }, - ] as const; - for (const [index, strategy] of RECONVERGE_STRATEGIES.entries()) { - const attempt = index + 1; - const pidsAfter = await strategy.run(pidsBefore); + for (const attempt of [1, 2]) { + const pidsAfter = await touchCatalogAndAwaitRealWatchLoop(activeSession, pidsBefore); assert( pidsAfter['store.catalog.service'] !== pidsBefore['store.catalog.service'], - `criterion 2 (attempt ${attempt}, ${strategy.label}): catalog.service must restart (new pid)`, + `criterion 2 (attempt ${attempt}, real watch loop): catalog.service must restart (new pid)`, ); for (const address of [ 'store.storefront', @@ -622,11 +522,11 @@ async function main(): Promise { assertEqual( pidsAfter[address], pidsBefore[address], - `criterion 2 (attempt ${attempt}, ${strategy.label}): ${address}'s pid must NOT change — only catalog.service was rebuilt`, + `criterion 2 (attempt ${attempt}, real watch loop): ${address}'s pid must NOT change — only catalog.service was rebuilt`, ); } console.log( - `[proving] PASS criterion 2 (attempt ${attempt}, ${strategy.label}): exactly catalog.service restarted, all four others unchanged`, + `[proving] PASS criterion 2 (attempt ${attempt}, real watch loop): exactly catalog.service restarted, all four others unchanged`, ); pidsBefore = pidsAfter; } diff --git a/test/integration/test/local-dev.integration.ts b/test/integration/test/local-dev.integration.ts index df79938cd..1c6bd22dc 100644 --- a/test/integration/test/local-dev.integration.ts +++ b/test/integration/test/local-dev.integration.ts @@ -1,10 +1,10 @@ /** * The S4 integration proof (local-dev spec, plan.md's S4 outcome; no CLI — * the `prisma dev` command itself is S5's scope): a fixture - * topology (compute × 2, postgres, bucket) lowered with `dev: true` and - * driven through the real `alchemy` binary against the dev stack module - * `dev` itself writes (`renderDevStackFile` from - * `@prisma/composer-cli/testing`). + * topology (compute × 2, postgres, bucket) lowered against the local target + * and converged in this process, the way `prisma dev` converges: `lower`, + * `localTargetProviders`, `devState` and `convergeInProcess` from + * `@prisma/composer/internal/converge` (ADR-0064). * * Only `@prisma/composer`, `@prisma/composer-cli` and `@prisma/composer-prisma-cloud` (9-public) are * imported (ADR-0028: examples/website/test import only the published @@ -27,7 +27,8 @@ * * WHY THIS IS A STANDALONE SCRIPT, NOT A `bun:test` FILE: * - * `bun test` was tried first. Under `bun test` specifically, a daemon + * `bun test` was tried first, when this script still ran the converge in an + * Alchemy child. Under `bun test` specifically, a daemon * spawned deep inside the alchemy child's own Effect runtime (originally * `LocalDatabaseProvider`'s own `prisma dev --name --detach` subprocess, * pre-REVISION — operator review of #162; the same class of issue applies @@ -49,23 +50,30 @@ * timeouts as before, just not started via `bun test`'s own harness. */ -import { spawnSync } from 'node:child_process'; import * as fs from 'node:fs'; import * as net from 'node:net'; import * as path from 'node:path'; import { Load } from '@prisma/composer'; import type { + ConfigPools, ContainerInstance, ExtensionDescriptor, LocalTargetAttachment, } from '@prisma/composer/config'; -import { containerEnv, devEnvironmentDir, preflightEnv } from '@prisma/composer/config'; +import { devEnvironmentDir } from '@prisma/composer/config'; +import { + convergeInProcess, + devState, + localTargetProviders, + lower, +} from '@prisma/composer/internal/converge'; +import { resolveLocalTargets } from '@prisma/composer/local-target'; import { nodeBuild } from '@prisma/composer/node/control'; -import { renderDevStackFile } from '@prisma/composer-cli/testing'; import { prismaCloud } from '@prisma/composer-prisma-cloud/control'; import { emulatorRegistryRoot } from '@prisma/composer-prisma-cloud/local-target'; import bgService from './fixtures/local-dev/bg-service.ts'; import appModule from './fixtures/local-dev/module.ts'; +import prismaConfig from './fixtures/local-dev/prisma.config.ts'; import webService from './fixtures/local-dev/web-service.ts'; // ——— Minimal plain-assert harness (no bun:test / describe / expect). Every @@ -91,35 +99,12 @@ const BKG = `${APP_NAME}.bkg`; const integrationDir = path.resolve(import.meta.dir, '..'); const fixtureDir = path.join(integrationDir, 'test', 'fixtures', 'local-dev'); const devDir = path.join(fixtureDir, devEnvironmentDir('dev')); -const stackDir = devDir; -const stackFile = path.join(stackDir, 'alchemy.run.ts'); -const stackFileRel = path.relative(fixtureDir, stackFile); -const configFile = path.join(fixtureDir, 'prisma.config.ts'); const webEntryFile = path.join(fixtureDir, 'built', 'web-server.mjs'); const logDir = path.join(devDir, 'logs'); -const ALCHEMY_TIMEOUT_MS = 60_000; +const CONVERGE_TIMEOUT_MS = 60_000; let convergeCount = 0; -function resolveBin(name: string): string | undefined { - let dir = integrationDir; - for (;;) { - const candidate = path.join(dir, 'node_modules', '.bin', name); - if (fs.existsSync(candidate)) return candidate; - const parent = path.dirname(dir); - if (parent === dir) return undefined; - dir = parent; - } -} - -function alchemyBin(): string { - const bin = resolveBin('alchemy'); - if (bin === undefined) { - throw new Error(`could not find node_modules/.bin/alchemy above ${integrationDir}`); - } - return bin; -} - /** Last `n` lines of a file, or a note explaining why there's nothing to show — never throws. */ function tailOf(filePath: string, n = 60): string { let content: string; @@ -186,64 +171,52 @@ async function assembleFixture(): Promise> { return { [WEB]: web, [BKG]: bkg }; } -/** The dev stack file `dev` itself writes, from the real generator. */ -function renderStack(bundles: Record): string { - return renderDevStackFile({ - entryPath: path.join(fixtureDir, 'module.ts'), - cwd: fixtureDir, - configFile, - name: APP_NAME, - environment: 'dev', - assembled: { bundles, devServers: {} }, - }); +interface DevConvergeInput { + readonly containers: ReadonlyMap; + readonly preflight: ReadonlyMap; + readonly pools: ConfigPools; } /** - * Runs one `alchemy deploy` against the generated dev stack file, exactly - * as a real dev session would (--stage dev, always — D3), against the - * emulator registry `emulatorRegistryRoot()` names. - * - * stdout/stderr are redirected straight to a log FILE (raw file descriptors, - * not Node's `encoding: 'utf8'` pipe-and-buffer capture) — confirmed live - * that piped capture silently loses output from a deeply nested grandchild - * (alchemy's own foreground-child watchdog, in turn running the stack file, - * in turn ensuring the emulator daemons, including `postgres-main`) - * specifically when the OUTER process is `bun test`; the identical command - * run from a plain shell, or with output going straight to a file, does not - * lose it. Bounded (never a silent indefinite hang — the chief risk is a - * daemon downloading/starting engines on first use with no visible output) - * regardless. + * One converge, exactly as a real dev session runs it: the app lowered + * against the local target's providers and Alchemy's local state store, then + * planned and applied in this process at the stage dev (D3). Alchemy's events + * and log lines go to a log file for the diagnostics dump. Bounded: a converge + * still running after CONVERGE_TIMEOUT_MS is interrupted. */ -function runAlchemyDeploy(containerEnvVars: Readonly>): { - status: number | null; - timedOut: boolean; - stdout: string; - stderr: string; - logFile: string; -} { +async function converge( + bundles: Record, + input: DevConvergeInput, +): Promise<{ ok: boolean; detail: string; logFile: string }> { convergeCount += 1; fs.mkdirSync(logDir, { recursive: true }); const logFile = path.join(logDir, `converge-${convergeCount}.log`); - const logFd = fs.openSync(logFile, 'w'); - let result: ReturnType; - try { - result = spawnSync(alchemyBin(), ['deploy', stackFileRel, '--yes', '--stage', 'dev'], { - cwd: fixtureDir, - stdio: ['ignore', logFd, logFd], - timeout: ALCHEMY_TIMEOUT_MS, - env: { ...process.env, ...containerEnvVars }, - }); - } finally { - fs.closeSync(logFd); - } - const output = fs.readFileSync(logFile, 'utf8'); - return { - status: result.status, - timedOut: result.signal === 'SIGTERM', - stdout: output, - stderr: '', - logFile, - }; + const lines: string[] = []; + const config = prismaConfig.composer; + if (config === undefined) throw new Error('the fixture prisma.config.ts has no composer section'); + const resolved = await resolveLocalTargets(config); + const stack = lower(appModule, config, { + name: APP_NAME, + environment: 'dev', + configPool: input.pools.config, + bundles, + containers: input.containers, + preflight: input.preflight, + providers: localTargetProviders(resolved, input.containers, devDir, input.pools), + state: devState(), + }); + const outcome = await convergeInProcess({ + operation: 'deploy', + stack, + stage: 'dev', + cwd: fixtureDir, + env: {}, + signal: AbortSignal.timeout(CONVERGE_TIMEOUT_MS), + onEvent: (event) => void lines.push(JSON.stringify(event)), + }); + lines.push(JSON.stringify(outcome)); + fs.writeFileSync(logFile, `${lines.join('\n')}\n`); + return { ok: outcome.kind === 'applied', detail: JSON.stringify(outcome), logFile }; } function readJson(file: string): unknown { @@ -383,7 +356,7 @@ function probeTcp(host: string, port: number, timeoutMs: number): Promise { console.log( - 'local dev (S4): compute + postgres + bucket, lowered with dev: true through a real alchemy converge', + 'local dev (S4): compute + postgres + bucket, lowered against the local target and converged in process', ); let descriptor: ExtensionDescriptor | undefined; @@ -479,22 +452,21 @@ async function main(): Promise { assert(readEmulatorEntry('buckets') !== undefined, 'buckets emulator must be registered'); assert(readEmulatorEntry('postgres') !== undefined, 'postgres emulator must be registered'); - // 7. Write the dev stack file and converge through the REAL alchemy binary. - fs.mkdirSync(stackDir, { recursive: true }); - fs.writeFileSync(stackFile, renderStack(bundles)); - // The CLI hands the alchemy process the preflight's payload this way too. - const containerEnvVars = { - ...containerEnv(new Map([[descriptor.id, devContainer]])), - ...preflightEnv( + // 7. Converge in this process, as `prisma dev` does. + const convergeInput: DevConvergeInput = { + containers: new Map([[descriptor.id, devContainer]]), + preflight: preflightPayload === undefined ? new Map() : new Map([[descriptor.id, preflightPayload]]), - ), + pools, }; - const first = runAlchemyDeploy(containerEnvVars); - if (first.status !== 0) { - throw new Error( - `alchemy deploy (first converge) failed (timedOut=${String(first.timedOut)}); see ${first.logFile}:\n${first.stdout}\n${first.stderr}`, - ); + const first = await converge(bundles, convergeInput); + if (!first.ok) { + throw new Error(`first converge failed: ${first.detail}; see ${first.logFile}`); } + assert( + !fs.existsSync(path.join(devDir, 'alchemy.run.ts')), + 'a dev converge writes no alchemy.run.ts', + ); // 8. SERVING: attach and HTTP round-trip the web service. A freshly // deployed child needs a moment to start listening, so both the health @@ -586,11 +558,9 @@ async function main(): Promise { ); // 12. Second converge, unchanged build: full no-op — same pids. - const second = runAlchemyDeploy(containerEnvVars); - if (second.status !== 0) { - throw new Error( - `alchemy deploy (no-op converge) failed (timedOut=${String(second.timedOut)}); see ${second.logFile}:\n${second.stdout}\n${second.stderr}`, - ); + const second = await converge(bundles, convergeInput); + if (!second.ok) { + throw new Error(`no-op converge failed: ${second.detail}; see ${second.logFile}`); } const servicesAfterNoop = await waitFor(async () => { const services = await listComputeServices(); @@ -617,12 +587,9 @@ async function main(): Promise { ); try { const rebundled = await assembleFixture(); - fs.writeFileSync(stackFile, renderStack(rebundled)); - const third = runAlchemyDeploy(containerEnvVars); - if (third.status !== 0) { - throw new Error( - `alchemy deploy (changed-artifact converge) failed (timedOut=${String(third.timedOut)}); see ${third.logFile}:\n${third.stdout}\n${third.stderr}`, - ); + const third = await converge(rebundled, convergeInput); + if (!third.ok) { + throw new Error(`changed-artifact converge failed: ${third.detail}; see ${third.logFile}`); } const servicesAfterChange = await waitFor(async () => { From 721d18ab80b2881f322c1c99fc7675aa6a159eb3 Mon Sep 17 00:00:00 2001 From: willbot Date: Mon, 2 Nov 2026 11:38:50 +0100 Subject: [PATCH 2/2] Address slice 3 review: complete app-file detection, shared converge steps, dev codes - dev records every app file (outside node_modules) the entry import loaded: under Node through the existing resolve hook (observe only), under Bun from the modules require.cache gained, which includes ES modules. A Bun.plugin onResolve sees only dynamic imports, so it is not used. These app definition files, plus each authoring module, are hashed and watched; an edit to any of them is definition-changed. - One step tracker and one converge-event mapper for deploy and dev. Each dev converge is a step (dev.converge, dev.rebuild.) with its resources inside it, and every step it opened is closed when it ends. - A refused dev converge is DEV.CONVERGE_RUNNING or DEV.WORKER_THREAD, on the first converge and in a rebuild's converge-failed event. - DEV.STATE_WRITE_FAILED is DEV.ENVIRONMENT_WRITE_FAILED. - stop() waits only for a running converge; a rebuild still in its pipeline returns before converging. - CONVERGE_CONTRACT stays 1: contract 1 was never published. - The store integration test edits a sub-module's module.ts while dev runs and checks for the restart notice and that nothing converges. - Docs use "app definition files" throughout; running-locally.md no longer overstates or understates what is detected. Co-Authored-By: Claude Opus 5.5 Signed-off-by: willbot Signed-off-by: Will Madden --- docs/design/03-domain-model/glossary.md | 6 +- docs/design/10-domains/local-dev.md | 17 +- ...l-envelopes-with-dotted-namespace-codes.md | 4 +- ...vice-with-its-build-adapters-dev-server.md | 2 +- ...ADR-0064-deploy-runs-alchemy-in-process.md | 14 +- docs/design/90-decisions/README.md | 6 +- docs/guides/deploying.md | 2 +- docs/guides/running-locally.md | 15 +- .../src/__tests__/record-app-files.test.ts | 85 +++++++ .../3-tooling/cli/src/converge-contract.ts | 2 +- .../3-tooling/cli/src/entry-resolution.ts | 56 ++++- .../src/family/__tests__/dev-converge.test.ts | 74 +++---- .../3-tooling/cli/src/family/commands/dev.ts | 54 +++-- .../3-tooling/cli/src/load-entry.ts | 8 +- .../operations/__tests__/operations.test.ts | 204 ++++++++++++++--- .../3-tooling/cli/src/operations/deploy.ts | 18 +- .../3-tooling/cli/src/operations/dev.ts | 40 ++-- .../src/operations/execute-deploy-destroy.ts | 115 ++-------- .../cli/src/operations/execute-dev.ts | 208 ++++++++++-------- .../3-tooling/cli/src/operations/steps.ts | 122 ++++++++++ skills/prisma-composer-core-concepts/SKILL.md | 10 +- .../test/local-dev-store.integration.ts | 74 +++++++ 22 files changed, 792 insertions(+), 344 deletions(-) create mode 100644 packages/0-framework/3-tooling/cli/src/__tests__/record-app-files.test.ts create mode 100644 packages/0-framework/3-tooling/cli/src/operations/steps.ts diff --git a/docs/design/03-domain-model/glossary.md b/docs/design/03-domain-model/glossary.md index 2dc0f2039..6de7fc399 100644 --- a/docs/design/03-domain-model/glossary.md +++ b/docs/design/03-domain-model/glossary.md @@ -421,7 +421,11 @@ to a plan. ### Converge -Making an environment match the lowered stack: Alchemy plans what to create, update, replace and delete, then applies it. `prisma deploy` and `prisma dev` converge in their own process through Alchemy's programmatic API; `destroy` converges in an Alchemy child process. The *app converge* is the app install's `lower` and in-process converge, with the local target's providers and state store for dev, which `deploy` and `dev` load from `@prisma/composer/internal/converge` ([ADR-0064](../90-decisions/ADR-0064-deploy-runs-alchemy-in-process.md)). Codes: `DEV.CONVERGE_FAILED`, `DEV.INTERRUPTED`, `DEPLOY.CONVERGE_RUNNING`, `DEPS.CONVERGE_UNRESOLVABLE`, `DEPS.CONVERGE_CONTRACT_MISMATCH`. +Making an environment match the lowered stack: Alchemy plans what to create, update, replace and delete, then applies it. `prisma deploy` and `prisma dev` converge in their own process through Alchemy's programmatic API; `destroy` converges in an Alchemy child process. The *app converge* is the app install's `lower` and in-process converge, with the local target's providers and state store for dev, which `deploy` and `dev` load from `@prisma/composer/internal/converge` ([ADR-0064](../90-decisions/ADR-0064-deploy-runs-alchemy-in-process.md)). Codes: `DEV.CONVERGE_FAILED`, `DEV.INTERRUPTED`, `DEV.CONVERGE_RUNNING`, `DEV.WORKER_THREAD`, `DEPLOY.CONVERGE_RUNNING`, `DEPS.CONVERGE_UNRESOLVABLE`, `DEPS.CONVERGE_CONTRACT_MISMATCH`. + +### App definition files + +The files that define an app for `prisma dev`: every module outside `node_modules` that importing the entry module loaded, static and dynamic imports alike, plus each service's authoring module. Dev imports them once per process, so an edit to one cannot reach a rebuild; dev reports it (`definition-changed`) and pauses rebuilds until it is restarted ([ADR-0064](../90-decisions/ADR-0064-deploy-runs-alchemy-in-process.md)). ### Control plane / Execution plane diff --git a/docs/design/10-domains/local-dev.md b/docs/design/10-domains/local-dev.md index b3c639664..f29ce8223 100644 --- a/docs/design/10-domains/local-dev.md +++ b/docs/design/10-domains/local-dev.md @@ -579,9 +579,9 @@ Deploy's rule holds: every failure names its fix. | required secret unset or empty | hard error listing every missing variable, the service that needs it, the schema's message, and where an adapter in `secrets` reads it; an optional one is not an error | | env-sourced param unbound | hard error listing the missing names, deploy-preflight style | | an adapter returns different values at a rebuild than at start | that the values changed and rebuilds are paused until dev restarts | -| the entry module or a file a service is declared in changes while dev runs | which files changed, that dev cannot load the app's definition again, and that rebuilds are paused until dev restarts | +| an app definition file (an app file the entry loaded) changes while dev runs | which files changed, that dev cannot load them again, and that rebuilds are paused until dev restarts | | the first converge fails | `DEV.CONVERGE_FAILED`, with Alchemy's errors in the message and in `meta.reasons`; `dev` stops the app's services before it returns the failure | -| a rebuild's converge fails | a warning with Alchemy's errors; the running app is kept and dev keeps watching | +| a rebuild's converge fails or is refused | a warning with the code and Alchemy's errors; the running app is kept and dev keeps watching | | Ctrl-C during the first converge | `DEV.INTERRUPTED`, reported as a message; `prisma dev` exits 130 | | a service exits before it listens, or does not listen within the readiness timeout | the address, the port, the exit code if it exited, and the log path; `dev` stops the app's services before it returns the failure | | an emulator does not accept connections (retried briefly; an error response from the emulator is never retried) | `DEV.EMULATOR_UNREACHABLE`, with the emulator's URL, from `dev` and `log` alike | @@ -634,13 +634,14 @@ The design's founding claim was proven against open-chat — a real, pre-existin loop picks rebuilds up on its own. A build descriptor that still returns no `watch` entries reports `[dev]
has no watchable inputs` at startup and needs its rebuilds triggered manually. -- **An edit to the app's definition needs a restart.** The entry module is +- **An edit to an app definition file needs a restart.** The entry module is imported once per process, and the converge runs in that process, so a - rebuild cannot see an edit to it. Dev hashes the entry module and each - service's declaring file when it starts; when one changes it says so and - pauses rebuilds until `prisma dev` is restarted (ADR-0064). An edit to - another file the entry imports, such as a sub-module's `module.ts`, is not - detected and takes effect at the next start. + rebuild cannot see an edit to it. The app definition files are every app + file (outside `node_modules`) that importing the entry loaded, plus each + service's authoring module. Dev records them while it imports the entry, + hashes and watches them, and when one changes it says so and pauses + rebuilds until `prisma dev` is restarted (ADR-0064). A file already loaded + before the entry, such as one `prisma.config.ts` imports, is not recorded. - **App-owned migrations are not run by `dev`** (by design — ADR-0022, spec § 4's `OrmMigration` line is for framework-run migrations only). An app that runs its own migrations (e.g. via `prisma db init`, like the diff --git a/docs/design/90-decisions/ADR-0044-errors-are-structural-envelopes-with-dotted-namespace-codes.md b/docs/design/90-decisions/ADR-0044-errors-are-structural-envelopes-with-dotted-namespace-codes.md index da7263683..29c5e8b6f 100644 --- a/docs/design/90-decisions/ADR-0044-errors-are-structural-envelopes-with-dotted-namespace-codes.md +++ b/docs/design/90-decisions/ADR-0044-errors-are-structural-envelopes-with-dotted-namespace-codes.md @@ -1,6 +1,6 @@ # ADR-0044: Errors are structural envelopes with dotted namespace codes -Amended by [ADR-0064](ADR-0064-deploy-runs-alchemy-in-process.md): `deploy` and `prisma dev` start no Alchemy child, so the child exit-status passthrough below applies only to `destroy`. A failed first `prisma dev` converge is `DEV.CONVERGE_FAILED` and exits 2; `DEV.INTERRUPTED` and `DEV.STATE_WRITE_FAILED` are added and `DEV.STACK_WRITE_FAILED` is retired. A failed `deploy` plan or apply is `DEPLOY.ENGINE_FAILED` and exits 2. `DEPLOY.INTERRUPTED`, `DEPLOY.SERVICE_TOKEN_REQUIRED`, `DEPLOY.CONVERGE_RUNNING`, `DEPLOY.WORKER_THREAD`, `DEPS.CONVERGE_UNRESOLVABLE` and `DEPS.CONVERGE_CONTRACT_MISMATCH` are added. An interrupted `prisma deploy` reports `DEPLOY.INTERRUPTED` as a message and ends with the engine's `CLI.ABORTED`, exiting 130 (143 for SIGTERM). +Amended by [ADR-0064](ADR-0064-deploy-runs-alchemy-in-process.md): `deploy` and `prisma dev` start no Alchemy child, so the child exit-status passthrough below applies only to `destroy`. A failed first `prisma dev` converge is `DEV.CONVERGE_FAILED` and exits 2; `DEV.INTERRUPTED`, `DEV.CONVERGE_RUNNING`, `DEV.WORKER_THREAD` and `DEV.ENVIRONMENT_WRITE_FAILED` are added and `DEV.STACK_WRITE_FAILED` is retired. A failed `deploy` plan or apply is `DEPLOY.ENGINE_FAILED` and exits 2. `DEPLOY.INTERRUPTED`, `DEPLOY.SERVICE_TOKEN_REQUIRED`, `DEPLOY.CONVERGE_RUNNING`, `DEPLOY.WORKER_THREAD`, `DEPS.CONVERGE_UNRESOLVABLE` and `DEPS.CONVERGE_CONTRACT_MISMATCH` are added. An interrupted `prisma deploy` reports `DEPLOY.INTERRUPTED` as a message and ends with the engine's `CLI.ABORTED`, exiting 130 (143 for SIGTERM). Amended 2026-10-23: the `CONFIG` registry gains ENTRY_MISSING, for a command or operation that has no entry module from the config or an override. @@ -111,7 +111,7 @@ Closed subcode registry: SERVICE_TOKEN_REQUIRED: `deploy`'s credential is not a service token, such as a `prisma login` session, or there is none in the environment ([ADR-0064](ADR-0064-deploy-runs-alchemy-in-process.md)). Temporary: it is retired when `@prisma/cli-engine` gives a command the active access token (the in-process credentials need, `needs: { credentials: 'process' }`, asked of `@prisma/cli-engine`), so a session can deploy. CONVERGE_RUNNING: another in-process converge holds this process's environment and working directory. WORKER_THREAD: `deploy` was called from a worker thread with a `cwd` other than the process's working directory, which a worker cannot change. -- `DEV` — PLATFORM_UNSUPPORTED, TARGET_UNSUPPORTED, CONTAINER_FAILED, TEARDOWN_FAILED, PREFLIGHT_FAILED, EMULATOR_FAILED, SERVICE_START_FAILED, CONVERGE_FAILED, INTERRUPTED, ATTACH_FAILED, STATE_WRITE_FAILED, EMULATORS_DIR_INVALID, EMULATOR_UNREACHABLE, SERVICE_READY_TIMEOUT_INVALID, NOT_STARTED_HERE, ADDRESS_UNKNOWN, NOT_RUNNING. EMULATOR_UNREACHABLE is raised by the emulator client when it cannot connect, so `log` raises it under `DEV` too; for a connection failure it takes the place of SERVICE_START_FAILED, ATTACH_FAILED and `LOG.ATTACH_FAILED`. `devUrl` also raises it when the emulator a node runs on is not running. NOT_STARTED_HERE (no `prisma dev` has recorded an app in the directory) is raised by `devUrl` and `devStop`; ADDRESS_UNKNOWN (the recorded app has no node at that address; the known addresses are listed) and NOT_RUNNING (the node exists but is not running; its state is named, and the reason and fix depend on it) by `devUrl` alone ([ADR-0053](ADR-0053-dev-url-dev-stop-and-emulator-list.md)). CONVERGE_FAILED: the first converge failed; Alchemy's errors are in the message and in `meta.reasons`. INTERRUPTED: the abort signal stopped the first converge; `prisma dev` reports it as a message and exits 130. STATE_WRITE_FAILED: the environment's folder under `.prisma-composer/dev/` could not be written *(STACK_WRITE_FAILED retired by [ADR-0064](ADR-0064-deploy-runs-alchemy-in-process.md))*. Both operations raise EMULATOR_FAILED when the local target fails without a code. `emulator list` fails only with EMULATORS_DIR_INVALID; an emulator it cannot list is reported in that emulator's entry with EMULATOR_UNREACHABLE or EMULATOR_FAILED. +- `DEV` — PLATFORM_UNSUPPORTED, TARGET_UNSUPPORTED, CONTAINER_FAILED, TEARDOWN_FAILED, PREFLIGHT_FAILED, EMULATOR_FAILED, SERVICE_START_FAILED, CONVERGE_FAILED, INTERRUPTED, CONVERGE_RUNNING, WORKER_THREAD, ATTACH_FAILED, ENVIRONMENT_WRITE_FAILED, EMULATORS_DIR_INVALID, EMULATOR_UNREACHABLE, SERVICE_READY_TIMEOUT_INVALID, NOT_STARTED_HERE, ADDRESS_UNKNOWN, NOT_RUNNING. EMULATOR_UNREACHABLE is raised by the emulator client when it cannot connect, so `log` raises it under `DEV` too; for a connection failure it takes the place of SERVICE_START_FAILED, ATTACH_FAILED and `LOG.ATTACH_FAILED`. `devUrl` also raises it when the emulator a node runs on is not running. NOT_STARTED_HERE (no `prisma dev` has recorded an app in the directory) is raised by `devUrl` and `devStop`; ADDRESS_UNKNOWN (the recorded app has no node at that address; the known addresses are listed) and NOT_RUNNING (the node exists but is not running; its state is named, and the reason and fix depend on it) by `devUrl` alone ([ADR-0053](ADR-0053-dev-url-dev-stop-and-emulator-list.md)). CONVERGE_FAILED: the first converge failed; Alchemy's errors are in the message and in `meta.reasons`. INTERRUPTED: the abort signal stopped the first converge; `prisma dev` reports it as a message and exits 130. CONVERGE_RUNNING and WORKER_THREAD: the converge was refused, as `deploy`'s `DEPLOY.CONVERGE_RUNNING` and `DEPLOY.WORKER_THREAD`; a refused rebuild carries the code in its `converge-failed` event. ENVIRONMENT_WRITE_FAILED: the environment's folder under `.prisma-composer/dev/` could not be written *(STACK_WRITE_FAILED retired by [ADR-0064](ADR-0064-deploy-runs-alchemy-in-process.md))*. Both operations raise EMULATOR_FAILED when the local target fails without a code. `emulator list` fails only with EMULATORS_DIR_INVALID; an emulator it cannot list is reported in that emulator's entry with EMULATOR_UNREACHABLE or EMULATOR_FAILED. - `LOG` — PLATFORM_UNSUPPORTED, ATTACH_FAILED, ADDRESS_UNKNOWN - `DEPS` — EXECUTOR_UNLOADABLE, EXTENSION_INCOMPATIBLE, CONVERGE_UNRESOLVABLE, CONVERGE_CONTRACT_MISMATCH *(EFFECT_VERSION_CONFLICT retired by [ADR-0049](ADR-0049-composers-configuration-is-the-composer-section-of-prisma-config.md))*. EXTENSION_INCOMPATIBLE: an installed extension is older than the Composer diff --git a/docs/design/90-decisions/ADR-0056-prisma-dev-runs-a-service-with-its-build-adapters-dev-server.md b/docs/design/90-decisions/ADR-0056-prisma-dev-runs-a-service-with-its-build-adapters-dev-server.md index c15d893e9..73fdafc36 100644 --- a/docs/design/90-decisions/ADR-0056-prisma-dev-runs-a-service-with-its-build-adapters-dev-server.md +++ b/docs/design/90-decisions/ADR-0056-prisma-dev-runs-a-service-with-its-build-adapters-dev-server.md @@ -1,6 +1,6 @@ # ADR-0056: `prisma dev` runs a service with its build adapter's dev server -Amended by [ADR-0064](ADR-0064-deploy-runs-alchemy-in-process.md) (2026-11-02): `prisma dev` converges in its own process and writes no dev stack file; `unassembled` is passed to `lower()` as a value. An edit to an authoring module no longer reconverges: the app is imported once, so dev reports the edit and pauses rebuilds until it is restarted. Edits to built output, and the serve step after every successful reconverge, are unchanged. +Amended by [ADR-0064](ADR-0064-deploy-runs-alchemy-in-process.md) (2026-11-02): `prisma dev` converges in its own process and writes no dev stack file; `unassembled` is passed to `lower()` as a value. An edit to an authoring module, or to any other app definition file (an app file the entry loaded), no longer reconverges: the app is imported once, so dev reports the edit and pauses rebuilds until it is restarted. Edits to built output, and the serve step after every successful reconverge, are unchanged. Amended 2026-10-23: `prisma dev` takes no positional entry; the entry is `entry` in the `composer` section of `prisma.config.ts`, or `--entry ` ([ADR-0050](ADR-0050-composer-runs-as-prisma-deploy-and-prisma-dev.md)). diff --git a/docs/design/90-decisions/ADR-0064-deploy-runs-alchemy-in-process.md b/docs/design/90-decisions/ADR-0064-deploy-runs-alchemy-in-process.md index 751ec7e69..c70ee3cdd 100644 --- a/docs/design/90-decisions/ADR-0064-deploy-runs-alchemy-in-process.md +++ b/docs/design/90-decisions/ADR-0064-deploy-runs-alchemy-in-process.md @@ -43,14 +43,14 @@ const stack = lower(root, config, { await convergeInProcess({ operation: 'deploy', stack, stage: environment, cwd, env: {}, signal, onEvent }); ``` -- **The same converge, with the local target.** Dev lowers against Composer's local target providers and Alchemy's local file store, not Alchemy's `dev: true` mode ([ADR-0041](ADR-0041-local-dev-runs-the-deploy-pipeline-against-local-providers.md)). Both functions come from the app's install, with `lower` and `convergeInProcess`, so the local providers share the app's copy of `alchemy` and `effect`. The entry now exports them, so `CONVERGE_CONTRACT` is 2. The converge sets no environment variables: dev needs no credential, and the local providers call no platform. +- **The same converge, with the local target.** Dev lowers against Composer's local target providers and Alchemy's local file store, not Alchemy's `dev: true` mode ([ADR-0041](ADR-0041-local-dev-runs-the-deploy-pipeline-against-local-providers.md)). Both functions come from the app's install, with `lower` and `convergeInProcess`, so the local providers share the app's copy of `alchemy` and `effect`. The entry exports them too; the contract covers them. The converge sets no environment variables: dev needs no credential, and the local providers call no platform. - **One converge at a time, each in its own scope.** Each converge closes its scope before it returns, so the state lease is held only while it runs. Rebuilds stay one at a time, with changes during a rebuild coalesced into one more, which also keeps dev within the one-converge-per-process rule above. -- **Progress through the dev event stream.** The operation emits `converge-resource` and `converge-message` events. `prisma dev` renders each resource as a step (`dev.converge.
`) and each log line as a message, and sends `console`'s stdout methods to stderr for the whole session. -- **Failures are structured.** A failed first converge is `DEV.CONVERGE_FAILED`, with the reasons in `meta.reasons`; an interrupted one is `DEV.INTERRUPTED`. A failed rebuild emits `converge-failed` with the cause and keeps the running app. Neither carries a stack file path or a reproduce command. The ADR-0044 child exit-status passthrough no longer applies to `prisma dev`: a failed first converge exits 2. -- **Ctrl-C.** The `dev` operation takes a `signal`. Ctrl-C during the first converge interrupts it; `prisma dev` reports `DEV.INTERRUPTED` as a message and exits 130. During a rebuild, stopping the session interrupts the converge and waits for it to end, and so for its lease to be released, before it stops the services. Ctrl-C while idle stops dev as before. -- **An edit to the app's definition asks for a restart.** Before, a rebuild re-ran the pipeline in the `prisma` process, which re-assembled every artifact but re-imported the entry from Node's or Bun's module cache, so its graph never changed. The Alchemy child then imported `prisma.config.ts` and the app afresh, so an edit to the topology took effect through the child, lowered against artifacts assembled from the old graph. In process there is only the graph imported at start, and neither runtime can drop a module from its cache. So dev hashes the files it already watches as authoring modules (the entry module and each service's `build.module` file) when it starts. When one of them changes, the session emits `definition-changed`, which `prisma dev` shows as a warning asking for a restart, and pauses rebuilds, as it does for an edit to `prisma.config.ts` (`config-changed`) or a change in what the adapters return (`values-changed`). An edit to an artifact takes effect as before. An edit to another file the entry imports (a sub-module's `module.ts`) is not detected; before, it took effect at the next rebuild, and now it takes effect at the next start. This amends [ADR-0056](ADR-0056-prisma-dev-runs-a-service-with-its-build-adapters-dev-server.md), where a topology edit reconverged. +- **Progress through the dev event stream.** Dev and deploy map Alchemy's events to steps through one shared mapper. Each dev converge is a step with its own id, `dev.converge` for the first and `dev.rebuild.` for the n-th rebuild, and each resource it touches is a step inside it; Alchemy's log lines are messages. When a converge ends, every step it opened is finished, as `failed` when the converge failed or was interrupted. `prisma dev` renders the steps and sends `console`'s stdout methods to stderr for the whole session. +- **Failures are structured.** A failed first converge is `DEV.CONVERGE_FAILED`, with the reasons in `meta.reasons`; an interrupted one is `DEV.INTERRUPTED`; a refused one is `DEV.CONVERGE_RUNNING` (another converge holds this process, such as a `deploy` called by the same host) or `DEV.WORKER_THREAD`. A failed or refused rebuild emits `converge-failed` with that code and the cause, and keeps the running app. Neither carries a stack file path or a reproduce command. The ADR-0044 child exit-status passthrough no longer applies to `prisma dev`: a failed first converge exits 2. +- **Ctrl-C.** The `dev` operation takes a `signal`. Ctrl-C during the first converge interrupts it; `prisma dev` reports `DEV.INTERRUPTED` as a message and exits 130. During a rebuild, stopping the session interrupts the converge and waits for it to end, and so for its lease to be released, before it stops the services. A rebuild still in its pipeline (loading, adapters, assembly) is not waited for: it returns before it would converge. Ctrl-C while idle stops dev as before. +- **An edit to an app definition file asks for a restart.** Before, a rebuild re-ran the pipeline in the `prisma` process, which re-assembled every artifact but re-imported the entry from Node's or Bun's module cache, so its graph never changed. The Alchemy child then imported `prisma.config.ts` and the app afresh, so an edit to the topology took effect through the child, lowered against artifacts assembled from the old graph. In process there is only the graph imported at start, and neither runtime can drop a module from its cache. So dev records the *app definition files*: every module outside `node_modules` that importing the entry loaded for the first time, static and dynamic imports alike, plus each service's authoring module. Under Node the existing resolve hook records each resolved `file:` URL and changes nothing; under Bun they are the modules `require.cache` gained during the import, which includes ES modules (a `Bun.plugin` `onResolve` sees only dynamic imports, so it cannot record them). Dev hashes and watches those files. When one changes, the session emits `definition-changed`, which `prisma dev` shows as a warning asking for a restart, and pauses rebuilds, as it does for an edit to `prisma.config.ts` (`config-changed`) or a change in what the adapters return (`values-changed`). An edit to an artifact takes effect as before. A module that was already loaded before the entry, such as one `prisma.config.ts` imports, is not recorded, and neither is what only it imports. This amends [ADR-0056](ADR-0056-prisma-dev-runs-a-service-with-its-build-adapters-dev-server.md), where a topology edit reconverged. -Rejected for the stale graph: importing the entry again under a new query string (it reloads only the entry, not the modules it imports, and keeps every old copy in memory); a resolve hook that tags the app's files with a generation (Node only, since Bun has no such hook, and it also keeps every old copy); and a child process or worker per converge (what this decision removes). +Rejected for the stale graph: importing the entry again under a new query string (it reloads only the entry, not the modules it imports); a resolve hook that tags the app's files with a generation so each rebuild imports them afresh (Node only: Bun has no such hook, so the two runtimes would behave differently; the memory it keeps is bounded by the app's own files per edit); and a child process or worker per converge (what this decision removes, and a graph holds functions and layers that cannot cross a worker boundary). ## Reasoning @@ -61,7 +61,7 @@ The bisection argument weakens with the API. A failed deploy now reports Alchemy ## Consequences - `deploy` and `prisma dev` no longer resolve Alchemy's bin or need Node on PATH under Bun. `DEPLOY.ALCHEMY_BIN_MISSING`, `DEPLOY.NODE_MISSING` and `DEPLOY.STACK_WRITE_FAILED` remain for `destroy`. -- New codes: `DEPLOY.INTERRUPTED`, `DEPLOY.SERVICE_TOKEN_REQUIRED` (temporary), `DEPLOY.CONVERGE_RUNNING`, `DEPLOY.WORKER_THREAD`, `DEPS.CONVERGE_UNRESOLVABLE` and `DEPS.CONVERGE_CONTRACT_MISMATCH`; for dev, `DEV.INTERRUPTED`, and `DEV.STATE_WRITE_FAILED` (the environment's folder could not be written) in place of `DEV.STACK_WRITE_FAILED`. +- New codes: `DEPLOY.INTERRUPTED`, `DEPLOY.SERVICE_TOKEN_REQUIRED` (temporary), `DEPLOY.CONVERGE_RUNNING`, `DEPLOY.WORKER_THREAD`, `DEPS.CONVERGE_UNRESOLVABLE` and `DEPS.CONVERGE_CONTRACT_MISMATCH`; for dev, `DEV.INTERRUPTED`, `DEV.CONVERGE_RUNNING`, `DEV.WORKER_THREAD`, and `DEV.ENVIRONMENT_WRITE_FAILED` (the environment's folder could not be written) in place of `DEV.STACK_WRITE_FAILED`. - `prisma dev` no longer declares `maySpawn`. `generate-dev-stack.ts`, the dev command's child spawn and settlement (`family/dev-converge-child.ts`), and `renderDevStackFile` on `@prisma/composer-cli/testing` are deleted. - The converge relies on two Alchemy modules outside `alchemy/Alchemist`: `alchemy/Util/LogRecord` (to read log records) and `alchemy/Interaction` (to send interaction output to stderr). - `prisma deploy` declares `needs: { credentials: true }` and no longer `maySpawn`. diff --git a/docs/design/90-decisions/README.md b/docs/design/90-decisions/README.md index 685fa3d64..44204dd9b 100644 --- a/docs/design/90-decisions/README.md +++ b/docs/design/90-decisions/README.md @@ -65,7 +65,7 @@ _Earlier drafts (ADR-0001, ADR-0002) were retired as the high-level design settl - [ADR-0041](ADR-0041-local-dev-runs-the-deploy-pipeline-against-local-providers.md) — `prisma-composer dev` runs the **same deploy pipeline** (Load → assemble → lower → Alchemy converge) against local implementations of the same Alchemy resource types, declared on an optional `localTarget` field of `ExtensionDescriptor` (a lazy thunk resolving a `LocalTargetDescriptor`; subpaths `@prisma/composer/local-target` and `@prisma/composer-prisma-cloud/local-target` — "dev" names only the user-facing command/prefix/state dir) (providers, container, preflight, emulators, attach, teardown — **no** `nodes`/`provisions`, so the lowering cannot diverge; no `state` either — dev uses Alchemy's own `localState()` through `LowerOptions.state`). The target runs **emulators per node kind**: Compute and buckets are machine-global, multi-tenant daemons (the Compute emulator owns the service child processes — deployment PUTs, crash supervision, logs; buckets serve the S3 wire over plain files on disk), while Postgres the Postgres emulator hosts one `@prisma/dev` server per database instance. Providers provision instances by communicating with the emulators during converge, and the dev command is a view through `attach`; `ServiceKey`/`S3Credentials`/`PgWarm`/`OrmMigration` are shared verbatim. Credential-free by requirement. Rejects a local Management API (reimplements another team's server-side semantics, drifts silently) and per-kind dev descriptors (an open-set parallel seam). *(Amended by ADR-0050: the command is `prisma dev`; logs are the `log` operation, not a command. Amended 2026-10-23: the entry is the `composer` section's `entry` or `--entry`, not a positional. Amended by ADR-0052: emulator instances are keyed by emulator-generated ids, not the app name, Composer maps each app's nodes to them per directory, and `prisma dev` stops the app's database servers when it ends.)* *(Amended by ADR-0056: a service runs the dev server its build adapter declares in place of its artifact, as a command deployment on the Compute emulator.)* *(Amended by ADR-0062: local dev runs as a named environment, `dev` by default.)* - [ADR-0042](ADR-0042-service-input-is-one-standard-schema.md) — A compute service declares its entire incoming configuration — config and secrets together — as one Standard Schema (`input`), read back through one typed accessor; `params`/`secrets` and `config()`/`secrets()` are replaced. The framework never introspects the schema (validate-only, per the spec): the operator's binding is the traversable structure (sourcing: literals, `envParam`, `envSecret`), the schema is the black-box judge of legality (invoked at deploy over the resolved binding with secrets as opaque `SecretString` boxes, and again at boot), and secretness is a leaf *type* enforced by validation in both directions. The wire format is one self-describing JSON document row per service with `$secret` pointers to platform variables; an env-bound key whose variable is unset resolves to key-omitted and the schema arbitrates absence — subsuming optional secrets and conditional config (`stripeId` only when `stripeEnabled`) without a framework DSL. *(Extended by ADR-0061 to modules.)* *(Amended 2026-10-30: the forwardable secret need is removed; a module takes a secret as an input field.)* *(Amended 2026-10-30, sixth slice: `field()` and `secretString()` ship from `@prisma/composer/arktype` and `@prisma/composer/zod`; `secretString()` accepts a secret pointer.)* *(Amended 2026-10-30, seventh slice: the deploy checks a service's schema with pointers in place; a field bound to `serviceOrigin()` or a plain `generatedParam()` is wrapped with `field()`; `service.input()` has no pointers.)* - [ADR-0043](ADR-0043-the-control-subpath-is-the-programmatic-deploy-surface.md) — `@prisma/composer/control` is the programmatic deploy surface: typed `deploy`/`destroy`/`dev`/`log` operations (structured inputs/results, no argv/console/exit) implemented in `@internal/cli`'s `src/operations/` and re-exported per ADR-0035; the CLI is a thin renderer over them. The entry's static graph stays import-light — each operation lazily imports its executor, so importing the subpath executes nothing, and a tree that cannot load the deploy stack surfaces as a structured `pipeline` failure — and `PRISMA_COMPOSER_DEPLOYMENT_RESULT_FILE` carries the deploy result across the process boundary: the alchemy child's report hook writes a serializable `DeploymentSummary` to the named file, the operation reads it back best-effort (absent/malformed = undefined summary, never a failure). Distinct from an extension's ADR-0017 `/control` entry. *(Amended by ADR-0049: each operation takes a required `config: { value, file }`, and the effect-conflict diagnosis is retired.)* *(Amended by ADR-0050: the `prisma` CLI renders two of the four operations, as `prisma deploy` and `prisma dev`; `destroy` and `log` are called from scripts.)* *(Amended by ADR-0053: `devUrl` and `devStop` join the operations.)* *(Amended by ADR-0059: `deploy` and `destroy` take `environment`, which `destroy` requires.)* *(Amended 2026-10-28: `devUrl` and `devStop` act on the app and environment in their input; `dev` and `log` take `environment`.)* *(Amended by ADR-0064: `deploy` returns the summary as a value; it uses no result file.)* -- [ADR-0044](ADR-0044-errors-are-structural-envelopes-with-dotted-namespace-codes.md) — Errors are structural envelopes with dotted `NAMESPACE.SUBCODE` codes (the shared prisma/prisma foundation, duplicated pending extraction): structured at origin with why/fix splits, no catch-all codes, bugs carry no code (exit 1 + report hint), recognition is structural (`CliStructuredError.is()`), operation results ride the shared `Result` `ok` discriminator, expected failures exit 2 — with the alchemy child-status passthrough as the documented exception. *(Amended by ADR-0049: `DEPS.EFFECT_VERSION_CONFLICT` is retired. Amended 2026-10-23: `CONFIG.ENTRY_MISSING` is added. Amended by ADR-0059: the `DEPLOY` codes say environment instead of stage, and `DEPS.EXTENSION_INCOMPATIBLE` is added.)* *(Amended 2026-10-30: `CONFIG.ADAPTER_FAILED` and `CONFIG.ADAPTER_INVALID` are added.)* *(Amended by ADR-0064: the child exit-status passthrough no longer applies to `deploy`, whose failures exit 2; `DEPLOY.INTERRUPTED`, `DEPLOY.SERVICE_TOKEN_REQUIRED` (temporary), `DEPLOY.CONVERGE_RUNNING`, `DEPLOY.WORKER_THREAD`, `DEPS.CONVERGE_UNRESOLVABLE` and `DEPS.CONVERGE_CONTRACT_MISMATCH` are added; an interrupted `prisma deploy` ends as `CLI.ABORTED` with exit 130. For `prisma dev` too the passthrough no longer applies; `DEV.INTERRUPTED` and `DEV.STATE_WRITE_FAILED` are added and `DEV.STACK_WRITE_FAILED` is retired.)* +- [ADR-0044](ADR-0044-errors-are-structural-envelopes-with-dotted-namespace-codes.md) — Errors are structural envelopes with dotted `NAMESPACE.SUBCODE` codes (the shared prisma/prisma foundation, duplicated pending extraction): structured at origin with why/fix splits, no catch-all codes, bugs carry no code (exit 1 + report hint), recognition is structural (`CliStructuredError.is()`), operation results ride the shared `Result` `ok` discriminator, expected failures exit 2 — with the alchemy child-status passthrough as the documented exception. *(Amended by ADR-0049: `DEPS.EFFECT_VERSION_CONFLICT` is retired. Amended 2026-10-23: `CONFIG.ENTRY_MISSING` is added. Amended by ADR-0059: the `DEPLOY` codes say environment instead of stage, and `DEPS.EXTENSION_INCOMPATIBLE` is added.)* *(Amended 2026-10-30: `CONFIG.ADAPTER_FAILED` and `CONFIG.ADAPTER_INVALID` are added.)* *(Amended by ADR-0064: the child exit-status passthrough no longer applies to `deploy`, whose failures exit 2; `DEPLOY.INTERRUPTED`, `DEPLOY.SERVICE_TOKEN_REQUIRED` (temporary), `DEPLOY.CONVERGE_RUNNING`, `DEPLOY.WORKER_THREAD`, `DEPS.CONVERGE_UNRESOLVABLE` and `DEPS.CONVERGE_CONTRACT_MISMATCH` are added; an interrupted `prisma deploy` ends as `CLI.ABORTED` with exit 130. For `prisma dev` too the passthrough no longer applies; `DEV.INTERRUPTED`, `DEV.CONVERGE_RUNNING`, `DEV.WORKER_THREAD` and `DEV.ENVIRONMENT_WRITE_FAILED` are added and `DEV.STACK_WRITE_FAILED` is retired.)* - [ADR-0045](ADR-0045-deploy-state-lives-behind-the-platform-state-api.md) — Deploy state lives behind the platform state API (the Management API implements Alchemy's stock `HttpStateApi` wire contract per Branch; composer's state layer is Alchemy's stock HTTP client), and deploys hold a server-side per-`(stack, stage)` lease (TTL 60s, heartbeated, released on exit; contention fails fast naming the holder; state operations without a live lease fail 409). Supersedes ADR-0010 (lock → lease) and the storage half of ADR-0034 (Branch scoping and lifetime stand; the visible per-stage database is gone); closes ADR-0012 as obsolete. No migration: legacy stages are refused until destroyed or deleted. - [ADR-0046](ADR-0046-the-orm-facade-is-a-peer-dependency.md) — `@prisma/composer-prisma-cloud` takes the Prisma ORM postgres facade (`@prisma/orm-postgres`) as a **peer** dependency at one exact version, not a regular dependency: Composer registers an extension pack against the application's copy of the target, and two copies of a shell in one tree means two codec/operation registries and two class identities — a value from one is rejected by the other, silently. As a peer, that combination fails at install instead. Every `@prisma/orm-*` spec in the workspace is one exact version and all name the same one (`scripts/lint-orm-pins.mjs`). `@prisma/orm-toolchain`, which Composer drives rather than extends, stays a regular dependency. Replaces ADR-0022's consequence bullet on how the ORM is installed. - [ADR-0047](ADR-0047-compute-assembly-preserves-safe-runtime-topology.md) — Compute assembly traces runtime files from the author-declared Node entry without rebundling app code, preserves only symlinks whose resolved targets remain inside the staged bundle, and installs narrowly runtime-gated bootstrap compatibility when a framework needs Node semantics. Each local mechanism is removed once the upstream Alchemy Compute provider owns the equivalent guarantee. Supersedes ADR-0005's blanket ban on symlinks and its assumption that directory output is already self-contained. @@ -77,7 +77,7 @@ _Earlier drafts (ADR-0001, ADR-0002) were retired as the high-level design settl - [ADR-0053](ADR-0053-dev-url-dev-stop-and-emulator-list.md) — Composer's command family ships `dev url
` (where a node of the app `prisma dev` runs from this directory answers: a service's URL, a database's connection string, a bucket's S3 endpoint; the URL alone on stdout, piped too), `dev stop` (stops that app's services and database servers, data kept), `dev --detach` (returns once every service answers and leaves the app running until `dev stop`; the foreground stays the default) and `emulator list` (every local emulator and its instances, with labels; needs no project). `devUrl` and `devStop` are `@prisma/composer/control` operations that go through a new optional `current({ devDir })` on the local-target seam, so core never reads the target's record; `listEmulators` lives in the Prisma Cloud domain and `@prisma/composer-cli` adds its command. New codes: `DEV.NOT_STARTED_HERE`, `DEV.ADDRESS_UNKNOWN`, `DEV.NOT_RUNNING`. Amends ADR-0050 and ADR-0043. *(Amended by ADR-0056: `devUrl`'s result gains `runs`. Amended by ADR-0057: `dev url` takes the full address. Amended 2026-10-27: the family also ships `deploy list`, which lists every node an app has deployed on a stage with its platform ID and URL; `@prisma/composer-cli` adds it. Amended by ADR-0059: `deploy list` takes `--environment`. Amended 2026-10-28: `dev url` and `dev stop` take `--environment`, default `dev`, and never act on the app last run; `dev url` takes the app from the address, `dev stop` from the entry or `--name`; `emulator list` shows each instance's environment.)* - [ADR-0054](ADR-0054-raw-http-is-the-untyped-tier-of-service-connections.md) — `@prisma/composer/http` ships `rawHttp()` and `rawHttpContract` (kind `http`, compared by kind only): the untyped tier of service-to-service connections, as `rawPostgres()` is for data. The provider declares its HTTP port; there is no service key, retry or idempotency, because the far end is the app's own server. The auth module's `api` port becomes a raw HTTP port and the provider-named `auth-api` kind is removed, because contracts describe interfaces, not providers. A `Contract` declares its own communication `style`, which the application topology reads. The untyped `http({ name })` is removed. A more specific HTTP kind filling a raw slot is the intended design but is not built, because contract kinds have no parent kinds. - [ADR-0055](ADR-0055-the-postgres-binding-builds-clients-with-the-apps-options.md) — The `postgres(contract)` binding gains `clientWith(options)`, which builds a typed Prisma ORM client with the runtime's own options (less `poolOptions`), such as pgvector's runtime extension or `middleware`, and `connection`, the pool every client shares, for an app that builds its client itself. The dependency declaration takes no options: they are how the service consumes the database, not what it requires, and passing the runtime's option type through means a new option needs no Composer change. `client` is unchanged. Composer never discovers an extension package from the contract's pack ids; a missing one still fails when the client is built, with the runtime's error. Rejected: options on the declaration, extensions on the contract. Amends ADR-0022, extends ADR-0040. -- [ADR-0056](ADR-0056-prisma-dev-runs-a-service-with-its-build-adapters-dev-server.md) — Under `prisma dev`, a service runs the dev server its build adapter declares, in place of its built artifact: `nextjs()` declares `next dev`; `node()` declares none unless the author writes `dev: [...]` on it. No flag. The build descriptor gains `serve(input) → DevServer | undefined` (`cwd`, `argv({ port, host })`, `display`), called before assembly; such a service is not assembled and lowers without its `Deployment` (`LowerOptions.unassembled` + the service descriptor's `reserve`), so its rows, instance, port and dependents' URLs stay. The Compute emulator runs the dev server as a *command deployment* (`{ command: { cwd, argv }, env, listens }`) on the service's instance, with the deployment's materialized env re-stashed address-free by `stashRows`, the pure function `run()` now uses, plus `PORT`/`HOST`, over the shell's env; readiness, logs, crash supervision, restart on env change, `--detach`, `dev stop` and session resume are unchanged and the dev session owns no processes. The session streams dev servers' logs inline; `prisma dev` also watches the entry and every service's authoring module; `devUrl` gains `runs`. New codes `DEV.DEV_SERVER_UNAVAILABLE`, `DEV.DEV_SERVER_UNSUPPORTED`. Amends ADR-0041 and ADR-0053; extends ADR-0005. *(Amended by ADR-0064: an edit to an authoring module is reported and pauses rebuilds until dev restarts, instead of reconverging.)* +- [ADR-0056](ADR-0056-prisma-dev-runs-a-service-with-its-build-adapters-dev-server.md) — Under `prisma dev`, a service runs the dev server its build adapter declares, in place of its built artifact: `nextjs()` declares `next dev`; `node()` declares none unless the author writes `dev: [...]` on it. No flag. The build descriptor gains `serve(input) → DevServer | undefined` (`cwd`, `argv({ port, host })`, `display`), called before assembly; such a service is not assembled and lowers without its `Deployment` (`LowerOptions.unassembled` + the service descriptor's `reserve`), so its rows, instance, port and dependents' URLs stay. The Compute emulator runs the dev server as a *command deployment* (`{ command: { cwd, argv }, env, listens }`) on the service's instance, with the deployment's materialized env re-stashed address-free by `stashRows`, the pure function `run()` now uses, plus `PORT`/`HOST`, over the shell's env; readiness, logs, crash supervision, restart on env change, `--detach`, `dev stop` and session resume are unchanged and the dev session owns no processes. The session streams dev servers' logs inline; `prisma dev` also watches the entry and every service's authoring module; `devUrl` gains `runs`. New codes `DEV.DEV_SERVER_UNAVAILABLE`, `DEV.DEV_SERVER_UNSUPPORTED`. Amends ADR-0041 and ADR-0053; extends ADR-0005. *(Amended by ADR-0064: an edit to an app definition file, authoring modules included, is reported and pauses rebuilds until dev restarts, instead of reconverging.)* - [ADR-0057](ADR-0057-a-nodes-address-is-its-full-path-from-the-app-name.md) — A node's address is the path of provision IDs starting at the root, whose address is the deployed app name (letters, digits, `-` and `_`; `shop`, `shop.catalog`, `shop.auth.api`), and it is the node's only identity. It is the `logicalId` of the node's platform row and topology node; Alchemy resource IDs are `
` for the main resource and `
.` for other pieces, which cannot collide because a module has children and no resources and a service or resource has resources and no children; `COMPOSER_` env keys are built from the full address with `-` turned into `_`; `serviceOrigin()` takes the full address. Every platform name Composer writes is the address, a label; the upgrade recreates Composer's Connections and bucket access keys with new credentials. Existing stages migrate on their next deploy: every resource declares `renamedFrom` its previous ID, and generated values, service keys and stream keys are renamed, not recreated; local dev's record is rewritten on first run. An older Composer deploying a migrated stage is not protected against. Rejected: unqualified child addresses, a second shorter "path" form, `-` suffixes on resource IDs, relative `serviceOrigin()`. Supersedes ADR-0051, carrying forward its identity rules. - [ADR-0058](ADR-0058-prisma-dev-reads-secrets-from-the-environment-and-never-invents-them.md) — Under `prisma dev` a secret or env-sourced param comes only from the run's environment variables: the shell, or a `.env` that `prisma.config.ts` loads (lead with `import 'dotenv/config'`; `process.loadEnvFile` fails under Bun). A required secret that is unset or empty stops dev with an error naming the variable, the service and both fixes; an optional one reaches the service as absent. Required or optional is decided by validating the input with the secret left out, never by reading the schema (ADR-0042). The dev preflight sends the alchemy process the absent names and an HMAC fingerprint of each pointed value on the existing preflight transport; the serializer leaves out only the names it lists, and the fingerprint fills the `:updatedAt` trigger member, so a changed value redeploys the services that read it. Each run takes values from its environment variables and never reuses one from an earlier run; a leftover `secrets.json` is deleted. Deploy is unchanged, so dev and deploy write different input documents when an optional secret is unset until deploy gets the same fix. A `.env` loaded by `prisma.config.ts` is also read by `prisma deploy`, which uploads missing secrets from it. Amends ADR-0041's secrets bullet. *(Amended 2026-10-30: dev reads the secrets and config pools; `dotenv()` is the documented loader.)* - [ADR-0059](ADR-0059-a-deployment-environment-is-called-an-environment.md) — A place the app runs is an **environment**: a target-neutral concept of Composer's core, used on every public surface (`--environment`, `destroy`'s target, guides, errors, skill), replacing "stage". Every environment has a name and none is special: `prisma deploy` defaults to `production`, `prisma dev` to `dev`. On Prisma Cloud, `production` is the Project's default Branch and any other environment is the Branch of that `gitName`; `dev` has no Branch. "Branch" is Prisma Cloud's term and its git-shaped workflow, and appears only where Composer reports what the target did. @@ -85,4 +85,4 @@ _Earlier drafts (ADR-0001, ADR-0002) were retired as the high-level design settl - [ADR-0061](ADR-0061-a-module-declares-its-input-as-a-compute-service-does.md) — A module declares its configuration as one Standard Schema, `input`, exactly as a compute service does; its body receives opaque per-key references it can only pass into children's inputs or derive from with `resolve()`, so the graph cannot branch on configuration. Module `params`/`paramNeed()` and `secrets`/`secret()` slots are removed. The root may declare `input`, filled by config adapters. *(Amended 2026-10-29: an input field is a `Value`; the module's schema validates its binding during Load's second pass; a truthiness check on a field is not caught, but always goes the same way.)* *(Amended 2026-10-29, third slice: `paramSource()` and `secretSource()` return values and take an options argument.)* *(Amended 2026-10-30: a root's input is the config pool.)* *(Amended 2026-10-30, fifth slice: the slots are removed; `ModuleNode`.)* *(Amended 2026-10-30, seventh slice: a module's schema checks each runtime value as its pointer; `paramSource()` loses `sentinel`.)* *(Amended 2026-10-30, eighth slice: only a plain field can be a `resolve()` dependency.)* - [ADR-0062](ADR-0062-environment-values-come-from-resolve-and-config-adapters.md) — The graph's structure is the same whatever inputs Composer is given; only values differ per environment. A module body holds values (`Value`), opaque stand-ins for data known later: deploy-time values, whose data the deploy has by the end of Load, and runtime values (`RuntimeValue`, which is `Value>`), whose data only the platform holds. `resolve(dependencies, fn)` computes a value from deploy-time values only, such as `environment.name` or a plain input field; `fn` receives their real values as one tuple after the graph is built, returns plain data, and never sees a pointer. Config adapters in the `composer` section's `config` list produce the configuration, which is the root's input and what `envParam()` reads; secrets adapters in its `secrets` list produce the secrets, which fill missing `envSecret()` platform variables. Both lists default to `[environmentVariables()]`; `dotenv()` reads `.env` files through `dotenv-flow`. Secrets stay out of deploy files, state and output through pointers and redaction. Every environment has a name and none is special: `prisma deploy` defaults `--environment` to `production`, `prisma dev` to `dev`; on Prisma Cloud `production` is the Project's default Branch. *(Amended by ADR-0064: `deploy` and `prisma dev` run Alchemy in their first process and write no configuration file; the second process applies to `destroy` only.)* - [ADR-0063](ADR-0063-a-node-depends-on-an-interface-or-exposes-one.md) — The two ends of a connection are named by what the node does with them: a node **depends on** an interface (`deps`) or **exposes** one (`expose`); an interface is a `Contract` in code; Input and Output are retired as port words, because the dataflow and ports-and-adapters traditions give the same two ends opposite names; the module body's `inputs` becomes `deps`; the topology's `in`/`out` stay, defined as the dependency end and the exposed end. -- [ADR-0064](ADR-0064-deploy-runs-alchemy-in-process.md) — `prisma deploy` and `prisma dev` (and their operations) run Alchemy's plan and apply in the `prisma` process through `alchemy/Alchemist`: no generated stack file, no Alchemy child. `lower()` takes containers, preflight payloads and the report hook as values, the summary comes back as a value, Alchemy's events become engine step events, failures are `DEPLOY.ENGINE_FAILED` or `DEPLOY.INTERRUPTED`, and Ctrl-C releases the state lease. The credential reaches Alchemy through `process.env`; a `prisma login` session needs `PRISMA_SERVICE_TOKEN` (`DEPLOY.SERVICE_TOKEN_REQUIRED`). Lowering and Alchemy load from the app's own install of `@prisma/composer`, which must speak the host's converge contract. Dev converges once at start and once per rebuild against the local target, reports progress through its own events, and asks for a restart after an edit to the app's definition, because the app is imported once per process (`definition-changed`). Supersedes ADR-0007 for `deploy` and `prisma dev`; amends ADR-0038, ADR-0043, ADR-0044, ADR-0049, ADR-0056 and ADR-0062. +- [ADR-0064](ADR-0064-deploy-runs-alchemy-in-process.md) — `prisma deploy` and `prisma dev` (and their operations) run Alchemy's plan and apply in the `prisma` process through `alchemy/Alchemist`: no generated stack file, no Alchemy child. `lower()` takes containers, preflight payloads and the report hook as values, the summary comes back as a value, Alchemy's events become engine step events, failures are `DEPLOY.ENGINE_FAILED` or `DEPLOY.INTERRUPTED`, and Ctrl-C releases the state lease. The credential reaches Alchemy through `process.env`; a `prisma login` session needs `PRISMA_SERVICE_TOKEN` (`DEPLOY.SERVICE_TOKEN_REQUIRED`). Lowering and Alchemy load from the app's own install of `@prisma/composer`, which must speak the host's converge contract. Dev converges once at start and once per rebuild against the local target, reports progress through its own events, and asks for a restart after an edit to an app definition file (any app file the entry loaded), because the app is imported once per process (`definition-changed`). Supersedes ADR-0007 for `deploy` and `prisma dev`; amends ADR-0038, ADR-0043, ADR-0044, ADR-0049, ADR-0056 and ADR-0062. diff --git a/docs/guides/deploying.md b/docs/guides/deploying.md index a4dfcbcda..6015312ba 100644 --- a/docs/guides/deploying.md +++ b/docs/guides/deploying.md @@ -745,7 +745,7 @@ What to know before embedding it: - **`deploy` runs the engine in your process.** It returns `summary` as a value and reports the engine's progress through `onEvent`; it prints nothing. While it plans and applies it sets the service token and workspace id in `process.env` and changes to `cwd` when your process is elsewhere, then restores both, so a process runs one deploy at a time (`DEPLOY.CONVERGE_RUNNING`), and from a worker thread `cwd` must be the process's working directory (`DEPLOY.WORKER_THREAD`). Deploy uses the lowering and the converge from the app's own install of `@prisma/composer`, found beside its `prisma.config.ts`, which must speak the same converge contract as the Composer running the deploy (`DEPS.CONVERGE_CONTRACT_MISMATCH` otherwise); the package versions may differ. Pass `signal` to interrupt it. `destroy` still runs the engine as a child process whose output streams to your process's stdio. - **`dev` resolves to `{ ok: true, value: session }` or a failure** — never an exit code. The session is `{ endpoints, stop(), closed }`, with - progress (`ready`, `converge-resource`, `converge-failed`, `config-changed`, `definition-changed`, `watch-error`, …) delivered through + progress (`ready`, `step-started`, `step-finished`, `converge-failed`, `config-changed`, `definition-changed`, `watch-error`, …) delivered through `onEvent`. The operation never installs signal handlers; wiring Ctrl-C to `session.stop()` is yours, and passing `signal` interrupts a running converge. Like `deploy`, `dev` runs the engine in your process, one converge at a time. - **`log` resolves to `{ ok: true, value: { appName, services, lines } }` or diff --git a/docs/guides/running-locally.md b/docs/guides/running-locally.md index c93c8f09f..ea90c6b28 100644 --- a/docs/guides/running-locally.md +++ b/docs/guides/running-locally.md @@ -66,11 +66,12 @@ and watches the file. When the file changes, `dev` prints that it changed and must be restarted, and it stops rebuilding: until you stop `dev` and start it again, a build change only repeats that notice. -`dev` also loads your app's definition once: the entry module and the -files your services are declared in, such as `module.ts` and each -`service.ts`. When one of them changes, `dev` prints which file changed and -that it must be restarted, and it stops rebuilding in the same way. An edit -to another file your entry imports takes effect the next time `dev` starts. +`dev` also loads your app definition files once: your entry module and +every file of yours it imports, directly or through other files, such as +each module's `module.ts`, each `service.ts` and their helpers (files in +`node_modules` are not included). When one of them changes, `dev` prints +which file changed and that it must be restarted, and it stops rebuilding in +the same way. `Ctrl-C` stops your app's service processes and its local Postgres servers, and exits; so does `dev` ending on SIGTERM or a failure. The data stays, and @@ -154,8 +155,8 @@ What else changes for such a service: adds `A dev server must listen on the port in its PORT variable.` A dev server that crashes later is restarted with backoff, as an artifact is. - **Edits to your source are the dev server's business.** `dev` does not - rebuild for them. A change to `module.ts` or a `service.ts` needs `dev` - restarted, as above. When a rebuild of another service changes what a dev + rebuild for them. A change to an app definition file, such as `module.ts` + or a `service.ts`, needs `dev` restarted, as above. When a rebuild of another service changes what a dev server's environment holds, such as a dependency's URL, the dev server is restarted: `` shop.web: its environment changed; `next dev` restarted ``. - **It has no terminal.** Its output goes through the service's log, so a dev diff --git a/packages/0-framework/3-tooling/cli/src/__tests__/record-app-files.test.ts b/packages/0-framework/3-tooling/cli/src/__tests__/record-app-files.test.ts new file mode 100644 index 000000000..2744f6891 --- /dev/null +++ b/packages/0-framework/3-tooling/cli/src/__tests__/record-app-files.test.ts @@ -0,0 +1,85 @@ +/** Which app files an entry import loaded, under Bun (this runner) and under Node (a spawned process). */ +import { afterEach, describe, expect, test } from 'bun:test'; +import { spawnSync } from 'node:child_process'; +import * as fs from 'node:fs'; +import * as os from 'node:os'; +import * as path from 'node:path'; +import { pathToFileURL } from 'node:url'; +import { recordAppFiles } from '../entry-resolution.ts'; + +const dirs: string[] = []; +afterEach(() => { + for (const dir of dirs.splice(0)) fs.rmSync(dir, { recursive: true, force: true }); +}); + +/** An app whose entry imports a sibling statically, a sub-module through a `tsconfig` path, a file dynamically, and a package in `node_modules`. */ +function appDir(paths: boolean): { dir: string; entry: string; expected: string[] } { + const dir = fs.realpathSync(fs.mkdtempSync(path.join(os.tmpdir(), 'composer-record-files-'))); + dirs.push(dir); + const write = (file: string, text: string) => { + fs.mkdirSync(path.dirname(path.join(dir, file)), { recursive: true }); + fs.writeFileSync(path.join(dir, file), text); + }; + write('package.json', JSON.stringify({ name: 'app', type: 'module' })); + write( + 'tsconfig.json', + JSON.stringify({ compilerOptions: { baseUrl: '.', paths: { '@sub/*': ['modules/*'] } } }), + ); + write( + 'node_modules/dep/package.json', + JSON.stringify({ name: 'dep', type: 'module', exports: './index.js' }), + ); + write('node_modules/dep/index.js', 'export const dep = 1;\n'); + write('service.ts', "export const service = 'service';\n"); + write('modules/catalog/module.ts', "export const catalog = 'catalog';\n"); + write('lazy.ts', "export const lazy = 'lazy';\n"); + write( + 'entry.ts', + [ + "import { service } from './service.ts';", + paths + ? "import { catalog } from '@sub/catalog/module.ts';" + : "import { catalog } from './modules/catalog/module.ts';", + "import { dep } from 'dep';", + "const { lazy } = await import('./lazy.ts');", + 'export default [service, catalog, dep, lazy];', + '', + ].join('\n'), + ); + return { + dir, + entry: path.join(dir, 'entry.ts'), + expected: ['entry.ts', 'service.ts', 'modules/catalog/module.ts', 'lazy.ts'] + .map((file) => path.join(dir, file)) + .sort(), + }; +} + +describe('recordAppFiles()', () => { + test('under Bun: static, tsconfig-path and dynamic imports, and nothing in node_modules', async () => { + const app = appDir(true); + + const { value, files } = await recordAppFiles(() => import(pathToFileURL(app.entry).href)); + + expect((value as { default: unknown[] }).default).toEqual(['service', 'catalog', 1, 'lazy']); + expect([...files].sort()).toEqual(app.expected); + }); + + test('under Node: static and dynamic imports, and nothing in node_modules', () => { + const app = appDir(false); + const script = path.join(app.dir, 'record.mjs'); + fs.writeFileSync( + script, + [ + `import { recordAppFiles } from ${JSON.stringify(pathToFileURL(path.join(import.meta.dir, '..', 'entry-resolution.ts')).href)};`, + `const { files } = await recordAppFiles(() => import(${JSON.stringify(pathToFileURL(app.entry).href)}));`, + 'console.log(JSON.stringify(files));', + ].join('\n'), + ); + + const result = spawnSync('node', [script], { encoding: 'utf8' }); + + expect(result.stderr).not.toContain('Error'); + expect(JSON.parse(result.stdout).sort()).toEqual(app.expected); + }); +}); diff --git a/packages/0-framework/3-tooling/cli/src/converge-contract.ts b/packages/0-framework/3-tooling/cli/src/converge-contract.ts index 0425e1a4a..76767bc7b 100644 --- a/packages/0-framework/3-tooling/cli/src/converge-contract.ts +++ b/packages/0-framework/3-tooling/cli/src/converge-contract.ts @@ -7,4 +7,4 @@ * when one of them changes in a way the other side cannot read. Imports * nothing, so a host can read it before loading the converge. */ -export const CONVERGE_CONTRACT = 2; +export const CONVERGE_CONTRACT = 1; diff --git a/packages/0-framework/3-tooling/cli/src/entry-resolution.ts b/packages/0-framework/3-tooling/cli/src/entry-resolution.ts index 71159601f..1e3ec9863 100644 --- a/packages/0-framework/3-tooling/cli/src/entry-resolution.ts +++ b/packages/0-framework/3-tooling/cli/src/entry-resolution.ts @@ -15,13 +15,61 @@ * * Bun resolves `./x.js` and `./x` to `x.ts` natively, so the hook is a * no-op under Bun. Nothing is registered there. + * + * The hook also observes, without changing anything: `recordAppFiles` uses + * it to learn which app files an entry import loaded. */ import { existsSync } from 'node:fs'; import * as mod from 'node:module'; +import * as path from 'node:path'; import { fileURLToPath } from 'node:url'; let registered = false; +let recording: Set | undefined; + +/** An app file: a `file:` module outside every `node_modules` directory. */ +function appFile(url: string): string | undefined { + if (!url.startsWith('file:')) return undefined; + const file = fileURLToPath(url); + return file.split(path.sep).includes('node_modules') ? undefined : file; +} + +function record(url: string): void { + const file = recording === undefined ? undefined : appFile(url); + if (file !== undefined) recording?.add(file); +} + +/** + * Runs `load` and returns the app files it imported: every module outside + * `node_modules` that the runtime loaded for the first time while it ran, + * static and dynamic imports alike. Under Node the resolve hook records them; + * under Bun they are the modules `require.cache` gained, which includes ES + * modules. A module already loaded before `load` started, and what only it + * imports, is not recorded. + */ +export async function recordAppFiles( + load: () => Promise, +): Promise<{ readonly value: T; readonly files: readonly string[] }> { + if (typeof process.versions.bun === 'string') { + const cache = mod.createRequire(import.meta.url).cache; + const before = new Set(Object.keys(cache)); + const value = await load(); + const files = Object.keys(cache).flatMap((key) => { + if (before.has(key) || !path.isAbsolute(key)) return []; + return key.split(path.sep).includes('node_modules') ? [] : [key]; + }); + return { value, files }; + } + registerEntryResolution(); + const files = new Set(); + recording = files; + try { + return { value: await load(), files: [...files] }; + } finally { + recording = undefined; + } +} /** * Registers the resolve hook once (idempotent). Call at the start of @@ -37,7 +85,9 @@ export function registerEntryResolution(): void { mod.registerHooks({ resolve(specifier, context, nextResolve) { try { - return nextResolve(specifier, context); + const resolved = nextResolve(specifier, context); + record(resolved.url); + return resolved; } catch (error) { if ( !isRelativeSpecifier(specifier) || @@ -48,7 +98,9 @@ export function registerEntryResolution(): void { } for (const candidate of sourceCandidates(new URL(specifier, context.parentURL))) { if (existsSync(fileURLToPath(candidate))) { - return nextResolve(candidate.href, context); + const resolved = nextResolve(candidate.href, context); + record(resolved.url); + return resolved; } } throw error; diff --git a/packages/0-framework/3-tooling/cli/src/family/__tests__/dev-converge.test.ts b/packages/0-framework/3-tooling/cli/src/family/__tests__/dev-converge.test.ts index 433d5694d..2cd19e4bd 100644 --- a/packages/0-framework/3-tooling/cli/src/family/__tests__/dev-converge.test.ts +++ b/packages/0-framework/3-tooling/cli/src/family/__tests__/dev-converge.test.ts @@ -55,64 +55,54 @@ describe('dev — converging in process', () => { expect(result.exitCode).toBe(130); }); - test('each resource is a step, a failed one with its message, and log lines are messages', async () => { + test('each converge is a step with its resources inside it, and log lines are messages', async () => { + const rebuild = { name: 'converge', id: 'dev.rebuild.2' } as const; + const web = { + name: 'apply-resource', + converge: 'dev.rebuild.2', + fqn: 'shop.web', + resourceType: 'Local.Compute', + } as const; const { result } = await runWithEvents( [ + { kind: 'step-started', step: rebuild }, + { kind: 'step-started', step: { ...web, status: 'updating' } }, + { kind: 'message', severity: 'info', text: 'Plan: 1 to update' }, { - kind: 'converge-resource', - fqn: 'shop.web', - resourceType: 'Local.Compute', - status: 'updating', - phase: 'running', + kind: 'step-finished', + step: { ...web, status: 'updated' }, + outcome: 'ok', + durationMs: 5, }, - { - kind: 'converge-resource', - fqn: 'shop.web', - resourceType: 'Local.Compute', - status: 'updated', - phase: 'done', - }, - { - kind: 'converge-resource', - fqn: 'shop.db', - resourceType: 'Local.Postgres', - status: 'fail', - phase: 'failed', - message: 'port in use', - }, - { kind: 'converge-message', severity: 'info', text: 'Plan: 1 to update' }, + { kind: 'step-finished', step: rebuild, outcome: 'ok', durationMs: 9 }, ], - (event) => event.kind === 'message' && event.text === 'Plan: 1 to update', + (event) => event.kind === 'step-finished' && event.id === 'dev.rebuild.2', ); + expect(result.events).toContainEqual({ + kind: 'step-started', + step: 'rebuild 2', + id: 'dev.rebuild.2', + }); expect(result.events).toContainEqual({ kind: 'step-started', step: 'updating shop.web', - id: 'dev.converge.shop.web', + id: 'dev.rebuild.2.shop.web', + parentId: 'dev.rebuild.2', data: { resourceType: 'Local.Compute' }, }); expect(result.events).toContainEqual({ kind: 'step-finished', step: 'updated shop.web', - id: 'dev.converge.shop.web', + id: 'dev.rebuild.2.shop.web', outcome: 'ok', - }); - expect(result.events).toContainEqual({ - kind: 'step-finished', - step: 'fail shop.db', - id: 'dev.converge.shop.db', - outcome: 'failed', + data: { durationMs: 5 }, }); expect(result.events).toContainEqual({ kind: 'message', - severity: 'warn', - text: 'shop.db: port in use', + severity: 'info', + text: 'Plan: 1 to update', }); - expect( - result.events.filter( - (event) => event.kind === 'step-started' && event.id === 'dev.converge.shop.web', - ), - ).toHaveLength(1); }); test('a failed rebuild is a warning with the cause, and no reproduce command', async () => { @@ -122,6 +112,7 @@ describe('dev — converging in process', () => { [ { kind: 'converge-failed', + code: 'DEV.CONVERGE_FAILED', message: 'ProviderError: the provider refused', reasons: [{ tag: 'ProviderError', message: 'the provider refused' }], }, @@ -133,14 +124,17 @@ describe('dev — converging in process', () => { kind: 'message', severity: 'warn', text, - data: { reasons: [{ tag: 'ProviderError', message: 'the provider refused' }] }, + data: { + code: 'DEV.CONVERGE_FAILED', + reasons: [{ tag: 'ProviderError', message: 'the provider refused' }], + }, }); expect(result.events.some((event) => event.kind === 'remediation')).toBe(false); }); test('an edit to the app definition renders as a warning that asks for a restart', async () => { const text = - "/repo/shop/src/service.ts changed. dev loaded the app's definition when it started and cannot load it again; restart dev to apply it. Rebuilds are paused until then."; + '/repo/shop/src/service.ts changed. dev loaded the app definition files when it started and cannot load them again; restart dev to apply the change. Rebuilds are paused until then.'; const { result } = await runWithEvents( [{ kind: 'definition-changed', files: ['/repo/shop/src/service.ts'] }], (event) => event.kind === 'message' && event.text === text, diff --git a/packages/0-framework/3-tooling/cli/src/family/commands/dev.ts b/packages/0-framework/3-tooling/cli/src/family/commands/dev.ts index 51722ede8..362e165f2 100644 --- a/packages/0-framework/3-tooling/cli/src/family/commands/dev.ts +++ b/packages/0-framework/3-tooling/cli/src/family/commands/dev.ts @@ -18,7 +18,7 @@ import * as path from 'node:path'; import type { EngineEvent } from '@prisma/cli-engine'; import { defineSessionCommand, flag } from '@prisma/cli-engine'; import { notOk, ok } from '@prisma/cli-engine/protocol'; -import type { DevEvent, DevInput } from '../../operations/dev.ts'; +import type { DevEvent, DevInput, DevStep } from '../../operations/dev.ts'; import { DEV_ENVIRONMENT, type ServiceEndpoint } from '../../operations/shared.ts'; import { withConsoleOnStderr } from '../console-to-stderr.ts'; import type { ComposerOperations } from '../family.ts'; @@ -35,6 +35,15 @@ function stopRequested(signal: AbortSignal): Promise { }); } +function devStepId(step: DevStep): string { + return step.name === 'converge' ? step.id : `${step.converge}.${step.fqn}`; +} + +function devStepLabel(step: DevStep): string { + if (step.name === 'apply-resource') return `${step.status} ${step.fqn}`; + return step.id === 'dev.converge' ? 'converge' : step.id.replace(/^dev\.rebuild\./, 'rebuild '); +} + /** The front door, ordered by address depth (fewest dots first) then lexicographically. */ function frontDoorOrder(endpoints: readonly ServiceEndpoint[]): readonly ServiceEndpoint[] { return [...endpoints].sort((a, b) => { @@ -53,7 +62,6 @@ function reportDevEvent( cwd: string, ): (event: DevEvent) => void { let stopFailed = false; - const openResources = new Set(); return (event) => { switch (event.kind) { @@ -133,32 +141,32 @@ function reportDevEvent( report({ kind: 'message', severity: 'warn', - text: `${event.files.join(', ')} changed. dev loaded the app's definition when it started and cannot load it again; restart dev to apply it. Rebuilds are paused until then.`, + text: `${event.files.join(', ')} changed. dev loaded the app definition files when it started and cannot load them again; restart dev to apply the change. Rebuilds are paused until then.`, }); return; - case 'converge-resource': { - const id = `dev.converge.${event.fqn}`; - const step = `${event.status} ${event.fqn}`; - if (!openResources.has(id)) { - openResources.add(id); - report({ kind: 'step-started', step, id, data: { resourceType: event.resourceType } }); - } - if (event.phase === 'running') return; - openResources.delete(id); + case 'step-started': + report({ + kind: 'step-started', + step: devStepLabel(event.step), + id: devStepId(event.step), + ...(event.step.name === 'apply-resource' + ? { parentId: event.step.converge, data: { resourceType: event.step.resourceType } } + : {}), + }); + return; + + case 'step-finished': report({ kind: 'step-finished', - step, - id, - outcome: event.phase === 'done' ? 'ok' : 'failed', + step: devStepLabel(event.step), + id: devStepId(event.step), + outcome: event.outcome, + data: { durationMs: event.durationMs }, }); - if (event.message !== undefined) { - report({ kind: 'message', severity: 'warn', text: `${event.fqn}: ${event.message}` }); - } return; - } - case 'converge-message': + case 'message': report({ kind: 'message', severity: event.severity, text: event.text }); return; @@ -167,7 +175,7 @@ function reportDevEvent( kind: 'message', severity: 'warn', text: `Converge failed; the running app is untouched and dev is still watching.\n${event.message}`, - data: { reasons: event.reasons }, + data: { code: event.code, reasons: event.reasons }, }); return; @@ -223,8 +231,8 @@ export const createDevCommand = (operations: ComposerOperations) => 'this machine.', description: 'Runs credential-free and watches the app for changes, reconverging when a ' + - "service's build output changes. An edit to the entry module or to a file a service " + - 'is declared in needs a restart of dev, which says so. ' + + "service's build output changes. An edit to an app definition file (the entry module, " + + 'or any other app file it loaded) needs a restart of dev, which says so. ' + "A service whose build adapter declares a dev server (Next.js's `next dev`, or the " + '`dev` command written on node()) runs that instead of its build, and its output is ' + 'shown here, prefixed with its address. ' + diff --git a/packages/0-framework/3-tooling/cli/src/load-entry.ts b/packages/0-framework/3-tooling/cli/src/load-entry.ts index 938c6fee9..5de077e65 100644 --- a/packages/0-framework/3-tooling/cli/src/load-entry.ts +++ b/packages/0-framework/3-tooling/cli/src/load-entry.ts @@ -10,13 +10,15 @@ import type { ModuleNode, ServiceNode } from '@internal/core'; import { isNode } from '@internal/core'; import { blindCast } from '@internal/foundation/casts'; import { CliStructuredError } from '@internal/foundation/errors'; -import { registerEntryResolution } from './entry-resolution.ts'; +import { recordAppFiles, registerEntryResolution } from './entry-resolution.ts'; import { explainJsxLoadError } from './jsx-load-error.ts'; export interface LoadedEntry { /** The resolved absolute path to the entry module on disk. */ readonly path: string; readonly root: ServiceNode | ModuleNode; + /** The app definition files: the entry and every other app file (outside `node_modules`) this import loaded for the first time. A later import of a cached entry yields only the entry. */ + readonly files: readonly string[]; } /** `unloadableFix`, when given, says where the path came from, so a wrong path names its source. */ @@ -28,8 +30,9 @@ export async function loadEntry( registerEntryResolution(); const resolvedPath = path.resolve(cwd, entryArg); let mod: unknown; + let files: readonly string[]; try { - mod = await import(pathToFileURL(resolvedPath).href); + ({ value: mod, files } = await recordAppFiles(() => import(pathToFileURL(resolvedPath).href))); } catch (error) { const explained = explainJsxLoadError(error, resolvedPath); if (explained !== undefined) { @@ -68,6 +71,7 @@ export async function loadEntry( return { path: resolvedPath, + files: [...new Set([resolvedPath, ...files])], root: blindCast< ServiceNode | ModuleNode, "isNode() plus the kind check above prove root is a service or module node at runtime; isNode's return type (the branded ServiceNode | ResourceNode | DependencyEnd | ModuleNode union) structurally lacks each kind's own fields, so TS cannot narrow further on its own" diff --git a/packages/0-framework/3-tooling/cli/src/operations/__tests__/operations.test.ts b/packages/0-framework/3-tooling/cli/src/operations/__tests__/operations.test.ts index acfafe675..f1f6108de 100644 --- a/packages/0-framework/3-tooling/cli/src/operations/__tests__/operations.test.ts +++ b/packages/0-framework/3-tooling/cli/src/operations/__tests__/operations.test.ts @@ -2368,7 +2368,7 @@ describe.skipIf(process.platform === 'win32')('dev()', () => { } }; - test("a rebuild's converge progress reaches the event stream", async () => { + test('each converge is a step around its resources, and a failed one closes every step it opened', async () => { const app = makeAppDir('hello-dev'); const events: DevEvent[] = []; let converges = 0; @@ -2389,45 +2389,91 @@ describe.skipIf(process.platform === 'win32')('dev()', () => { kind: 'resource', fqn: 'hello-dev.app', resourceType: 'Local.Compute', - status: 'updated', - phase: 'done', + status: converges === 1 ? 'created' : 'updating', + phase: converges === 1 ? 'done' : 'running', }); request.onEvent?.({ kind: 'log', severity: 'info', text: 'applied' }); - return APPLIED; + return converges === 1 ? APPLIED : FAILED; }, events, ); if (!start.ok) throw start.failure; change(); - await waitUntil( - () => converges === 2 && events.filter((e) => e.kind === 'ready').length === 2, - ); + await waitUntil(() => events.some((event) => event.kind === 'converge-failed')); await start.value.stop(); }); - const progress = events.filter( - (event) => event.kind === 'converge-resource' || event.kind === 'converge-message', - ); - expect(progress).toEqual([ - { - kind: 'converge-resource', - fqn: 'hello-dev.app', - resourceType: 'Local.Compute', - status: 'updated', - phase: 'done', - }, - { kind: 'converge-message', severity: 'info', text: 'applied' }, - { - kind: 'converge-resource', - fqn: 'hello-dev.app', - resourceType: 'Local.Compute', - status: 'updated', - phase: 'done', - }, - { kind: 'converge-message', severity: 'info', text: 'applied' }, + const steps = events.flatMap((event): object[] => { + if (event.kind === 'step-started') return [{ started: event.step }]; + if (event.kind === 'step-finished') return [{ finished: event.step, outcome: event.outcome }]; + if (event.kind === 'message') return [{ message: event.text }]; + return []; + }); + const resource = (converge: string, status: string) => ({ + name: 'apply-resource', + converge, + fqn: 'hello-dev.app', + resourceType: 'Local.Compute', + status, + }); + expect(steps).toEqual([ + { started: { name: 'converge', id: 'dev.converge' } }, + { started: resource('dev.converge', 'created') }, + { finished: resource('dev.converge', 'created'), outcome: 'ok' }, + { message: 'applied' }, + { finished: { name: 'converge', id: 'dev.converge' }, outcome: 'ok' }, + { started: { name: 'converge', id: 'dev.rebuild.1' } }, + { started: resource('dev.rebuild.1', 'updating') }, + { message: 'applied' }, + { finished: resource('dev.rebuild.1', 'updating'), outcome: 'failed' }, + { finished: { name: 'converge', id: 'dev.rebuild.1' }, outcome: 'failed' }, ]); }, 15_000); + test("stopping during a rebuild's pipeline returns without waiting for it, and the rebuild does not converge", async () => { + const app = makeAppDir('hello-dev'); + let change: () => void = () => undefined; + let assembles = 0; + let release: () => void = () => undefined; + const assembling = new Promise((resolve) => { + release = resolve; + }); + let converges = 0; + + await silently(async () => { + const start = await devWithDeps( + { + config: composerConfig(devConfigWith(runningAttachment)), + entry: app.entryPath, + cwd: app.dir, + }, + { + runAssembler: async (node: ServiceNode) => { + assembles += 1; + if (assembles > 1) await assembling; + return fakeAssembler(node); + }, + converge: appConverge(async () => { + converges += 1; + return APPLIED; + }), + watch: (targets, onChange) => { + if (targets.some((target) => target.paths.includes(app.entryPath))) change = onChange; + return { ready: Promise.resolve(), stop: () => undefined }; + }, + }, + ); + if (!start.ok) throw start.failure; + change(); + await waitUntil(() => assembles === 2); + await start.value.stop(); + release(); + await new Promise((resolve) => setTimeout(resolve, 50)); + }); + + expect(converges).toBe(1); + }, 15_000); + test('a failed rebuild is reported with its cause, keeps the app running, and the next change converges again', async () => { const app = makeAppDir('hello-dev'); const events: DevEvent[] = []; @@ -2455,6 +2501,7 @@ describe.skipIf(process.platform === 'win32')('dev()', () => { expect(events.filter((event) => event.kind === 'converge-failed')).toEqual([ { kind: 'converge-failed', + code: 'DEV.CONVERGE_FAILED', message: 'ProviderError: the provider refused', reasons: [{ tag: 'ProviderError', message: 'the provider refused' }], }, @@ -2575,6 +2622,95 @@ describe.skipIf(process.platform === 'win32')('dev()', () => { expect(converges).toBe(1); }, 15_000); + test('an edit to any app file the entry loaded is reported, not only the entry and service files', async () => { + const app = makeAppDir('hello-dev'); + const helper = path.join(app.dir, 'inputs.ts'); + fs.writeFileSync(helper, 'export const inputs = {};\n'); + fs.writeFileSync( + app.entryPath, + fs + .readFileSync(app.entryPath, 'utf8') + .replace(' inputs: {},', ' inputs,') + .replace('\nexport default', "\nimport { inputs } from './inputs.ts';\nexport default"), + ); + const events: DevEvent[] = []; + let converges = 0; + + await silently(async () => { + const { start, change } = await devWithBuildWatch( + app, + async () => { + converges += 1; + return APPLIED; + }, + events, + ); + if (!start.ok) throw start.failure; + fs.appendFileSync(helper, '// an edit\n'); + change(); + await new Promise((resolve) => setTimeout(resolve, 200)); + await start.value.stop(); + }); + + expect(events.filter((event) => event.kind === 'definition-changed')).toEqual([ + { kind: 'definition-changed', files: [helper] }, + ]); + expect(converges).toBe(1); + }, 15_000); + + test.each([ + ['already-running', 'DEV.CONVERGE_RUNNING'], + ['worker-thread', 'DEV.WORKER_THREAD'], + ] as const)( + 'a first converge refused as %s is %s', + async (reason, code) => { + const app = makeAppDir('hello-dev'); + + const result = await silently(async () => { + const { start } = await devWithBuildWatch( + app, + async () => ({ kind: 'refused', reason }), + [], + ); + return start; + }); + + if (result.ok) throw new Error('expected a refusal'); + expect(result.failure.code).toBe(code); + }, + 15_000, + ); + + test("a refused rebuild's converge-failed carries the refusal's code", async () => { + const app = makeAppDir('hello-dev'); + const events: DevEvent[] = []; + let converges = 0; + + await silently(async () => { + const { start, change } = await devWithBuildWatch( + app, + async () => { + converges += 1; + return converges === 1 ? APPLIED : { kind: 'refused', reason: 'already-running' }; + }, + events, + ); + if (!start.ok) throw start.failure; + change(); + await waitUntil(() => events.some((event) => event.kind === 'converge-failed')); + await start.value.stop(); + }); + + expect(events.filter((event) => event.kind === 'converge-failed')).toEqual([ + { + kind: 'converge-failed', + code: 'DEV.CONVERGE_RUNNING', + message: 'Another converge is already running in this process.', + reasons: [], + }, + ]); + }, 15_000); + test('a save that leaves the definition files unchanged still rebuilds', async () => { const app = makeAppDir('hello-dev'); const events: DevEvent[] = []; @@ -2744,7 +2880,15 @@ describe.skipIf(process.platform === 'win32')('dev()', () => { }); expect(result.ok).toBe(true); - expect(events).toEqual(['ready', 'unwatchable', 'stopping', 'stop-error', 'stopped']); + expect(events).toEqual([ + 'step-started', + 'step-finished', + 'ready', + 'unwatchable', + 'stopping', + 'stop-error', + 'stopped', + ]); }, 15_000); test('the DevSession contract: closed settles only via stop(), stop() is idempotent, and no process signal handler is ever registered', async () => { @@ -2837,7 +2981,7 @@ describe.skipIf(process.platform === 'win32')('dev()', () => { expect(watches).toBe(0); expect(stops).toBe(0); - expect(events).toEqual(['ready']); + expect(events).toEqual(['step-started', 'step-finished', 'ready']); }, 15_000); test('a host onEvent that throws cannot prevent closed from settling', async () => { @@ -2903,7 +3047,7 @@ describe.skipIf(process.platform === 'win32')('dev()', () => { expect(result.ok).toBe(false); if (result.ok) throw new Error('unreachable'); - expect(result.failure.code).toBe('DEV.STATE_WRITE_FAILED'); + expect(result.failure.code).toBe('DEV.ENVIRONMENT_WRITE_FAILED'); expect(convergeRan).toBe(false); }, 15_000); }); diff --git a/packages/0-framework/3-tooling/cli/src/operations/deploy.ts b/packages/0-framework/3-tooling/cli/src/operations/deploy.ts index a88947765..834404ebe 100644 --- a/packages/0-framework/3-tooling/cli/src/operations/deploy.ts +++ b/packages/0-framework/3-tooling/cli/src/operations/deploy.ts @@ -11,6 +11,7 @@ import { notOk, ok, type Result } from '@internal/foundation/result'; import type { ComposerConfigSource } from '../composer-config.ts'; import type { DeploymentSummary } from '../deployment-summary.ts'; import { executorLoadFailure, type OperationDeps } from './shared.ts'; +import type { StepEvent } from './steps.ts'; /** A deploy step. `assemble-service` steps run inside `assemble`, one per service, and `apply-resource` steps inside `apply`, one per resource Alchemy touches. */ export type DeployStep = @@ -25,22 +26,7 @@ export type DeployStep = readonly status: string; }; -export type DeployEvent = - | { readonly kind: 'step-started'; readonly step: DeployStep } - | { - readonly kind: 'message'; - readonly severity: 'info' | 'warn' | 'verbose'; - /** A line Alchemy logged while it planned or applied. */ - readonly text: string; - } - | { - readonly kind: 'step-finished'; - readonly step: DeployStep; - readonly outcome: 'ok' | 'failed'; - readonly durationMs: number; - /** What the step measured, as its build adapter or deploy target reported it. The fields vary by adapter and target. */ - readonly data?: Readonly> | undefined; - }; +export type DeployEvent = StepEvent; /** The environment `deploy` targets when none is named. */ export const DEFAULT_DEPLOY_ENVIRONMENT = 'production'; diff --git a/packages/0-framework/3-tooling/cli/src/operations/dev.ts b/packages/0-framework/3-tooling/cli/src/operations/dev.ts index 398fec047..12bce2b2f 100644 --- a/packages/0-framework/3-tooling/cli/src/operations/dev.ts +++ b/packages/0-framework/3-tooling/cli/src/operations/dev.ts @@ -11,8 +11,25 @@ import type { CliStructuredError } from '@internal/foundation/errors'; import { notOk, type Result } from '@internal/foundation/result'; import type { ComposerConfigSource } from '../composer-config.ts'; import { executorLoadFailure, type OperationDeps, type ServiceEndpoint } from './shared.ts'; +import type { StepEvent } from './steps.ts'; + +/** A step of a converge. Each converge is one `converge` step, `dev.converge` for the first and `dev.rebuild.` for the n-th rebuild, and each resource Alchemy touches is an `apply-resource` step inside it. */ +export type DevStep = + | { readonly name: 'converge'; readonly id: string } + | { + readonly name: 'apply-resource'; + /** The id of the converge step it belongs to. */ + readonly converge: string; + /** The resource's id in the stack. */ + readonly fqn: string; + readonly resourceType: string; + /** Alchemy's status word for what is happening to it, such as `creating` or `created`. */ + readonly status: string; + }; export type DevEvent = + /** A converge's steps and Alchemy's log lines. Every step a converge opens is finished when it ends, as `failed` when it failed or was interrupted. */ + | StepEvent /** Initial front door + after each successful re-converge. */ | { readonly kind: 'ready'; readonly endpoints: readonly ServiceEndpoint[] } | { readonly kind: 'unwatchable'; readonly address: string } @@ -25,27 +42,12 @@ export type DevEvent = | { readonly kind: 'config-changed'; readonly file: string } /** What the `config` or `secrets` adapters return changed since the session started. The session keeps the values it started with and stops rebuilding until it is restarted. */ | { readonly kind: 'values-changed' } - /** A file that defines the app (the entry module, or a file a service is declared in) changed. The session keeps the app it loaded at start and stops rebuilding until it is restarted. */ + /** An app definition file changed: one of the app files the entry module loaded when the session started. The session keeps the app it loaded and stops rebuilding until it is restarted. */ | { readonly kind: 'definition-changed'; readonly files: readonly string[] } - /** One resource's progress in a converge: the first one, or a rebuild's. */ - | { - readonly kind: 'converge-resource'; - readonly fqn: string; - readonly resourceType: string; - /** Alchemy's own status word, such as `creating` or `updated`. */ - readonly status: string; - readonly phase: 'running' | 'done' | 'failed'; - readonly message?: string | undefined; - } - /** A line Alchemy logged during a converge. */ - | { - readonly kind: 'converge-message'; - readonly severity: 'info' | 'warn' | 'verbose'; - readonly text: string; - } - /** A rebuild's converge failed. The app keeps running, still watching. */ + /** A rebuild's converge failed, or was refused because another converge holds this process. The app keeps running, still watching. */ | { readonly kind: 'converge-failed'; + readonly code: 'DEV.CONVERGE_FAILED' | 'DEV.CONVERGE_RUNNING' | 'DEV.WORKER_THREAD'; readonly message: string; readonly reasons: readonly { readonly tag: string; readonly message: string }[]; } @@ -85,7 +87,7 @@ export interface DevInput { readonly detach?: boolean | undefined; readonly cwd?: string | undefined; readonly onEvent?: ((event: DevEvent) => void) | undefined; - /** Interrupts a running converge. The first converge then fails with `DEV.INTERRUPTED`; a rebuild's ends quietly. It does not stop the session: `stop()` does. */ + /** Interrupts a running converge. The first converge then fails with `DEV.INTERRUPTED`; a rebuild's ends quietly. It does not stop the session: `stop()` does. One process runs one converge at a time, so a `deploy` in the same process while a converge runs is refused, and so is a dev converge while a deploy runs. */ readonly signal?: AbortSignal | undefined; } diff --git a/packages/0-framework/3-tooling/cli/src/operations/execute-deploy-destroy.ts b/packages/0-framework/3-tooling/cli/src/operations/execute-deploy-destroy.ts index 6e44afe1c..7d7c72c94 100644 --- a/packages/0-framework/3-tooling/cli/src/operations/execute-deploy-destroy.ts +++ b/packages/0-framework/3-tooling/cli/src/operations/execute-deploy-destroy.ts @@ -20,7 +20,7 @@ import { CliStructuredError, InternalError } from '@internal/foundation/errors'; import { notOk, ok, okVoid, type Result } from '@internal/foundation/result'; import { type AppConverge, loadAppConverge } from '../app-converge.ts'; import type { ComposerConfigSource } from '../composer-config.ts'; -import type { ConvergeEvent, ConvergeOutcome, LoweredStack } from '../converge-in-process.ts'; +import type { ConvergeOutcome, LoweredStack } from '../converge-in-process.ts'; import { capFailureCause, convergeRedactor, @@ -62,6 +62,7 @@ import { requireContainerForEnvironment, toStructured, } from './shared.ts'; +import { reportConvergeEvent, stepTracker } from './steps.ts'; const ALCHEMY_STATE_DIR = '.alchemy'; @@ -71,58 +72,17 @@ function hasNoLocalDeployState(cwd: string): boolean { return !(fs.existsSync(stateDir) && fs.readdirSync(stateDir).length > 0); } -/** - * Times each deploy step for the host's `onEvent`. A step still open when the - * run fails is closed as failed, innermost first. A host `onEvent` that throws - * is the host's bug and cannot fail the deploy. - */ +/** Times each deploy step for the host's `onEvent`. A step still open when the run fails is closed as failed, innermost first. */ function deploySteps(onEvent: ((event: DeployEvent) => void) | undefined) { - const open = new Map(); - const keyOf = (step: DeployStep): string => - step.name === 'assemble-service' - ? `${step.name}:${step.address}` - : step.name === 'apply-resource' - ? `${step.name}:${step.fqn}` - : step.name; - const emit = (event: DeployEvent): void => { - try { - onEvent?.(event); - } catch { - // Rendering is the host's; its failures are not the deploy's. - } - }; - const finish = ( - step: DeployStep, - outcome: 'ok' | 'failed', - data?: Readonly>, - ): void => { - const entry = open.get(keyOf(step)); - if (entry === undefined) return; - open.delete(keyOf(step)); - const durationMs = Math.round(performance.now() - entry.startedAt); - emit({ kind: 'step-finished', step, outcome, durationMs, data }); - }; - return { - start(step: DeployStep): void { - open.set(keyOf(step), { step, startedAt: performance.now() }); - emit({ kind: 'step-started', step }); - }, - finish(step: DeployStep, data?: Readonly>): void { - finish(step, 'ok', data); - }, - fail(step: DeployStep): void { - finish(step, 'failed'); - }, - failOpen(): void { - for (const { step } of [...open.values()].reverse()) finish(step, 'failed'); - }, - isOpen(step: DeployStep): boolean { - return open.has(keyOf(step)); - }, - message(severity: 'info' | 'warn' | 'verbose', text: string): void { - emit({ kind: 'message', severity, text }); - }, - }; + return stepTracker( + (step) => + step.name === 'assemble-service' + ? `${step.name}:${step.address}` + : step.name === 'apply-resource' + ? `${step.name}:${step.fqn}` + : step.name, + onEvent, + ); } type DeploySteps = ReturnType; @@ -323,7 +283,7 @@ async function runStackPipeline( try { outcome = await runStackPipelineInner(action, opts, reporters); } catch (error) { - opts.steps.failOpen(); + opts.steps.closeOpen('failed'); opts.steps.start({ name: 'record' }); // A defect, not a structured failure — still the end of the run, and the // only chance to record that it ended at all. @@ -336,7 +296,7 @@ async function runStackPipeline( opts.steps.finish({ name: 'record' }); throw error; } - if (!outcome.ok) opts.steps.failOpen(); + if (!outcome.ok) opts.steps.closeOpen('failed'); opts.steps.start({ name: 'record' }); await finishReporters( reporters, @@ -823,42 +783,6 @@ async function prepareConverge( }); } -/** One Alchemy progress event as the deploy's step events: each resource is a step inside `apply`. */ -function reportConvergeEvent( - steps: DeploySteps, - event: ConvergeEvent, - redact: (text: string) => string, -): void { - if (event.kind === 'log') { - steps.message(event.severity, redact(withComposerAdvice(event.text))); - return; - } - const step: DeployStep = { - name: 'apply-resource', - fqn: event.fqn, - resourceType: event.resourceType, - status: event.status, - }; - switch (event.phase) { - case 'running': - if (!steps.isOpen(step)) steps.start(step); - return; - case 'done': - if (!steps.isOpen(step)) steps.start(step); - steps.finish(step); - return; - case 'failed': - if (!steps.isOpen(step)) steps.start(step); - steps.fail(step); - if (event.message !== undefined) { - steps.message('warn', redact(`${event.fqn}: ${withComposerAdvice(event.message)}`)); - } - return; - case 'pending': - return; - } -} - /** The structured failure a converge that did not apply settles as. */ function convergeFailure( outcome: Exclude, @@ -972,7 +896,16 @@ async function convergeDeploy( cwd, env, signal: opts.signal, - onEvent: (event) => reportConvergeEvent(steps, event, redact), + onEvent: (event) => + reportConvergeEvent( + steps, + event, + redact, + (resource): DeployStep => ({ + name: 'apply-resource', + ...resource, + }), + ), }); } catch (error) { return notOk( diff --git a/packages/0-framework/3-tooling/cli/src/operations/execute-dev.ts b/packages/0-framework/3-tooling/cli/src/operations/execute-dev.ts index fa3c6f6dd..7d559343b 100644 --- a/packages/0-framework/3-tooling/cli/src/operations/execute-dev.ts +++ b/packages/0-framework/3-tooling/cli/src/operations/execute-dev.ts @@ -11,7 +11,7 @@ import { createHash } from 'node:crypto'; import * as fs from 'node:fs'; import * as path from 'node:path'; import { isDeepStrictEqual } from 'node:util'; -import type { Graph, ServiceNode } from '@internal/core'; +import type { Graph, ModuleNode, ServiceNode } from '@internal/core'; import type { ConfigPools, ContainerInstance, PrismaAppConfig } from '@internal/core/config'; import type { DevServer } from '@internal/core/deploy'; import type { LocalTargetAttachment, LocalTargetDescriptor } from '@internal/core/local-target'; @@ -19,7 +19,7 @@ import { devEnvironmentDir, resolveLocalTargets } from '@internal/core/local-tar import { CliStructuredError, InternalError } from '@internal/foundation/errors'; import { notOk, ok, type Result } from '@internal/foundation/result'; import { type AppConverge, loadAppConverge } from '../app-converge.ts'; -import type { ConvergeEvent, ConvergeOutcome } from '../converge-in-process.ts'; +import type { ConvergeOutcome } from '../converge-in-process.ts'; import { convergeRedactor, poolSecrets, withComposerAdvice } from '../deployment-summary.ts'; import { authoringModuleTargets, @@ -29,7 +29,7 @@ import { } from '../dev/watch.ts'; import { type PipelineDeps, type PipelineResult, runPipeline } from '../pipeline.ts'; import { validateEnvironmentName } from '../validate-environment.ts'; -import type { DevEvent, DevInput, DevSession } from './dev.ts'; +import type { DevEvent, DevInput, DevSession, DevStep } from './dev.ts'; import { claimDevEnvironmentName, requireDevEnvironmentName } from './dev-environment-name.ts'; import { withEmulatorRetry } from './emulator-retry.ts'; import { @@ -41,6 +41,7 @@ import { type ServiceEndpoint, toStructured, } from './shared.ts'; +import { reportConvergeEvent, stepTracker } from './steps.ts'; function failureMessage(error: unknown): string { return error instanceof Error ? error.message : String(error); @@ -135,32 +136,48 @@ function samePools(started: ConfigPools, now: ConfigPools): boolean { ); } -/** What a converge that did not apply failed with, as text; undefined for an interruption. */ -function convergeFailureText( - outcome: Exclude, +type FailedConverge = Exclude; + +/** The code and text a converge that failed or was refused settles as. */ +function convergeFailure( + outcome: FailedConverge, redact: (text: string) => string, -): string | undefined { - switch (outcome.kind) { - case 'interrupted': - return undefined; - case 'refused': - return outcome.reason === 'already-running' - ? 'Another converge is already running in this process.' - : 'dev cannot converge in a worker thread when its cwd is not the process working directory.'; - case 'failed': - return redact(withComposerAdvice(convergeReasonsText(outcome.reasons))); +): { + readonly code: 'DEV.CONVERGE_FAILED' | 'DEV.CONVERGE_RUNNING' | 'DEV.WORKER_THREAD'; + readonly message: string; + readonly reasons: readonly { readonly tag: string; readonly message: string }[]; +} { + if (outcome.kind === 'refused') { + return outcome.reason === 'already-running' + ? { + code: 'DEV.CONVERGE_RUNNING', + message: 'Another converge is already running in this process.', + reasons: [], + } + : { + code: 'DEV.WORKER_THREAD', + message: + 'dev cannot converge in a worker thread when its cwd is not the process working directory.', + reasons: [], + }; } - return 'The converge ended with an outcome this Composer does not know.'; + return { + code: 'DEV.CONVERGE_FAILED', + message: redact(withComposerAdvice(convergeReasonsText(outcome.reasons))), + reasons: outcome.reasons.map(({ tag, message }) => ({ tag, message: redact(message) })), + }; } -function failureReasonsOf( - outcome: ConvergeOutcome, - redact: (text: string) => string, -): readonly { readonly tag: string; readonly message: string }[] { - return outcome.kind === 'failed' - ? outcome.reasons.map(({ tag, message }) => ({ tag, message: redact(message) })) - : []; -} +const REFUSAL_HELP = { + 'DEV.CONVERGE_RUNNING': { + why: "A converge sets the process's environment and working directory while Alchemy runs, so two cannot overlap.", + fix: 'Wait for the other converge to finish, or run dev in its own process.', + }, + 'DEV.WORKER_THREAD': { + why: 'Alchemy reads `.env` and keeps `.alchemy` in the working directory, and a worker thread cannot change it.', + fix: "Call dev from the main thread, or with the process's working directory as its cwd.", + }, +} as const; function thrownOutcome(error: unknown): ConvergeOutcome { return { @@ -196,7 +213,8 @@ export async function executeDev( let pipeline: Awaited>; let resolved: ReadonlyMap; let appConverge: AppConverge; - // The files that define the app, hashed as they were when it was imported. + // The app definition files: every app file the entry import loaded, and + // each service's authoring module, hashed as they were when it loaded. let startedDefinitions: ReadonlyMap; const containers = new Map(); const preflightPayloads = new Map(); @@ -236,9 +254,12 @@ export async function executeDev( const { graph, name } = pipeline; const config = pipeline.configSource.value; startedDefinitions = new Map( - authoringModuleTargets(pipeline.entryModule.path, graph) - .flatMap((target) => target.paths) - .map((file) => [file, contentHash(file)]), + [ + ...pipeline.entryModule.files, + ...authoringModuleTargets(pipeline.entryModule.path, graph).flatMap( + (target) => target.paths, + ), + ].map((file) => [file, contentHash(file)]), ); // Lowering and Alchemy run from the app's own install, found before @@ -333,46 +354,42 @@ export async function executeDev( process.env, poolSecrets(pipeline.pools, config.secrets !== undefined), ); - const reportConverge = (event: ConvergeEvent): void => { - if (event.kind === 'log') { - emit({ - kind: 'converge-message', - severity: event.severity, - text: redact(withComposerAdvice(event.text)), - }); - return; - } - if (event.phase === 'pending') return; - emit({ - kind: 'converge-resource', - fqn: event.fqn, - resourceType: event.resourceType, - status: event.status, - phase: event.phase, - ...(event.message === undefined - ? {} - : { message: redact(withComposerAdvice(event.message)) }), - }); - }; + const steps = stepTracker( + (step) => (step.name === 'converge' ? step.id : `${step.converge}:${step.fqn}`), + emit, + ); /** * Lowers `current` against the local target and plans and applies it in - * this process. The plan's scope, and the state lease it holds, close - * before this returns. + * this process, as the step `id`. The plan's scope, and the state lease it + * holds, close before this returns, and so does every step it opened. */ const converge = async ( current: PipelineResult, signal: AbortSignal | undefined, + id: string, ): Promise => { try { claimDevEnvironmentName(devDir, environment); } catch (error) { - throw toStructured('DEV.STATE_WRITE_FAILED', error); + throw toStructured('DEV.ENVIRONMENT_WRITE_FAILED', error); } const root = current.graph.root.node; if (root.kind !== 'module') { throw new InternalError('runPipeline admitted a root that is not a module'); } + steps.start({ name: 'converge', id }); + const outcome = await lowerAndConverge(current, root, signal, id); + steps.closeOpen(outcome.kind === 'applied' ? 'ok' : 'failed'); + return outcome; + }; + + const lowerAndConverge = async ( + current: PipelineResult, + root: ModuleNode, + signal: AbortSignal | undefined, + id: string, + ): Promise => { try { const stack = appConverge.lower(root, config, { name: current.name, @@ -392,7 +409,17 @@ export async function executeDev( cwd, env: {}, signal, - onEvent: reportConverge, + onEvent: (event) => + reportConvergeEvent( + steps, + event, + redact, + (resource): DevStep => ({ + name: 'apply-resource', + converge: id, + ...resource, + }), + ), }); } catch (error) { return thrownOutcome(error); @@ -446,28 +473,32 @@ export async function executeDev( // failure result, not a rejection out of dev(). let first: ConvergeOutcome; try { - first = await converge(pipeline, input.signal); + first = await converge(pipeline, input.signal, 'dev.converge'); } catch (error) { await stopStartedServices(); if (CliStructuredError.is(error)) return notOk(error); throw error; } + if (first.kind === 'interrupted') { + await stopStartedServices(); + return notOk( + new CliStructuredError('DEV.INTERRUPTED', 'dev was interrupted while it converged.', { + why: 'It stopped while Alchemy planned or applied the app. Resources it had already changed stay as they are.', + fix: 'Run dev again to finish.', + }), + ); + } if (first.kind !== 'applied') { await stopStartedServices(); - const text = convergeFailureText(first, redact); + const failure = convergeFailure(first, redact); return notOk( - text === undefined - ? new CliStructuredError('DEV.INTERRUPTED', 'dev was interrupted while it converged.', { - why: 'It stopped while Alchemy planned or applied the app. Resources it had already changed stay as they are.', - fix: 'Run dev again to finish.', - }) - : new CliStructuredError( - 'DEV.CONVERGE_FAILED', - `The app could not be converged.\n${text}`, - { - meta: { reasons: failureReasonsOf(first, redact) }, - }, - ), + failure.code === 'DEV.CONVERGE_FAILED' + ? new CliStructuredError( + failure.code, + `The app could not be converged.\n${failure.message}`, + { meta: { reasons: failure.reasons } }, + ) + : new CliStructuredError(failure.code, failure.message, REFUSAL_HELP[failure.code]), ); } @@ -541,16 +572,22 @@ export async function executeDev( // Watch loop until the session is stopped: rebuild → re-assemble → // re-converge; a converge failure keeps the running app and keeps watching. - // Authoring modules are watched too: the app was imported once, in this - // process, so an edit to one cannot reach the converge, and the session - // reports it and pauses instead (ADR-0064). + // The app definition files are watched too: the app was imported once, in + // this process, so an edit to one cannot reach the converge, and the + // session reports it and pauses instead (ADR-0064). const bundleWatch = watchTargetsFrom(pipeline.assembled.bundles); - const definitionTargets = authoringModuleTargets(pipeline.entryModule.path, pipeline.graph); - const targets = [...bundleWatch.targets, ...definitionTargets]; + const definitionFiles = [...startedDefinitions.keys()]; + const targets = [ + ...bundleWatch.targets, + ...definitionFiles.map((file) => ({ address: pipeline.graph.root.id, paths: [file] })), + ]; for (const address of bundleWatch.unwatchable) { emit({ kind: 'unwatchable', address }); } + let stopping = false; + let rebuilds = 0; + let converging: Promise | undefined; const watchDeps: PipelineDeps = { runAssembler: deps.runAssembler, devServers: true, @@ -561,7 +598,6 @@ export async function executeDev( emit({ kind: 'watch-error', message: failureMessage(error) }); const configFile = pipeline.configSource.file; const startedConfig = contentHash(configFile); - const definitionFiles = [...startedDefinitions.keys()]; let configChanged = false; let valuesChanged = false; let definitionChanged = false; @@ -601,23 +637,23 @@ export async function executeDev( input.config, watchDeps, ); + if (stopping) return; if (!samePools(pipeline.pools, rePipeline.pools)) { if (!valuesChanged) emit({ kind: 'values-changed' }); valuesChanged = true; return; } - const outcome = await converge(rePipeline, convergeSignal); + rebuilds += 1; + converging = converge(rePipeline, convergeSignal, `dev.rebuild.${String(rebuilds)}`); + const outcome = await converging.finally(() => { + converging = undefined; + }); + if (outcome.kind === 'interrupted') return; if (outcome.kind !== 'applied') { - const message = convergeFailureText(outcome, redact); - if (message !== undefined) { - emit({ - kind: 'converge-failed', - message, - reasons: failureReasonsOf(outcome, redact), - }); - } + emit({ kind: 'converge-failed', ...convergeFailure(outcome, redact) }); return; } + if (stopping) return; // A converge can move what a dev server's environment holds (a // dependency's port, a rotated key); the emulator restarts it then. await serveDevServers( @@ -638,7 +674,6 @@ export async function executeDev( // directory, which the previous rebuild's converge may still be reading, // and one process runs one converge at a time. Changes during a rebuild // coalesce into one more after it. - let stopping = false; let rebuilding: Promise | undefined; let changedDuringRebuild = false; const rebuildUntilSettled = async (): Promise => { @@ -681,8 +716,9 @@ export async function executeDev( startedConfigWatch.stop(); void (async () => { // A converge still running is interrupted above; its scope closes, - // releasing the state lease, before the services stop. - await rebuilding; + // releasing the state lease, before the services stop. A rebuild + // still in its pipeline returns before it converges. + await converging?.catch(() => undefined); // A service that refuses to stop is surfaced, not swallowed — // teardown continues, `stopped` still fires, `closed` still settles. for (const attachment of attachments) { diff --git a/packages/0-framework/3-tooling/cli/src/operations/steps.ts b/packages/0-framework/3-tooling/cli/src/operations/steps.ts new file mode 100644 index 000000000..25658ad23 --- /dev/null +++ b/packages/0-framework/3-tooling/cli/src/operations/steps.ts @@ -0,0 +1,122 @@ +/** + * Step events for the operations that converge (`deploy` and `dev`): a + * tracker that times each step and closes what is left open, and the one + * mapping from Alchemy's converge events to resource steps both share. + */ +import type { ConvergeEvent } from '../converge-in-process.ts'; +import { withComposerAdvice } from '../deployment-summary.ts'; + +/** A step event, over the operation's own step type. */ +export type StepEvent = + | { readonly kind: 'step-started'; readonly step: Step } + | { + readonly kind: 'message'; + readonly severity: 'info' | 'warn' | 'verbose'; + /** A line Alchemy logged while it planned or applied. */ + readonly text: string; + } + | { + readonly kind: 'step-finished'; + readonly step: Step; + readonly outcome: 'ok' | 'failed'; + readonly durationMs: number; + /** What the step measured, as its build adapter or deploy target reported it. The fields vary by adapter and target. */ + readonly data?: Readonly> | undefined; + }; + +/** One resource Alchemy is touching, as a converge event names it. */ +export interface ResourceStatus { + readonly fqn: string; + readonly resourceType: string; + readonly status: string; +} + +/** + * Times each step for the host's `onEvent`. Steps are matched by `keyOf`, so + * a step can finish under a newer status than it started with. A host + * `onEvent` that throws is the host's bug and cannot fail the operation. + */ +export function stepTracker( + keyOf: (step: Step) => string, + onEvent: ((event: StepEvent) => void) | undefined, +) { + const open = new Map(); + const emit = (event: StepEvent): void => { + try { + onEvent?.(event); + } catch { + // Rendering is the host's; its failures are not the operation's. + } + }; + const finish = ( + step: Step, + outcome: 'ok' | 'failed', + data?: Readonly>, + ): void => { + const entry = open.get(keyOf(step)); + if (entry === undefined) return; + open.delete(keyOf(step)); + const durationMs = Math.round(performance.now() - entry.startedAt); + emit({ kind: 'step-finished', step, outcome, durationMs, data }); + }; + return { + start(step: Step): void { + open.set(keyOf(step), { step, startedAt: performance.now() }); + emit({ kind: 'step-started', step }); + }, + finish(step: Step, data?: Readonly>): void { + finish(step, 'ok', data); + }, + fail(step: Step): void { + finish(step, 'failed'); + }, + /** Closes every open step with `outcome`, innermost first. */ + closeOpen(outcome: 'ok' | 'failed'): void { + for (const { step } of [...open.values()].reverse()) finish(step, outcome); + }, + isOpen(step: Step): boolean { + return open.has(keyOf(step)); + }, + message(severity: 'info' | 'warn' | 'verbose', text: string): void { + emit({ kind: 'message', severity, text }); + }, + }; +} + +export type StepTracker = ReturnType>; + +/** One converge event as step events: each resource is a step, made by `resourceStep`; a log line is a message. */ +export function reportConvergeEvent( + steps: StepTracker, + event: ConvergeEvent, + redact: (text: string) => string, + resourceStep: (resource: ResourceStatus) => Step, +): void { + if (event.kind === 'log') { + steps.message(event.severity, redact(withComposerAdvice(event.text))); + return; + } + const step = resourceStep({ + fqn: event.fqn, + resourceType: event.resourceType, + status: event.status, + }); + switch (event.phase) { + case 'running': + if (!steps.isOpen(step)) steps.start(step); + return; + case 'done': + if (!steps.isOpen(step)) steps.start(step); + steps.finish(step); + return; + case 'failed': + if (!steps.isOpen(step)) steps.start(step); + steps.fail(step); + if (event.message !== undefined) { + steps.message('warn', redact(`${event.fqn}: ${withComposerAdvice(event.message)}`)); + } + return; + case 'pending': + return; + } +} diff --git a/skills/prisma-composer-core-concepts/SKILL.md b/skills/prisma-composer-core-concepts/SKILL.md index a8478dc68..29cdfc989 100644 --- a/skills/prisma-composer-core-concepts/SKILL.md +++ b/skills/prisma-composer-core-concepts/SKILL.md @@ -601,8 +601,9 @@ that surprise: environment variables; Composer's values win. `dev` prints ``
: runs `` in `` before the app comes up and streams the dev server's output prefixed `[
]` (other services' logs are not - streamed). Source edits are the dev server's own hot reload; an edit to the - entry or a service's authoring module needs `dev` restarted (it says so), + streamed). Source edits are the dev server's own hot reload; an edit to an + app definition file (the entry or any app file it imports) needs `dev` + restarted (it says so), and `dev` restarts a dev server only when its environment variables changed (``
: its environment changed; `` restarted``). It has no terminal (no colours or key shortcuts). The standalone build is not run @@ -655,8 +656,9 @@ that surprise: app, including one an earlier `--detach` left running (a failed redeploy has already replaced that service's process), and exits non-zero: fix the build and run `prisma dev --detach` again. -4. `dev` reads `prisma.config.ts` and loads the app's definition (the entry - and each service's authoring module) once, at start, and watches them: +4. `dev` reads `prisma.config.ts` and loads the app definition files (the + entry and every app file it imports, outside `node_modules`) once, at + start, and watches them: after an edit it says so and pauses rebuilds until you restart `dev`. 5. `dev` reads secrets only from the `secrets` adapters. Put them where a `secrets` adapter reads them: the shell by default, or `.env.dev` with `secrets: [dotenv(), environmentVariables()]`. A required secret that is unset or empty stops `dev` with an error naming it; an optional one reaches the service as absent. A changed value takes effect on the next `prisma dev`. A `.env` that only the framework loads (Next.js) does not count: Composer reads secrets before the dev server starts. 6. The emulators are machine-wide daemons shared by every `dev`, registered diff --git a/test/integration/test/local-dev-store.integration.ts b/test/integration/test/local-dev-store.integration.ts index 5ed1de1c3..ab5f68fca 100644 --- a/test/integration/test/local-dev-store.integration.ts +++ b/test/integration/test/local-dev-store.integration.ts @@ -41,6 +41,9 @@ * app-scoped and unconditionally restarts every service on the next * redeploy — S3's own pinned "a stopped service always starts on * redeploy" behavior — which would defeat this proof entirely). + * A sub-module's `module.ts` is an app definition file: an edit to it + * while dev runs is reported as needing a restart, and a later artifact + * touch converges nothing (the app is imported once per process). * 3. Postgres persistence: a row is written directly against the real * local Postgres URL (read from the postgres-main daemon's own * database listing — the daemon owns instance state, REVISED — @@ -411,6 +414,69 @@ const catalogDist = path.join(storeDir, 'modules', 'catalog', 'dist', 'server.mj const DEV_STACK_FILE = path.join(storeDir, '.prisma-composer', 'dev', 'dev', 'alchemy.run.ts'); const REAL_WATCH_TIMEOUT_MS = 30_000; +const catalogModule = path.join(storeDir, 'modules', 'catalog', 'src', 'module.ts'); + +/** The `--json` frames written after the first `from` characters of the session's log. */ +function framesAfter(session: DevSession, from: number): Record[] { + return readLog(session.logPath) + .slice(from) + .split('\n') + .flatMap((line) => { + if (!line.startsWith('{')) return []; + try { + return [JSON.parse(line) as Record]; + } catch { + return []; + } + }); +} + +/** + * Edits catalog's `module.ts`, which the store's entry imports through the + * `@store/catalog` workspace package, and waits for dev's restart notice + * naming it. Then touches catalog's built artifact and checks that no + * rebuild converges and catalog keeps its pid. Restores the file after. + */ +async function editSubModuleAndAwaitRestartNotice( + session: DevSession, + pidsBefore: Record, +): Promise { + const from = readLog(session.logPath).length; + const original = fs.readFileSync(catalogModule, 'utf8'); + try { + fs.writeFileSync(catalogModule, `${original}\n// proving-script definition edit\n`); + await waitForAsync(async () => { + const notice = framesAfter(session, from).find( + (frame) => + frame['kind'] === 'message' && + typeof frame['text'] === 'string' && + frame['text'].includes(catalogModule) && + frame['text'].includes('restart dev'), + ); + return notice === undefined ? undefined : true; + }, REAL_WATCH_TIMEOUT_MS); + + fs.appendFileSync( + catalogDist, + `\n// proving-script touch after a definition edit ${Date.now()}\n`, + ); + await new Promise((resolve) => setTimeout(resolve, 5_000)); + const rebuilt = framesAfter(session, from).some( + (frame) => + frame['kind'] === 'endpoint' || + (typeof frame['id'] === 'string' && frame['id'].startsWith('dev.rebuild.')), + ); + assert(!rebuilt, 'no rebuild converges after an app definition file changed'); + const pids = await pidsByAddress(APP_NAME); + assertEqual( + pids['store.catalog.service'], + pidsBefore['store.catalog.service'], + 'catalog.service keeps its pid after an app definition file changed', + ); + } finally { + fs.writeFileSync(catalogModule, original); + } +} /** * Touches ONLY catalog's built artifact — the same file the running @@ -531,6 +597,14 @@ async function main(): Promise { pidsBefore = pidsAfter; } + // ——— A sub-module's module.ts is an app definition file ——— + // The app is imported once, in the prisma process, so dev cannot apply an + // edit to it: it must say so and stop converging, not rebuild the old graph. + await editSubModuleAndAwaitRestartNotice(activeSession, pidsBefore); + console.log( + "[proving] PASS: an edit to a sub-module's module.ts is reported as needing a restart, and nothing converges", + ); + // ——— Criterion 3 (write half): a row through the real local Postgres URL ——— const dbUrl = await readCatalogDbUrl(); await withSql(dbUrl, async (sql) => {