From 9d6b3b531c6ccc022dcd4b25f9d9885f84364cff Mon Sep 17 00:00:00 2001 From: simpros <38320875+simpros@users.noreply.github.com> Date: Fri, 2 Oct 2026 21:11:12 +0200 Subject: [PATCH 1/2] =?UTF-8?q?Docs=20corpus:=20rewrite=20every=20publishe?= =?UTF-8?q?d=20page=20in=20its=20Di=C3=A1taxis=20mode=20(#272)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- README.md | 48 ++-- docs/adopting-a-repo.md | 138 +++++----- docs/ci-integration.md | 177 ++++++------- docs/cli-reference.md | 70 ++--- docs/getting-started.md | 115 ++++---- docs/herdr-integration.md | 27 +- docs/onboarding-prompt.md | 8 +- docs/operator-deploy.md | 532 ++++++++++++++++++++------------------ docs/previews.md | 317 +++++++++++------------ docs/site/assemble.ts | 6 +- docs/telemetry.md | 50 ++-- docs/troubleshooting.md | 15 +- llms.txt | 6 +- 13 files changed, 756 insertions(+), 753 deletions(-) diff --git a/README.md b/README.md index b37146ac..1859ad82 100644 --- a/README.md +++ b/README.md @@ -5,47 +5,47 @@ # sprout Every pull request gets its own preview: an isolated database on one shared -instance plus an optional live app, for teams that review running code instead -of diffs. Self-hosted, no platform lock-in — everything is torn down when the -PR closes. +instance plus an optional live app, for teams that review running code +instead of diffs. Self-hosted, no platform lock-in — everything is torn down +when the PR closes. ```text PR opened / push PR closed - │ │ - ▼ ▼ - sprout deploy ──► gateway ──► sprout teardown - │ │ - ├─ CREATE DATABASE ├─ stop containers - ├─ start app (+ Traefik) └─ DROP DATABASE - └─ optional seed image - │ - ▼ - preview URL for this PR + │ │ + ▼ ▼ + sprout deploy ──► gateway ──► sprout teardown + │ │ + ├─ CREATE DATABASE ├─ stop containers + ├─ start app (+ Traefik) └─ DROP DATABASE + └─ optional seed image + │ + ▼ + preview URL for this PR ``` ## Adopt in three files -`.sprout.yaml`, a CI include, and the `Dockerfile` your app already ships — -plus two CI variables from your operator. The published gateway image +`.sprout.yaml`, a CI include, and the `Dockerfile` the app already ships — +plus two CI variables from the operator. The published gateway image reports anonymous [install telemetry](docs/telemetry.md) by default -(`SPROUT_TELEMETRY=off` stops it). Humans: +(`SPROUT_TELEMETRY=off` stops it). Humans start at [Getting started](docs/getting-started.md). Agents: paste the -[Onboarding prompt](docs/onboarding-prompt.md) into your coding harness and it -wires the repo for you. Live pitch: +[Onboarding prompt](docs/onboarding-prompt.md) into a coding harness and it +wires the repo. Live pitch: [the marketing page](https://simpros.github.io/sprout/). ## Docs -- **[Public docs](https://simpros.github.io/sprout/docs/index.html)** — concept, guides, reference +- **[Public docs](https://simpros.github.io/sprout/docs/index.html)** — guides and reference ([source](docs/site/index.html) · [llms.txt](llms.txt)) -- [Getting started](docs/getting-started.md) — first preview -- [Adopting a repo](docs/adopting-a-repo.md) — `.sprout.yaml` manifest reference +- [Getting started](docs/getting-started.md) — first preview in one sitting +- [Adopting a repo](docs/adopting-a-repo.md) — write `.sprout.yaml`, migrate at startup - [CI integration](docs/ci-integration.md) — GitHub / GitLab wiring, reset - [Previews](docs/previews.md) — databases, seeding, services, mail - [Operator deploy](docs/operator-deploy.md) — compose stack, env, Traefik -- [CLI reference](docs/cli-reference.md) — every command -- [Troubleshooting](docs/troubleshooting.md) — error catalogue -- [Telemetry](docs/telemetry.md) — anonymous install reporting, off switch +- [CLI reference](docs/cli-reference.md) — every command and flag +- [Troubleshooting](docs/troubleshooting.md) — symptom → fix catalogue +- [Telemetry](docs/telemetry.md) — what is sent, what it costs, off switch - [Onboarding prompt](docs/onboarding-prompt.md) — paste into a coding harness - [`examples/adopting-repo/README.md`](examples/adopting-repo/README.md) — copy-paste starter - [`CONTEXT.md`](CONTEXT.md) — domain vocabulary diff --git a/docs/adopting-a-repo.md b/docs/adopting-a-repo.md index 76d218d3..b6c5c739 100644 --- a/docs/adopting-a-repo.md +++ b/docs/adopting-a-repo.md @@ -1,28 +1,25 @@ # Adopting a repo -`.sprout.yaml` is the config-as-code file in an adopting repo: slug, preview -hostname template, optional health-check settings, optional companion service -routing metadata, optional `build` / `seed` image blocks, optional `db` -provider block, and computed env values. This page is the contract. +Write the `.sprout.yaml` config-as-code file at the repo root and the app +entrypoint that runs inside the preview. Each section below is one task. -- First preview? Start in [Getting started](getting-started.md). -- CI wiring for your forge? See [CI integration](ci-integration.md). -- Lifecycle (databases, seeding, services, mail)? See [Previews](previews.md). -- Commands and debugging? See [CLI reference](cli-reference.md). +- First preview: [Getting started](getting-started.md). +- CI wiring for the forge: [CI integration](ci-integration.md). +- Lifecycle (databases, seeding, services, mail): [Previews](previews.md). +- Commands and debugging: [CLI reference](cli-reference.md). Copy-paste app files live in [`examples/adopting-repo/README.md`](../examples/adopting-repo/README.md). ## Manifest keys (`.sprout.yaml`) -The CLI reads this file locally and sends parsed values to the gateway. -Unknown keys are rejected (`unknown key: `). This table is the -contract; the notes directly below it (connection env, value grammar, merge -order) are part of the contract. Seeding order lives in +Look up any key here. The CLI reads this file locally and sends parsed +values to the gateway. Unknown keys are rejected (`unknown key: `). +The table plus the three subsections directly below it (connection env, +value grammar, merge order) are the contract. Seeding order lives in [Previews](previews.md#seed-run-order-and-resume), service merge rules in [Previews](previews.md#service-images-merge-leave-clear-lifecycle), mail in -[Previews](previews.md#email-from-a-preview). Test -pointers live in [CLI reference](cli-reference.md#test-coverage-maintainers). +[Previews](previews.md#email-from-a-preview). | Key | Required | Default | Purpose | |---|---|---|---| @@ -61,8 +58,9 @@ pointers live in [CLI reference](cli-reference.md#test-coverage-maintainers). ### Connection env: names, roles, reservation, port -The gateway injects these connection variables into preview app, service, -and seed containers. Which set you get follows `db.provider`: +Read this before naming a connection variable. The gateway injects these +connection variables into preview app, service, and seed containers. Which +set arrives follows `db.provider`: Postgres (`db.provider: postgres`, the default): @@ -72,15 +70,12 @@ PGAPPUSER PGAPPPASSWORD (dual only — see db.roles below) ``` `db.roles` selects the Postgres credential axis (`single` | `dual`, -Postgres only; rejected on `sqlite` / `none`). `dual` provisions the -restricted companion role and injects `PGAPPUSER` / `PGAPPPASSWORD` -exactly as before; `single` provisions no companion and injects neither -name anywhere. The default is derived: `dual` when `preview.env` -remaps `PGAPPUSER` or `PGAPPPASSWORD`, otherwise `single` — an adopter -who never mentions the companion never gets one, and existing remap -users keep working with zero config change. An explicit `db.roles` -wins over the derivation, and explicit `db.roles: single` plus a -companion remap fails fast at manifest parse and at the deploy route +Postgres only; rejected on `sqlite` / `none`). The default is derived: +`dual` when `preview.env` remaps `PGAPPUSER` or `PGAPPPASSWORD`, +otherwise `single` — a repo that never mentions the companion gets none, +and existing remap users keep working with no config change. An explicit +`db.roles` wins over the derivation, and explicit `db.roles: single` plus +a companion remap fails fast at manifest parse and at the deploy route (`preview.env. conflicts with db.roles single`). SQLite (`db.provider: sqlite`): @@ -94,12 +89,12 @@ at all — every `preview.env` entry is rejected at manifest parse (`preview.env.PGHOST requires db.provider postgres`) and at the gateway deploy route, and a `seed:` block is rejected the same way (`seed requires db.provider postgres or sqlite (db.provider is none)`). -See [Previews](previews.md#no-database-previews) for the lifecycle. +See [Previews](previews.md#no-database-previews). No `PG*` keys are injected for a SQLite preview, and no `DATABASE_URL` for a Postgres one — `preview.env` entries for the other backend fail at manifest parse (`preview.env.PGHOST requires db.provider postgres`). -See [Previews](previews.md#sqlite-previews) for the volume behaviour. +See [Previews](previews.md#sqlite-previews). - **Owner** (`PGUSER` / `PGPASSWORD`): the static preview login (`SPROUT_PG_USER`). Owns each preview database — use this for migrations. @@ -135,23 +130,24 @@ example, and the opt-out live in exactly one place — ### Env value grammar -`preview.app_env` / `seed.env` values are a plain string, `{ generate: -stable_per_pr }`, or `{ required: true }`. Strings may interpolate -`{hostname}`, `{pr_id}`, `{commit_sha}` (`{commit_sha}` follows the forge -SHA, `CI_COMMIT_SHA` on GitLab / `GITHUB_SHA` on GitHub). `generate` -derives a per-MR secret (HMAC of repo, MR, key, keyed by the deploy token -— keep the token stable for the MR lifetime). `required` must be supplied -by CI; missing keys fail before the gateway call naming the key. +Write `preview.app_env` / `seed.env` values in one of three forms: a plain +string, `{ generate: stable_per_pr }`, or `{ required: true }`. Strings +may interpolate `{hostname}`, `{pr_id}`, `{commit_sha}` (`{commit_sha}` +follows the forge SHA, `CI_COMMIT_SHA` on GitLab / `GITHUB_SHA` on +GitHub). `generate` derives a per-MR secret (HMAC of repo, MR, key, keyed +by the deploy token — keep the token stable for the MR lifetime). +`required` must be supplied by CI; missing keys fail before the gateway +call naming the key. ### Env merge order -App: yaml first, then `SPROUT_APP_ENV` / each `--app-env-file` in order, -then `--app-env` flags (later wins per key). Seed: yaml first, then -`SPROUT_SEED_ENV` / each `--seed-env-file` in order, then `--seed-env` -flags. Seed args: yaml `seed.args` first, then `--seed-arg` flags -appended. +Layer env so later wins per key. App: yaml first, then `SPROUT_APP_ENV` / +each `--app-env-file` in order, then `--app-env` flags. Seed: yaml first, +then `SPROUT_SEED_ENV` / each `--seed-env-file` in order, then +`--seed-env` flags. Seed args: yaml `seed.args` first, then `--seed-arg` +flags appended. -Minimal app-only manifest (defaults apply): +Write a minimal app-only manifest (defaults apply): ```yaml slug: myapp @@ -159,53 +155,52 @@ preview: hostname: "pr-{pr_id}.myapp.preview.example.com" ``` -Seeded manifest (as in the quickstart): add `health:` + `seed:`. `seed: {}` -alone enables seeding with the conventional `Dockerfile.seed`. +Write a seeded manifest (as in the quickstart): add `health:` + `seed:`. +`seed: {}` alone enables seeding with the conventional `Dockerfile.seed`. ## App image: migrate at startup -Contract above is normative; this section is entrypoint examples only. -Connection names, owner/companion roles, and the port rule live in -Connection env — snippets below assume the default `PG*` map (`dual` -adds `PGAPP*`; `single` omits them). - -Your app image must: +Make the app entrypoint do these three things in order. The connection +names, owner/companion roles, and the port rule live in Connection env — +snippets below assume the default `PG*` map (`dual` adds `PGAPP*`; +`single` omits them): 1. Wait until Postgres accepts connections. 2. Run migrations as the **owner** against the injected database name (default `PGDATABASE`); `GRANT` to the companion role for RLS instead of creating roles. 3. Start the web server, connecting runtime queries as `PGAPPUSER` when - you need RLS. + RLS scoping is needed. -There is **no mandatory wrapper image** from sprout. Copy an entrypoint -that fits your stack. +There is no mandatory wrapper image from sprout. Copy an entrypoint that +fits the stack. Migrations must be **idempotent** — synchronize +re-deploys keep the same database and re-run migrate on every container +start. ### Dual-role (RLS) previews -Set `db.roles: dual` for product databases that use a privileged owner -+ restricted RLS role — previews then work without cluster `CREATEROLE` -on the preview login (without the flag there is no companion role and -no `PGAPP*` names to read): +Scope product databases that use a privileged owner + restricted RLS role. +Set `db.roles: dual` so previews work without cluster `CREATEROLE` on the +preview login (without it there is no companion role and no `PGAPP*` names +to read): 1. Migrate with `PGUSER` / `PGPASSWORD` (owner). 2. `GRANT` the needed table/sequence privileges to the role in `PGAPPUSER` (and enable RLS / policies as in production). 3. Open the app pool with `PGAPPUSER` / `PGAPPPASSWORD` (remap via - `preview.env` if that matches your product env names). + `preview.env` when the product env names differ). ### Extra app env (non-connection) -Example only — grammar, merge order, and the gateway reservation rule -live above. Adopters often need runtime env beyond the connection fields -(`BETTER_AUTH_SECRET`, app URLs, trusted origins, etc.): pass those as -`preview.app_env` / `seed.env` in `.sprout.yaml`, a masked file-type -`SPROUT_APP_ENV` / `SPROUT_SEED_ENV` dotenv blob, repeatable -`--app-env-file` / `--seed-env-file`, or repeatable `--app-env KEY=VALUE` / -`--seed-env KEY=VALUE`. Forge File-var wiring lives in -[`templates/README.md`](../templates/README.md); placeholders expand in -every layer. File-type CI variables do NOT survive component `inputs:` -expansion — map file-type blobs at job runtime via `variables:` +Pass runtime env beyond the connection fields (`BETTER_AUTH_SECRET`, app +URLs, trusted origins) as `preview.app_env` / `seed.env` in +`.sprout.yaml`, a masked file-type `SPROUT_APP_ENV` / `SPROUT_SEED_ENV` +dotenv blob, repeatable `--app-env-file` / `--seed-env-file`, or +repeatable `--app-env KEY=VALUE` / `--seed-env KEY=VALUE`. Grammar, merge +order, and the gateway reservation rule live above. Forge File-var wiring +lives in [`templates/README.md`](../templates/README.md); placeholders +expand in every layer. File-type CI variables do NOT survive component +`inputs:` expansion — map file-type blobs at job runtime via `variables:` (`SPROUT_APP_ENV: $MY_ENV_FILE`), never as `app_env_file:` inputs. Example: @@ -225,7 +220,7 @@ preview: ### Shell entrypoint (any runtime) -See [`examples/adopting-repo/docker-entrypoint.sh`](../examples/adopting-repo/docker-entrypoint.sh) +Copy [`examples/adopting-repo/docker-entrypoint.sh`](../examples/adopting-repo/docker-entrypoint.sh) (default `PG*` names): ```bash @@ -246,12 +241,11 @@ bun run db:migrate exec bun run start ``` -Migrations must be **idempotent** — synchronize re-deploys keep the same -database and re-run migrate on every container start. - ## Deploy token setup -One-time per adopting repo (operator or lead dev with admin token): +Mint one deploy token per adopting repo (operator or lead dev with the +admin token), then store the returned token as the repo's `SPROUT_TOKEN` +secret: ```bash export SPROUT_URL=https://sprout.example.com @@ -261,8 +255,6 @@ sprout admin token create \ --repo "https://github.com/${GITHUB_REPOSITORY}" ``` -Add the returned token to the repo's `SPROUT_TOKEN` secret. - Reviewers may see brief 502 responses while the app migrates and starts — Traefik routes exist before the app is healthy. diff --git a/docs/ci-integration.md b/docs/ci-integration.md index b0662a3b..bbd4a950 100644 --- a/docs/ci-integration.md +++ b/docs/ci-integration.md @@ -1,17 +1,18 @@ # CI integration -One preview per merge / pull request, driven by CI — no forge webhooks. +Drive one preview per merge / pull request from CI — no forge webhooks. GitLab uses the published `preview` component; GitHub uses the reusable `preview` workflow. Both are thin bootstrappers over `sprout ci preview` / -`sprout ci teardown`: all preview logic lives in the CLI. +`sprout ci teardown`: all preview logic lives in the CLI. Each section +below is one task. -- First preview? See [Getting started](getting-started.md). -- Manifest keys? See [Adopting a repo](adopting-a-repo.md). -- Commands reference? See [CLI reference](cli-reference.md). +- First preview: [Getting started](getting-started.md). +- Manifest keys: [Adopting a repo](adopting-a-repo.md). +- Command flags: [CLI reference](cli-reference.md). ## GitLab component -`.sprout.yaml` at the repo root (see +Wire `.sprout.yaml` at the repo root (see [Adopting a repo](adopting-a-repo.md)), two required masked CI variables (plus optional `GITLAB_TOKEN` for MR notes), and one include. No adopter shell scripts: @@ -23,68 +24,70 @@ include: inputs: { stage: deploy } ``` -Replace `/sprout-ci` with the component project path on your GitLab -instance and `v0.8.3` with the sprout release you adopt. Where the component -project is unavailable on your instance, use the `include: remote` fallback -documented in [`templates/README.md`](../templates/README.md) -(remote includes must set `sprout_version` explicitly to the tag in the URL). +Replace `/sprout-ci` with the component project path on the +instance and `v0.8.3` with the adopted sprout release. Where the component +project is unavailable, use the `include: remote` fallback documented in +[`templates/README.md`](../templates/README.md) (remote includes must set +`sprout_version` explicitly to the tag in the URL). -What you get: +What the three jobs do: -- `sprout-preview` job (merge-request pipelines only): installs the pinned, +- `sprout-preview` (merge-request pipelines only): installs the pinned, checksum-verified `sprout` binary (version = component version), builds + pushes the app image (`CI_REGISTRY_IMAGE:`) and — when `.sprout.yaml` - sets `seed` — the seed image (same repository; commit-scoped - `-seed` tag rebuilt on every run, or — with explicit `seed.inputs` — - a `seed-` tag content-addressed over those inputs, where an - authenticated `docker manifest inspect` skips the build + push when that - tag already exists and logs `seed image reused: `), deploys, writes `PREVIEW_URL=` to the - `sprout-preview.env` dotenv artifact that feeds `environment:url`, and - best-effort posts/updates the MR note with the preview URL (forge failures - only warn with the forge's error body, never the token). The URL comes from the - CLI's output only — never reconstruct the hostname in CI. -- `sprout-stop-preview` job (`on_stop`, Stop button / MR close / merge / + sets `seed` — the seed image (commit-scoped `-seed` tag rebuilt on + every run, or — with explicit `seed.inputs` — a `seed-` tag + content-addressed over those inputs, where an authenticated + `docker manifest inspect` skips the build + push when that tag already + exists and logs `seed image reused: `), deploys, writes `PREVIEW_URL=` + to the `sprout-preview.env` dotenv artifact that feeds `environment:url`, + and best-effort posts/updates the MR note with the preview URL (forge + failures only warn with the forge's error body, never the token). The URL + comes from the CLI's output only — never reconstruct the hostname in CI. +- `sprout-stop-preview` (`on_stop`, Stop button / MR close / merge / `auto_stop_in` expiry): `sprout ci teardown` (idempotent — exit 0 when already gone). -- `sprout-reset` job (manual, instant): `sprout ci reset` — wipe the preview +- `sprout-reset` (manual, instant): `sprout ci reset` — wipe the preview database and redeploy + seed from scratch (data wiped). Press **Run** on it - inside the MR pipeline when you want the reset *now* (GitLab has no - description-edit → pipeline trigger, and a fresh - "Run pipeline" arrives as `CI_PIPELINE_SOURCE=web`, which `sprout ci` - refuses — so the instant path lives inside the existing MR pipeline). + inside the MR pipeline for the reset *now* (GitLab has no + description-edit → pipeline trigger, and a fresh "Run pipeline" arrives as + `CI_PIPELINE_SOURCE=web`, which `sprout ci` refuses — so the instant path + lives inside the existing MR pipeline). - Prerequisites owned by the component: dind service, registry login, `apk` packages. Adopters declare no packages and no Docker setup. Prerequisites on the GitLab side: merge-request pipelines -(`CI_PIPELINE_SOURCE=merge_request_event`; branch pipelines are refused with -a named error) and a runner that can run privileged `docker:dind`. The -operator must have deployed the [operator compose stack](operator-deploy.md) and -minted the deploy token. Pull credentials on the gateway follow the deploy -`app_image` host (`SPROUT_REGISTRY_AUTHS_JSON` per host, empty = anonymous). +(`CI_PIPELINE_SOURCE=merge_request_event`; branch pipelines are refused +with a named error) and a runner that can run privileged `docker:dind`. +The operator must have deployed the [operator compose stack](operator-deploy.md) +and minted the deploy token. Pull credentials on the gateway follow the +deploy `app_image` host (`SPROUT_REGISTRY_AUTHS_JSON` per host, empty = +anonymous). ### Component → CLI ownership The component YAML above calls exactly three `sprout ci` subcommands — -everything else is manual (run from a merge-request pipeline or laptop): +everything else is manual (run from a merge-request pipeline or laptop). +Flag details live in [CLI reference](cli-reference.md#ci-commands). | Component job | CLI call | Owns | |---|---|---| -| `sprout-preview` | `sprout ci preview --tail … --dotenv-file … [--app-env-file …] [--seed-env-file …] [--reseed]` | Install check aside, the CLI builds + pushes the app image and — when `.sprout.yaml` sets `seed` — the seed image (always rebuilt, unless explicit `seed.inputs` opt into content-addressed reuse: an existing tag skips the rebuild, reuse is logged, a failed check rebuilds), deploys (with `--reseed` when the flag is passed), writes `PREVIEW_URL=` to the dotenv artifact, dumps the gateway log tail on failure, and posts/updates the MR note (best-effort). | +| `sprout-preview` | `sprout ci preview --tail … --dotenv-file … [--app-env-file …] [--seed-env-file …] [--reseed]` | Builds + pushes the app image and — when `.sprout.yaml` sets `seed` — the seed image (always rebuilt, unless explicit `seed.inputs` opt into content-addressed reuse: an existing tag skips the rebuild, reuse is logged, a failed check rebuilds), deploys (with `--reseed` when the flag is passed), writes `PREVIEW_URL=` to the dotenv artifact, dumps the gateway log tail on failure, and posts/updates the MR note (best-effort). | | `sprout-stop-preview` | `sprout ci teardown` (no flags) | Idempotent teardown; rewrites the MR note in place ("preview was removed"). No Docker daemon, no registry login on this path. | | `sprout-reset` | `sprout ci reset --tail … --dotenv-file … [--app-env-file …] [--seed-env-file …]` | Wipe the preview database and redeploy + seed from scratch (data wiped). No rebuild: reuses the already pushed images for the commit, so no Docker daemon and no registry login on this path. | Manual helpers (never called by the component): `sprout ci reseed -s …` (re-run the seed against the existing database) and `sprout ci logs -[--tail N]` (container logs through the gateway). Full flags in -[CLI reference](cli-reference.md#ci-commands). +[--tail N]` (container logs through the gateway). ### Component inputs (`templates/preview.yml`) -Full prose reference: [`templates/README.md`](../templates/README.md). -Contract tests: `templates/preview.test.ts` (inputs are exactly this set — -no Dockerfile guards, no extra-args hatch; self-contained, no -`spec:include`, no global keywords). There are no service-related -component inputs — companions go through `sprout ci preview --service`. +Set these inputs on the include. Full prose reference: +[`templates/README.md`](../templates/README.md). Contract tests: +`templates/preview.test.ts` (inputs are exactly this set — no Dockerfile +guards, no extra-args hatch; self-contained, no `spec:include`, no global +keywords). There are no service-related component inputs — companions go +through `sprout ci preview --service`. | Input | Default | Purpose | |---|---|---| @@ -108,11 +111,12 @@ component inputs — companions go through `sprout ci preview --service`. | `SPROUT_SEED_ENV` | File | yes | Optional seed secrets, same shape | | `GITLAB_TOKEN` | Variable | yes | Optional MR-note token with note-write. When set, the CLI posts/updates the MR note with it (`PRIVATE-TOKEN`); otherwise it falls back to `CI_JOB_TOKEN` (`JOB-TOKEN`), which can read but not create notes on some instances (401) | -Secrets never appear in job logs, CLI output, or the MR note (the CLI prints only `preview_url=`). +Secrets never appear in job logs, CLI output, or the MR note (the CLI prints +only `preview_url=`). ## GitHub Actions -One caller workflow + secrets — parity with the GitLab component. The +Add one caller workflow + secrets — parity with the GitLab component. The **canonical** caller is [`examples/adopting-repo/.github/workflows/sprout.yml`](../examples/adopting-repo/.github/workflows/sprout.yml) — copy it rather than pasting fragments from this guide: @@ -137,9 +141,8 @@ jobs: SPROUT_APP_ENV: ${{ secrets.SPROUT_APP_ENV }} ``` -What you get (owned by the reusable workflow -`.github/workflows/preview.yml`, a thin -bootstrapper over `sprout ci preview` / `sprout ci teardown`): +What the reusable workflow (`.github/workflows/preview.yml`, a thin +bootstrapper over `sprout ci preview` / `sprout ci teardown`) does: - Preview on `opened` / `synchronize` / `reopened`: installs the pinned, checksum-verified `sprout` binary, builds + pushes images @@ -153,10 +156,9 @@ bootstrapper over `sprout ci preview` / `sprout ci teardown`): or laptop with the same secrets — no ticked box, no marker token. - Per-PR serial runs via a `sprout-preview-` concurrency group. -Caller permissions: a reusable workflow cannot elevate permissions, so the -caller must grant `contents: read` (checkout), `pull-requests: write` (PR -comment), and `packages: write` (image push to -`ghcr.io`). +The caller must grant `contents: read` (checkout), `pull-requests: write` +(PR comment), and `packages: write` (image push to `ghcr.io`) — a reusable +workflow cannot elevate permissions. Secrets (repo settings): @@ -168,11 +170,11 @@ Secrets (repo settings): | `SPROUT_SEED_ENV` | no | Seed secrets, same shape | No file-type variables on GitHub: the blobs above are written to -`$RUNNER_TEMP` and read automatically by the CLI. When 48 KB is not enough, -commit a repo-relative dotenv file and pass it via the `app_env_file` / -`seed_env_file` inputs instead of the secret — same escape hatch as the -GitLab `variables:` mapping, without the config-time expansion trap -(inputs are plain paths, never variable references). +`$RUNNER_TEMP` and read automatically by the CLI. When 48 KB is not +enough, commit a repo-relative dotenv file and pass it via the +`app_env_file` / `seed_env_file` inputs instead of the secret — same +escape hatch as the GitLab `variables:` mapping, without the config-time +expansion trap (inputs are plain paths, never variable references). Deliberate differences from the GitLab component: there is no `on_stop` / `auto_stop_in` equivalent — teardown on `closed` plus the gateway sweep is @@ -192,7 +194,7 @@ Pick the asset that matches the host libc (names are honest): | `sprout-linux-x64-musl` | musl | Alpine runners; install `libstdc++` | ```bash -TAG=v0.8.3 # pin ≥ the release that ships glibc `sprout-linux-x64` +TAG=v0.8.3 # glibc hosts curl -fsSL -o /usr/local/bin/sprout \ @@ -208,7 +210,8 @@ chmod +x /usr/local/bin/sprout sprout --version ``` -CLI environment for hand-rolled jobs (the reusable workflow sets this itself): +CLI environment for hand-rolled jobs (the reusable workflow sets this +itself): ```yaml env: @@ -218,22 +221,23 @@ env: ## Reset a preview -One command wipes the preview database and redeploys + seeds from scratch: +Wipe the preview database and redeploy + seed from scratch with one +command: ```bash sprout ci reset ``` It tears down this MR's preview (containers + database + any -`preview.volumes` data volumes), then deploys again -with the already pushed images for the commit — no rebuild — re-running -migrations and the seed behind the health gate, then printing `preview_url=` -like `sprout ci preview`. The MR/PR note gains a `Reset: at ` +`preview.volumes` data volumes), then deploys again with the already +pushed images for the commit — no rebuild — re-running migrations and the +seed behind the health gate, then printing `preview_url=` like +`sprout ci preview`. The MR/PR note gains a `Reset: at ` line; the note is never duplicated. -Wipe vs re-seed in one sentence each: `sprout ci reset` wipes the database -and redeploys from scratch (data wiped), while `--reseed` / `sprout ci -reseed -s …` re-runs the seed against the existing database (data kept). +Wipe vs re-seed: `sprout ci reset` wipes the database and redeploys from +scratch (data wiped), while `--reseed` / `sprout ci reseed -s …` re-runs +the seed against the existing database (data kept). If the reset leaves the preview unhealthy, pull container logs through the gateway (`sprout ci logs [--tail N]`, same output as `sprout logs`) and fix @@ -262,39 +266,36 @@ hand-rolled job or laptop with the same secrets. ## Migration from a hand-rolled script -If your `.gitlab-ci.yml` currently installs the CLI, builds/pushes images, -and calls `sprout deploy` by hand, replace the whole job with the component. -Hand-rolled setups typically carry `scripts/ci/sprout-preview.sh` -plus a seed-image script, `SPROUT_VERSION` / `SPROUT_SHA256` project -variables, a custom CLI installer block, and flag-by-flag -`--app-env` assembly. Concretely, delete: +Replace a hand-rolled `.gitlab-ci.yml` job (manual CLI install, +build/push, `sprout deploy` by hand) with the component. Concretely, +delete: 1. The `curl` install block (version pin, asset selection, `chmod`, `libstdc++` handling) — the component installs the pinned, - checksum-verified binary matching the component version. This retires the - custom installer and the `SPROUT_VERSION` / `SPROUT_SHA256` project + checksum-verified binary matching the component version. This retires + the custom installer and the `SPROUT_VERSION` / `SPROUT_SHA256` project variables: the version is now the component version (`sprout_version` input, pinned automatically on component includes). -2. The `docker build` / `docker push` steps and registry-login script — the - component owns the dind service and login; `sprout ci preview` builds and - pushes the app image and, when `.sprout.yaml` configures `seed`, the seed - image. This retires `scripts/ci/sprout-preview.sh` and the seed-image - script (image coordinates move into `.sprout.yaml` `build` / `seed` - blocks). -3. The `sprout deploy -i … -s …` invocation, `preview_url=` scraping, dotenv - writing, and `sprout teardown` script — the component calls +2. The `docker build` / `docker push` steps and registry-login script — + the component owns the dind service and login; `sprout ci preview` + builds and pushes the app image and, when `.sprout.yaml` configures + `seed`, the seed image. This retires `scripts/ci/sprout-preview.sh` and + the seed-image script (image coordinates move into `.sprout.yaml` + `build` / `seed` blocks). +3. The `sprout deploy -i … -s …` invocation, `preview_url=` scraping, + dotenv writing, and `sprout teardown` script — the component calls `sprout ci preview` / `sprout ci teardown`, writes the dotenv artifact, and posts the MR note. Flag-by-flag `--app-env` assembly in shell goes away with it: extra env moves into `.sprout.yaml` (`preview.app_env` / - `seed.env`), the `SPROUT_APP_ENV` / `SPROUT_SEED_ENV` file-type variables, - or the `app_env_file` / `seed_env_file` component inputs. + `seed.env`), the `SPROUT_APP_ENV` / `SPROUT_SEED_ENV` file-type + variables, or the `app_env_file` / `seed_env_file` component inputs. 4. Any reconstruction of the preview hostname in CI (string munging - `pr-.host`) — read `PREVIEW_URL` / `preview_url=` from the CLI output - instead. + `pr-.host`) — read `PREVIEW_URL` / `preview_url=` from the CLI + output instead. After the migration no adopter shell script remains: the component declares -all three jobs (`sprout-preview`, `sprout-stop-preview`, `sprout-reset`), and -every behavior the old script hand-built (install, build/push, deploy, +all three jobs (`sprout-preview`, `sprout-stop-preview`, `sprout-reset`), +and every behavior the old script hand-built (install, build/push, deploy, dotenv, MR note, log dump on failure) is owned by `sprout ci preview` / `sprout ci teardown`. The `.sprout.yaml` `seed:` block replaces the `-s` plumbing: when present, the CLI builds + pushes the seed image and deploys diff --git a/docs/cli-reference.md b/docs/cli-reference.md index 266bc59e..ffb5d495 100644 --- a/docs/cli-reference.md +++ b/docs/cli-reference.md @@ -1,14 +1,11 @@ # CLI reference -The `sprout` CLI talks to the gateway API. Auth: set `SPROUT_URL` and -`SPROUT_TOKEN` (except `health`). Install a release binary (glibc: -`sprout-linux-x64`; Alpine: `sprout-linux-x64-musl`) or run from this -monorepo via `bun run --cwd apps/cli`. Full flags per command: -`sprout --help`. - -- First preview? See [Getting started](getting-started.md). -- CI wiring? See [CI integration](ci-integration.md). -- Manifest keys? See [Adopting a repo](adopting-a-repo.md). +`sprout` talks to the gateway API. Auth: `SPROUT_URL` and `SPROUT_TOKEN` +(except `health`). Install a release binary (glibc: `sprout-linux-x64`; +Alpine: `sprout-linux-x64-musl`) or run from this monorepo via +`bun run --cwd apps/cli`. Full flags per command: `sprout +--help`. Identity (repo + PR id) is inferred from CI env or git remote; +override with `--repo` where supported. ## Commands @@ -26,9 +23,6 @@ monorepo via `bun run --cwd apps/cli`. Full flags per command: | `sprout ci ` | CI helper: build/push/deploy, teardown, reseed (data kept) or reset (data wiped), or logs from CI env. | | `sprout worktree-db ` | Local per-worktree Postgres DB (no gateway). | -Identity (repo + PR id) is inferred from CI env or git remote; override with -`--repo` where supported. - ## `ci` commands `usage: sprout ci …` — repo, MR id, and @@ -62,13 +56,13 @@ deploy error exits non-zero. The MR note is best-effort in both directions Low-level equivalents (`sprout deploy -i … -s …`, `sprout teardown`, `sprout logs`) still work for GitHub Actions and laptops — see -[CI integration](ci-integration.md#github-actions). Their env/seed/service flags mirror the `ci` -surface (`-i`, `-s`, `--reseed`, `--service`, `--clear-services`, -`--app-env[-file]`, `--seed-env[-file]`, `--seed-arg`). +[CI integration](ci-integration.md#github-actions). Their env/seed/service +flags mirror the `ci` surface (`-i`, `-s`, `--reseed`, `--service`, +`--clear-services`, `--app-env[-file]`, `--seed-env[-file]`, `--seed-arg`). ## Debugging -Low-level flow: when a preview is red, pull container logs through the gateway (no Docker +When a preview is red, pull container logs through the gateway (no Docker socket on the CI runner or laptop): ```bash @@ -79,20 +73,22 @@ sprout logs --tail 200 # sprout logs --tail 200 --repo "https://github.com/org/repo" ``` -Repo resolution matches `deploy` / `drop`: `--repo`, else `GITHUB_REPOSITORY` / -`CI_PROJECT_URL`, else `git remote get-url origin`. The deploy token is scoped -to one canonical repo — it cannot read another repo's previews. +Repo resolution: `--repo`, else `GITHUB_REPOSITORY` / `CI_PROJECT_URL`, +else `git remote get-url origin`. The deploy token is scoped to one +canonical repo — it cannot read another repo's previews. -Output is live app container logs, then seed logs when available. While the -seed container is still running, `sprout logs` reads it live; after a failed -seed the gateway has already captured stdout/stderr into `seed_log` (before -remove) so the seed section still shows why seeding died. Status polling keeps -a short `last_error_detail` (`exit=7`, `timeout`) — not the log blob. -Successful seeds do not keep seed output. +Output: live app container logs, then seed logs when available. While the +seed container is still running, `sprout logs` reads it live; after a +failed seed the gateway has already captured stdout/stderr into `seed_log` +(before remove) so the seed section still shows why seeding died. Status +polling keeps a short `last_error_detail` (`exit=7`, `timeout`) — not the +log blob. Successful seeds do not keep seed output. Fixes for what the +logs show live in [Troubleshooting](troubleshooting.md). ## Deploy token setup -One-time per adopting repo (operator or lead dev with admin token): +One-time per adopting repo (operator or lead dev with admin token). +Canonical recipe: [Adopting a repo](adopting-a-repo.md#deploy-token-setup). ```bash export SPROUT_URL=https://sprout.example.com @@ -107,23 +103,28 @@ echo, or commit a token value. ## Worktree DB (local provisioner) -Parallel agents on one machine can clash on shared Postgres credentials. The -CLI provisions an isolated DB + LOGIN role per worktree **without** talking -to the gateway (operator detail; full notes in -[Operator deploy](operator-deploy.md#worktree-db-local-provisioner)): +Provision an isolated DB + LOGIN role per worktree **without** talking to +the gateway. Operator detail: [Operator deploy](operator-deploy.md#worktree-db-local-provisioner). ```bash sprout worktree-db provision --slug --env-file --admin-url "$ADMIN_DSN" sprout worktree-db drop --slug --admin-url "$ADMIN_DSN" ``` +| Flag | Purpose | +|---|---| +| `--slug ` | Worktree key: lowercase, `[^a-z0-9-]` → `-`, collapse runs, max 40 chars. Distinct from adopting-repo **slug**. Objects are named `sprout_wt_` (hyphens → underscores). `drop` refuses any name outside the `sprout_wt_` prefix. | +| `--env-file ` | Connection vars file (created if missing; atomic temp+rename). When `PGPASSWORD` is already present, that password is reused so live connections are not rotated. Writes `DATABASE_URL` plus canonical `PGHOST` / `PGPORT` / `PGUSER` / `PGPASSWORD` / `PGDATABASE` unless renamed. | +| `--admin-url ` | Admin DSN with `CREATEROLE` (or superuser). | +| `--rename LOGICAL=NAME` | Rename a logical key (`DATABASE_URL`, `PGHOST`, …). Repeatable. | + ## Test coverage (maintainers) -Repo-relative paths for the reference contract above: +Repo-relative paths for the contract above: -- Manifest parsing (all `.sprout.yaml` keys, including `preview.env` remap, - `app_env` / `seed.env` grammar, services, health, `build`/`seed` blocks): - `apps/cli/src/yaml.test.ts` +- Manifest parsing (all `.sprout.yaml` keys, including `preview.env` + remap, `app_env` / `seed.env` grammar, services, health, `build`/`seed` + blocks): `apps/cli/src/yaml.test.ts` - Env value grammar (placeholders, `generate`, `required`): `apps/cli/src/app-env.test.ts`, `apps/cli/src/app-env-values.test.ts` - CLI env layering (yaml + blob + files + flags): @@ -156,7 +157,6 @@ Repo-relative paths for the reference contract above: `apps/server/src/http/deploy-mail.test.ts` - Mailbox / From presentation (MR note lines, settled deploy output): `apps/cli/src/commands/forge-note.test.ts` -- Send-and-read-back through Mailpit's API: `e2e/mail.test.ts` - Substituted hostname on a live deploy: `e2e/lifecycle.test.ts` - Component inputs / dotenv / `on_stop` wiring: `templates/preview.test.ts` diff --git a/docs/getting-started.md b/docs/getting-started.md index 77cf71ed..d6e55ea5 100644 --- a/docs/getting-started.md +++ b/docs/getting-started.md @@ -1,77 +1,70 @@ # Getting started -First preview in one sitting: one `.sprout.yaml`, two CI variables, one CI -include. Five minutes on a repo that already has a `Dockerfile`. +Reach a first preview in one sitting: one `.sprout.yaml`, two CI variables, +one CI include. About five minutes on a repo that already ships a +`Dockerfile` that serves HTTP. -- New adopter? Read this page only, then run the onboarding prompt in your - coding harness — the same block as - [Onboarding prompt](onboarding-prompt.md). -- Need an exact key? See [Adopting a repo](adopting-a-repo.md). -- Need CI details for your forge? See [CI integration](ci-integration.md). -- Running the gateway itself? See [Operator deploy](operator-deploy.md). +This lesson takes the GitLab path. On GitHub the steps are the same shape +with a caller workflow instead of an include — branch at step 3 to +[CI integration](ci-integration.md#github-actions) and rejoin at +[What you get](#what-you-get). + +- Exact key contract: [Adopting a repo](adopting-a-repo.md). +- Gateway setup (operator work): [Operator deploy](operator-deploy.md). ## Prerequisites - A repo with a `Dockerfile` that serves HTTP. -- An operator-deployed gateway URL plus a deploy token for your repo - (ask your operator; bootstrapping is in - [Operator deploy](operator-deploy.md#bootstrap-admin-token)). -- GitLab merge-request pipelines (or GitHub `pull_request` workflows). +- A gateway URL plus a deploy token for the repo from the operator. + Bootstrapping is operator work: + [Operator deploy](operator-deploy.md#bootstrap-admin-token). +- Merge-request pipelines (GitLab) or `pull_request` workflows (GitHub). Copy-paste app files live in [`examples/adopting-repo/README.md`](../examples/adopting-repo/README.md). ## 1. `.sprout.yaml` at the repo root -Seeded preview (the `health:` block is required whenever `seed:` is -configured): +Write this file. Replace `myapp` and the domain with values confirmed with +a human — never invent a domain. The template must contain `{pr_id}` as a +bare host: no scheme, port, or path. ```yaml slug: myapp preview: hostname: "pr-{pr_id}.myapp.preview.example.com" -health: - path: /health - interval: 2s - timeout: 120s - expect: 200 -seed: - dockerfile: Dockerfile.seed - env: - FIXTURE_SET: demo ``` -Omit the `seed:` block for an app-only preview (the `health:` block can go -too — the gateway defaults to `GET /health` every `2s` for up to `120s`, -expecting `200`). Confirm the `slug` and the hostname template with a human -before committing: never invent a domain. Full key contract in +Check the file loads with the CLI's own loader (`apps/cli/src/yaml.ts`, +`parseSproutYaml`). Unknown keys are rejected (`unknown key: `), so +typos fail here, not at deploy. + +When this deploys, add seeding next: [Adopting a repo](adopting-a-repo.md#manifest-keys-sproutyaml). -Minimal app-only manifest: +## 2. CI variables (you set these) -```yaml -slug: myapp -preview: - hostname: "pr-{pr_id}.myapp.preview.example.com" -``` +Set two masked variables in the repo's CI settings: -## 2. CI variables (you set these) +- `SPROUT_URL` — the gateway URL. (Alternatively pass the `sprout_url` + input; one of the two is required.) +- `SPROUT_TOKEN` — the deploy token scoped to this repo's canonical id. -| Variable | Type | Purpose | -|---|---|---| -| `SPROUT_URL` | Variable, masked, required | Gateway URL (or pass the `sprout_url` / `sprout_url` input instead) | -| `SPROUT_TOKEN` | Variable, masked, required | Deploy token scoped to the repo's canonical id | -| `GITLAB_TOKEN` | Variable, masked, optional (GitLab) | Note-write token for the MR note; without it the CLI falls back to `CI_JOB_TOKEN` (best-effort note either way) | +For GitLab MR notes, an optional `GITLAB_TOKEN` (masked) lets the CLI +create notes; without it the CLI falls back to `CI_JOB_TOKEN` on a +best-effort basis either way. -Optional: `SPROUT_APP_ENV` / `SPROUT_SEED_ENV` as masked **File** variables -holding dotenv blobs for app / seed secrets. Never commit a secret to the -manifest — declare `{ required: true }` and supply it here. +App or seed secrets never go in the manifest. Declare +`{ required: true }` there and supply values here as masked **File** +variables (`SPROUT_APP_ENV` / `SPROUT_SEED_ENV` dotenv blobs). ## 3. CI wiring (one include) -GitLab (`.gitlab-ci.yml` — one include, no scripts): +Add the include to `.gitlab-ci.yml`. Replace `/sprout-ci` with the +component project path on the instance and `v0.8.3` with the adopted +release: ```yaml include: @@ -79,38 +72,26 @@ include: inputs: { stage: deploy } ``` -GitHub (caller workflow — the canonical caller is -[`examples/adopting-repo/.github/workflows/sprout.yml`](../examples/adopting-repo/.github/workflows/sprout.yml)): - -```yaml -jobs: - preview: - uses: simpros/sprout/.github/workflows/preview.yml@v0.8.3 - with: - sprout_version: v0.8.3 - secrets: - SPROUT_URL: ${{ secrets.SPROUT_URL }} - SPROUT_TOKEN: ${{ secrets.SPROUT_TOKEN }} -``` - -Full wiring, variables, reset, and notes in [CI integration](ci-integration.md). +Open a merge request. The `sprout-preview` job installs the pinned CLI, +builds and pushes the app image, deploys, and posts the MR note with the +preview URL. ## What you get -- Open / synchronize: `sprout ci preview` builds + pushes images, deploys, - writes `PREVIEW_URL=`, posts the MR/PR note. Read the URL from the CLI - output — never reconstruct the hostname in CI. -- Close / merge: `sprout ci teardown` (idempotent). +- Open / synchronize: the job deploys and writes `PREVIEW_URL=`. Read the + URL from the CLI output — never reconstruct the hostname in CI. +- Close / merge: `sprout ci teardown` runs via `on_stop` (idempotent — + exit 0 when already gone). - Manual wipe + redeploy: `sprout ci reset` (data wiped). -- Sweep recovers if teardown is missed. -- The gateway reports anonymous [install telemetry](telemetry.md) by default - (`SPROUT_TELEMETRY=off` stops it) — operator concern, nothing to do here. +- A missed teardown is recovered by the gateway sweep. +- The gateway reports anonymous [install telemetry](telemetry.md) by + default (`SPROUT_TELEMETRY=off` stops it) — operator concern, nothing to + do in this lesson. ## Next step Run the agent block in [Onboarding prompt](onboarding-prompt.md), or verify -by hand: the manifest must pass the CLI loader (`apps/cli/src/yaml.ts`), -then `sprout doctor` and a first `sprout ci preview` run from CI. +by hand: `sprout doctor`, then a first `sprout ci preview` run from CI. ## See also diff --git a/docs/herdr-integration.md b/docs/herdr-integration.md index 6061cbf6..03c9ae97 100644 --- a/docs/herdr-integration.md +++ b/docs/herdr-integration.md @@ -1,19 +1,19 @@ # Herdr review integration (operator machine) -Optional operator-side automation that reviews every merge / pull request and -fixes feedback without human relay. Adopters need nothing for it: no extra CI -job, no extra variable, no `.sprout.yaml` key. This page is for the operator -who runs it on their own machine, so reviewers know where the bot comments -come from and where to look when they don't arrive. The `~/.hermes` / +Run operator-side automation that reviews every merge / pull request and +applies feedback without human relay. Adopters need nothing for it: no extra +CI job, no extra variable, no `.sprout.yaml` key. This page is for the +operator who runs it on their own machine, so reviewers know where the bot +comments come from and where to look when they don't arrive. Each section +below is one piece: **trigger → output → log location**. The `~/.hermes` / `~/.herdr-eyes` paths below live on the operator machine, not in this repo, and may drift — the scripts themselves are the source of truth. -All four pieces are cron-driven scripts on the operator machine. Each section -below follows the same shape: **trigger → output → log location**. - ## pr-review-watch — thermo-nuclear review per MR +Review each push with the read-only code-quality rubric: + - **Trigger:** cron tick (every few minutes). Polls open MRs/PRs in the watcher's repo registry, compares each head SHA against its state file. New MR or new push (changed head SHA) starts one review round. @@ -43,6 +43,8 @@ below follows the same shape: **trigger → output → log location**. ## pr-review-fix — auto-fix loop +Apply review feedback and push to the MR branch: + - **Trigger:** kicked detached by `pr-review-watch` after any `REQUEST CHANGES` verdict (guarded so one review round kicks exactly one fix), or run manually against a review worktree. @@ -67,6 +69,8 @@ below follows the same shape: **trigger → output → log location**. ## herdr-eyes tick — agent monitor +Track agent worktrees and post status transitions: + - **Trigger:** cron tick (every 2 minutes), script-only (no agent). Diffs the live herdr snapshot against its state file. - **What it does:** tracks one Discord channel per agent worktree (created on @@ -85,6 +89,8 @@ below follows the same shape: **trigger → output → log location**. ## sprout-chain — next-ticket kicker +Kick the next unblocked ticket, at most one per run: + - **Trigger:** cron tick (every 10 minutes), script-only. Scans the sprout tracker for the milestone's next unblocked ticket. - **What it does:** kicks at most one ticket per run: the issue must be open, @@ -101,8 +107,9 @@ below follows the same shape: **trigger → output → log location**. ## Discord output format (one-liners) -Full findings always live on the forge (the MR/PR comment). Discord carries -one short line per event — no technical detail, no report paste: +Keep Discord to one short line per event. Full findings always live on the +forge (the MR/PR comment) — Discord carries no technical detail, no report +paste: ```text 🧊 mr-review # · round · verdict · 🔗 diff --git a/docs/onboarding-prompt.md b/docs/onboarding-prompt.md index dc20b866..1ef89a26 100644 --- a/docs/onboarding-prompt.md +++ b/docs/onboarding-prompt.md @@ -1,9 +1,9 @@ # Onboarding prompt -One copy-paste block. Paste it into your coding harness (Claude Code, Cursor, -Codex, …) in the repo you want to preview. The agent reads the docs itself, -inspects your repo, confirms the domain-shaped values, and writes only the -config + CI wiring. +One copy-paste block for the repo being onboarded. Paste it into a coding +harness (Claude Code, Cursor, Codex, …). The agent reads the docs, inspects +the repo, confirms the two human values (slug, hostname template), and +writes only config + CI wiring. --- diff --git a/docs/operator-deploy.md b/docs/operator-deploy.md index e5b8c8b7..71a5d131 100644 --- a/docs/operator-deploy.md +++ b/docs/operator-deploy.md @@ -1,22 +1,23 @@ # Operator deploy Deploy **Postgres**, the **sprout gateway**, and **Traefik** once per -environment. Adopting repos then call `sprout deploy` / `sprout teardown` from -CI — no per-repo server setup. +environment. Adopting repos then call `sprout deploy` / `sprout teardown` +from CI — no per-repo server setup. Each section below is one task. ## Prerequisites - Docker Engine with Compose v2 (`docker compose`) -- Host access to publish ports (local smoke defaults: gateway `7331`, Traefik - HTTP `8880`) — or readiness to join existing Docker networks for production -- For production-shaped deploys: an existing Traefik and/or Postgres you can +- Host access to publish ports (local smoke defaults: gateway `7331`, + Traefik HTTP `8880`) — or readiness to join existing Docker networks for + production +- For production-shaped deploys: an existing Traefik and/or Postgres to attach to (Coolify-managed Traefik is fine; sprout does not call Coolify) - Bun is **not** required for the compose path (image build uses the `oven/bun` base). Bun is only needed for host `bun run dev`. ## Quick start (local smoke) -From the repo root: +Bring up the reference stack from the repo root: ```bash cp compose.env.example compose.env @@ -34,23 +35,23 @@ curl -sf http://127.0.0.1:${TRAEFIK_HTTP_PORT:-8880}/ || true `.env` / `.env.example` are for the gateway process on the host (`bun run dev`). -The stack in `docker-compose.yml` is the reference **operator compose stack** -for local development and CI smoke. The **E2E acceptance harness** (`e2e/`) -runs the same file with `--env-file e2e/compose.e2e.env` — see +The stack in `docker-compose.yml` is the reference **operator compose +stack** for local development and CI smoke. The **E2E acceptance harness** +(`e2e/`) runs the same file with `--env-file e2e/compose.e2e.env` — see [`e2e/README.md`](../e2e/README.md) or `bun run test:e2e`. Default network -names (`sprout-traefik`, `sprout-postgres`) are project-local so -a smoke `up` does not collide with an existing Coolify Traefik network named +names (`sprout-traefik`, `sprout-postgres`) are project-local so a smoke +`up` does not collide with an existing Coolify Traefik network named `traefik`. Host ports are `SPROUT_GATEWAY_HOST_PORT` (default 7331) and `TRAEFIK_HTTP_PORT` (default 8880); the harness ports come from `e2e/compose.e2e.env`. ### Gateway Docker image -The gateway is **one image, one process** (root `Dockerfile`), and the image -also embeds the `sprout` CLI (same Bun lockfile / version pin as the gateway) -so operators can `docker exec` against localhost without a host-side install. -Compose builds it via `build: .`; you can also build and tag it alone for -registry push or external orchestrators: +Ship one image, one process (root `Dockerfile`). The image also embeds the +`sprout` CLI (same Bun lockfile / version pin as the gateway) so operators +can `docker exec` against localhost without a host-side install. Compose +builds it via `build: .`; build and tag it alone for registry push or +external orchestrators: ```bash # From the repo root (reproducible with Bun 1.4.0 base + frozen lockfile) @@ -62,12 +63,15 @@ docker build -t ghcr.io/simpros/sprout:0.8.3 \ ``` Image label `org.opencontainers.image.version` mirrors `SPROUT_VERSION` -(Dockerfile default tracks the monorepo pin; prefer a published release tag such -as `v0.8.3` / image `:0.8.3` in production). Pin operators and CI to a release -tag or GHCR digest — not an untagged local build — when publishing previews. +(Dockerfile default tracks the monorepo pin; prefer a published release tag +such as `v0.8.3` / image `:0.8.3` in production). Pin operators and CI to a +release tag or GHCR digest — not an untagged local build — when publishing +previews. ## Architecture +Route PR traffic through the three services: + ```text ┌─────────────┐ PR traffic ─────►│ Traefik │ network: $SPROUT_TRAEFIK_NETWORK @@ -85,12 +89,14 @@ tag or GHCR digest — not an untagged local build — when publishing previews. └─────────────┘ ``` -- **Postgres** hosts all preview logical databases (`sprout__pr`). +- **Postgres** hosts all preview logical databases + (`sprout__pr`). - **Gateway** administers databases, starts preview containers, and sets - Traefik routing labels. It mounts the Docker socket and joins both networks. + Traefik routing labels. It mounts the Docker socket and joins both + networks. - **Traefik** terminates HTTP for preview hostnames. Preview app containers - attach only to `traefik` + `postgres`; seed containers attach to `postgres` - only. + attach only to `traefik` + `postgres`; seed containers attach to + `postgres` only. ### Defaults @@ -109,16 +115,16 @@ Durable runtime identity uses product `sprout*` names: ### Dual network attach -The gateway reads two Docker network names from the environment: +Point the gateway at two Docker network names: | Variable | Purpose | |---|---| | `SPROUT_TRAEFIK_NETWORK` | Network shared with Traefik. Preview **app** containers join this network so Traefik can route traffic via Docker labels. | | `SPROUT_POSTGRES_NETWORK` | Network shared with Postgres. Gateway, preview **app**, and **seed** containers join this network so they can reach the database by hostname. | -Compose declares both networks with `name: ${SPROUT_…}` so the same variable is the -single source for the Docker network name and the gateway env (defaults are -project-local): +Compose declares both networks with `name: ${SPROUT_…}` so the same +variable is the single source for the Docker network name and the gateway +env (defaults are project-local): ```yaml networks: @@ -128,46 +134,49 @@ networks: name: ${SPROUT_POSTGRES_NETWORK:-sprout-postgres} ``` -When the gateway creates a Postgres preview app container it attaches **both** -networks. Postgres seed containers get **Postgres only** — they never need -Traefik reachability. SQLite preview containers (app, services, seed) mount -their named volume instead and join **Traefik only**; a SQLite-only gateway -needs no `SPROUT_POSTGRES_NETWORK` at all. +When the gateway creates a Postgres preview app container it attaches +**both** networks. Postgres seed containers get **Postgres only** — they +never need Traefik reachability. SQLite preview containers (app, services, +seed) mount their named volume instead and join **Traefik only**; a +SQLite-only gateway needs no `SPROUT_POSTGRES_NETWORK` at all. ## One-click Coolify (Docker Compose Empty) -For a one-click Coolify resource with its own Postgres, paste -[`deploy/coolify/gateway.compose.yml`](../deploy/coolify/gateway.compose.yml) and follow -[`deploy/coolify/README.md`](../deploy/coolify/README.md) (wildcard DNS, -preview hostname template, optional mail/forwardAuth). Both +Stand up a gateway with its own Postgres inside Coolify. Paste +[`deploy/coolify/gateway.compose.yml`](../deploy/coolify/gateway.compose.yml) +and follow [`deploy/coolify/README.md`](../deploy/coolify/README.md) +(wildcard DNS, preview hostname template, optional mail/forwardAuth). Both `SPROUT_TRAEFIK_NETWORK` and `SPROUT_POSTGRES_NETWORK` point at the predefined external `coolify` network — the only name known at template time that the Coolify proxy (`coolify-proxy`) is attached to. ## Production-shaped deploy (external / Coolify Traefik) -sprout does **not** manage Traefik or call the Coolify API. It registers -routes by setting standard [Traefik Docker labels](https://doc.traefik.io/traefik/providers/docker/) -on preview app containers. - -To coexist with an **externally managed Traefik** (including Coolify's), use the -overlay instead of forking the reference file: +Coexist with an **externally managed Traefik** (including Coolify's) +without forking the reference file. sprout does **not** manage Traefik or +call the Coolify API. It registers routes by setting standard [Traefik +Docker +labels](https://doc.traefik.io/traefik/providers/docker/) +on preview app containers. Use the overlay: -1. Set `SPROUT_TRAEFIK_NETWORK` / `SPROUT_POSTGRES_NETWORK` in `compose.env` to the - existing network names (Coolify's predefined shared network is `coolify`). +1. Set `SPROUT_TRAEFIK_NETWORK` / `SPROUT_POSTGRES_NETWORK` in + `compose.env` to the existing network names (Coolify's predefined shared + network is `coolify`). 2. For HTTPS routers, set Traefik TLS knobs to match that proxy — e.g. `SPROUT_TRAEFIK_ENTRYPOINTS=https` and optionally `SPROUT_TRAEFIK_CERTRESOLVER=letsencrypt` on Coolify (HTTP-01 is fine for - small fleets). If you need a single wildcard / DNS-01 for rate limits or - fleet growth, follow [Wildcard preview certificate (DNS-01)](#wildcard-preview-certificate-dns-01) - — often you keep the name `letsencrypt` after converting that resolver. - Leave both unset for HTTP-only Traefik (bundled compose / local). Empty - entrypoints keeps TLS off even if certresolver is set. + small fleets). If a single wildcard / DNS-01 is needed for rate limits or + fleet growth, follow [Wildcard preview certificate + (DNS-01)](#wildcard-preview-certificate-dns-01) — often the name + `letsencrypt` is kept after converting that resolver. Leave both unset + for HTTP-only Traefik (bundled compose / local). Empty entrypoints keeps + TLS off even if certresolver is set. 3. For an SSO gate on preview hosts (Traefik forwardAuth), set both `SPROUT_TRAEFIK_MIDDLEWARES` (a single name, e.g. `voidauth`) and `SPROUT_FORWARDAUTH_ADDRESS` (a URL Traefik can reach). The gateway emits the middleware **definition** and the router **attachment** as Docker - labels — no Traefik static/file config. Leave both empty for open previews. + labels — no Traefik static/file config. Leave both empty for open + previews. 4. Ensure those networks exist (`docker network create …` if needed). 5. Bring up **only the gateway** against external networks: @@ -180,20 +189,20 @@ The overlay marks both networks `external: true` and disables the bundled `traefik` / `postgres` services (via profiles). Point `SPROUT_PREVIEW_POSTGRES_URL` at the external Postgres admin DSN. -Also set **`SPROUT_PG_HOST`** (and `SPROUT_PG_PORT` if not 5432) to the hostname -preview app and seed containers use to reach Postgres on -`SPROUT_POSTGRES_NETWORK`. That is often different from the host in the admin DSN -(dual-homed setups). The bundled default `postgres` only works when a service -with that DNS name exists on the network. +Also set **`SPROUT_PG_HOST`** (and `SPROUT_PG_PORT` if not 5432) to the +hostname preview app and seed containers use to reach Postgres on +`SPROUT_POSTGRES_NETWORK`. That is often different from the host in the +admin DSN (dual-homed setups). The bundled default `postgres` only works +when a service with that DNS name exists on the network. The gateway creates or syncs the preview login (`SPROUT_PG_USER` / -`SPROUT_PG_PASSWORD`) on boot from the admin DSN — no manual `CREATE ROLE` and -no compose one-shot. The admin role needs `CREATEROLE` (or superuser); otherwise -boot fails with a clear error. +`SPROUT_PG_PASSWORD`) on boot from the admin DSN — no manual `CREATE ROLE` +and no compose one-shot. The admin role needs `CREATEROLE` (or superuser); +otherwise boot fails with a clear error. Also ensure the external Traefik has `--providers.docker=true` and -`--providers.docker.exposedbydefault=false` (or equivalent) so only labelled -containers are published. +`--providers.docker.exposedbydefault=false` (or equivalent) so only +labelled containers are published. Label conventions the gateway applies: @@ -210,10 +219,10 @@ When TLS is enabled (`SPROUT_TRAEFIK_ENTRYPOINTS` non-empty), also: TLS is opt-in: unset/blank entrypoints → HTTP labels only (works with the bundled Traefik `web` entrypoint). Entrypoints without certresolver uses -Traefik's default/builtin cert. Entrypoint and certresolver names are **not** -Coolify-specific — set them to match your Traefik (e.g. Coolify often uses -`https` + `letsencrypt`; stock Traefik quickstarts often use `websecure` + a -custom resolver name). +Traefik's default/builtin cert. Entrypoint and certresolver names are +**not** Coolify-specific — set them to match the Traefik in use (e.g. +Coolify often uses `https` + `letsencrypt`; stock Traefik quickstarts often +use `websecure` + a custom resolver name). When forwardAuth is enabled (both `SPROUT_TRAEFIK_MIDDLEWARES` and `SPROUT_FORWARDAUTH_ADDRESS` non-empty), also: @@ -226,10 +235,11 @@ When forwardAuth is enabled (both `SPROUT_TRAEFIK_MIDDLEWARES` and (`` is the single middleware name from `SPROUT_TRAEFIK_MIDDLEWARES`.) ForwardAuth is opt-in and Coolify-safe: Coolify's Traefik uses the Docker -provider only, so sprout must emit **both** the middleware definition and the -router attachment on preview containers (no Traefik file/static config). -Both env vars must be set together (or both empty); setting only one fails at -gateway boot. `SPROUT_TRAEFIK_MIDDLEWARES` is one name (commas rejected). +provider only, so sprout must emit **both** the middleware definition and +the router attachment on preview containers (no Traefik file/static +config). Both env vars must be set together (or both empty); setting only +one fails at gateway boot. `SPROUT_TRAEFIK_MIDDLEWARES` is one name +(commas rejected). Per-repo alternative: adopting repos can attach their own previews to Traefik routers or middlewares defined outside the gateway with @@ -240,50 +250,52 @@ manifest keys change one repo's previews. Gateway-emitted keys stay reserved — a manifest key colliding with one fails the deploy fast instead of silently overriding routing. -**VoidAuth / SSO setup:** add the preview domain (e.g. `*.internal.example.com`) -in the VoidAuth UI, then point `SPROUT_FORWARDAUTH_ADDRESS` at the forward-auth -endpoint (e.g. `https://auth.example.com/api/authz/forward-auth`). That URL -must be reachable from Traefik (not only from browsers). Unauthenticated -visits redirect to login; authenticated visits reach the preview app. +**VoidAuth / SSO setup:** add the preview domain (e.g. +`*.internal.example.com`) in the VoidAuth UI, then point +`SPROUT_FORWARDAUTH_ADDRESS` at the forward-auth endpoint (e.g. +`https://auth.example.com/api/authz/forward-auth`). That URL must be +reachable from Traefik (not only from browsers). Unauthenticated visits +redirect to login; authenticated visits reach the preview app. -Coolify-managed Traefik already watches the Docker socket; sprout -preview containers appear alongside Coolify apps as long as they share the -Traefik network. +Coolify-managed Traefik already watches the Docker socket; sprout preview +containers appear alongside Coolify apps as long as they share the Traefik +network. ### Per-preview access gate (`preview.auth`) -Adopters opt in per repo with `preview.auth: basic | link` (see +Offer adopters the per-repo opt-in (`preview.auth: basic | link`, see [Previews](previews.md#preview-access)). `basic` needs nothing from the operator. `link` needs both `SPROUT_PREVIEW_AUTH_SECRET` (HMAC root for shareable tokens) and `SPROUT_PREVIEW_AUTH_ADDRESS` (the gateway's in-network forwardAuth URL, e.g. `http://gateway:7331/v1/internal/preview-auth`) — set together or both -empty; a partial set fails gateway boot naming the missing key. The -address must be Traefik-reachable without leaving the Docker network: -that is the whole point (an external auth service has no Cloudflare-free -path to the preview edge, and Cloudflare rewrites `X-Forwarded-Host`, so -an external gate authorised everything). Never point it at a public URL. +empty; a partial set fails gateway boot naming the missing key. The address +must be Traefik-reachable without leaving the Docker network: that is the +whole point (an external auth service has no Cloudflare-free path to the +preview edge, and Cloudflare rewrites `X-Forwarded-Host`, so an external +gate authorised everything). Never point it at a public URL. ### Wildcard preview certificate (DNS-01) -Per-preview Let's Encrypt certs hit the **50 certificates / registered domain / -week** rate limit under fleet growth or ACME-store wipes. One wildcard on the -preview domain (`*.previews.example.com`) removes per-deploy issuance: Traefik -serves the stored cert for every preview host without ordering a new one. +Stop per-preview Let's Encrypt issuance under fleet growth. Per-preview +certs hit the **50 certificates / registered domain / week** rate limit +under fleet growth or ACME-store wipes. One wildcard on the preview domain +(`*.previews.example.com`) removes per-deploy issuance: Traefik serves the +stored cert for every preview host without ordering a new one. -sprout still emits optional `tls.certresolver` labels (#102). Traefik resolves -via that named resolver and must find a covering cert in **that** resolver's -store. With `*.PREVIEW_DOMAIN` already there, Traefik should log a store hit / -`No ACME certificate generation required` for the preview FQDN — not a new LE -order. Wrong resolver name → per-host orders resume (the rate-limit failure -mode). +sprout still emits optional `tls.certresolver` labels (#102). Traefik +resolves via that named resolver and must find a covering cert in +**that** resolver's store. With `*.PREVIEW_DOMAIN` already there, Traefik +should log a store hit / `No ACME certificate generation required` for the +preview FQDN — not a new LE order. Wrong resolver name → per-host orders +resume (the rate-limit failure mode). Ready-to-apply fragments (placeholders only): [`deploy/traefik/README.md`](../deploy/traefik/README.md). #### Prerequisites (secrets / access — not in this repo) -You need these before executing the runbook; the repo ships fragments only: +Collect these before the runbook; the repo ships fragments only: | Need | Why | |---|---| @@ -292,8 +304,8 @@ You need these before executing the runbook; the repo ships fragments only: | Ability to inject provider env vars into the Traefik process | Provider plugins read tokens from the environment (names are provider-specific) | | Durable volume (or equivalent) for ACME storage | Cert + account must survive Traefik restarts/upgrades | -Do **not** invent provider field names here — look up Traefik's docs for your -DNS provider. Fill placeholders in +Do **not** invent provider field names here — look up Traefik's docs for the +DNS provider in use. Fill placeholders in [`deploy/traefik/certificates-resolver.dns.yml`](../deploy/traefik/certificates-resolver.dns.yml) for the resolver body used in step 2. @@ -302,34 +314,36 @@ for the resolver body used in step 2. | Path | When | Gateway certresolver | |---|---|---| | **Default (simple)** | Traefik's job for this zone is preview TLS, or DNS API creds already cover every host Traefik terminates | Keep existing name (e.g. `letsencrypt`) | -| **Coexistence (optional)** | You must leave Coolify / other apps on HTTP-01 | Point sprout at the new DNS-01 name (e.g. `letsencrypt-dns`) | +| **Coexistence (optional)** | Coexistence with HTTP-01 for other apps must remain | Point sprout at the new DNS-01 name (e.g. `letsencrypt-dns`) | #### Runbook -1. **Identify the DNS provider** for the preview domain (the zone that serves - `*.previews.example.com`). Create API credentials with permission to manage - TXT records for ACME. Record the Traefik provider name and required env vars - from Traefik's DNS Providers list. +1. **Identify the DNS provider** for the preview domain (the zone that + serves `*.previews.example.com`). Create API credentials with permission + to manage TXT records for ACME. Record the Traefik provider name and + required env vars from Traefik's DNS Providers list. 2. **Resolver setup** — pick one branch from the table above: - - **Default — convert in place.** In Traefik static config (or Coolify proxy - settings), replace the existing resolver's `httpChallenge` block with the - `dnsChallenge` shape from + - **Default — convert in place.** In Traefik static config (or Coolify + proxy settings), replace the existing resolver's `httpChallenge` block + with the `dnsChallenge` shape from [`deploy/traefik/certificates-resolver.dns.yml`](../deploy/traefik/certificates-resolver.dns.yml) - (same key, often `letsencrypt`; keep the **same** durable `storage` path). - Converting replaces the challenge type for **every** consumer of that - resolver name — if any other app must stay on HTTP-01, stop and use - coexistence instead. Inject `` into the Traefik - process. No second resolver, no second `acme.json`, no rename dance. - - - **Coexistence — add a second resolver.** Traefik requires one storage file - per certificates resolver — reusing the HTTP-01 path (e.g. `/data/acme.json`) - causes init conflicts or a corrupted store. Copy the fragment body under a - **new** key (e.g. `letsencrypt-dns:`), set `storage` to a distinct path - (e.g. `/data/acme-dns.json`), and leave the HTTP-01 resolver untouched. - Paste **under** your existing `certificatesResolvers:` map (do not overwrite - the whole static ACME document). Merged shape: + (same key, often `letsencrypt`; keep the **same** durable `storage` + path). Converting replaces the challenge type for **every** consumer + of that resolver name — if any other app must stay on HTTP-01, stop + and use coexistence instead. Inject `` into the + Traefik process. No second resolver, no second `acme.json`, no rename + dance. + + - **Coexistence — add a second resolver.** Traefik requires one storage + file per certificates resolver — reusing the HTTP-01 path (e.g. + `/data/acme.json`) causes init conflicts or a corrupted store. Copy + the fragment body under a **new** key (e.g. `letsencrypt-dns:`), set + `storage` to a distinct path (e.g. `/data/acme-dns.json`), and leave + the HTTP-01 resolver untouched. Paste **under** the existing + `certificatesResolvers:` map (do not overwrite the whole static ACME + document). Merged shape: ```yaml certificatesResolvers: @@ -366,12 +380,12 @@ for the resolver body used in step 2. docker compose -f deploy/traefik/wildcard-bootstrap.compose.yml down ``` - The bootstrap compose attaches `tls.domains[0].main=*.${PREVIEW_DOMAIN}` so - Traefik requests a wildcard (DNS-01), not a single-host cert for + The bootstrap compose attaches `tls.domains[0].main=*.${PREVIEW_DOMAIN}` + so Traefik requests a wildcard (DNS-01), not a single-host cert for `bootstrap.*`. -4. **Point sprout at that resolver** (names must match Traefik). Default keeps - Coolify quickstart unchanged; coexistence points at the new name: +4. **Point sprout at that resolver** (names must match Traefik). Default + keeps Coolify quickstart unchanged; coexistence points at the new name: ```bash # compose.env / gateway env @@ -382,33 +396,33 @@ for the resolver body used in step 2. Deploy a preview. Traefik should terminate TLS with the stored wildcard; **no new LE order** for that hostname (store hit / `No ACME certificate generation required` in Traefik ACME logs). Other Coolify apps can keep - using `letsencrypt` (HTTP-01) on the coexistence path; preview hosts must - use the resolver that holds `*.PREVIEW_DOMAIN`. + using `letsencrypt` (HTTP-01) on the coexistence path; preview hosts + must use the resolver that holds `*.PREVIEW_DOMAIN`. -5. **Monitor renewal.** Let's Encrypt wildcards renew before expiry (~30 days - out). Confirm Traefik still has DNS credentials and that ACME storage is - writable (the DNS-01 resolver's path — coexistence: the distinct second - file). Optionally dry-run against the staging CA (`caServer`) on a - non-prod Traefik first. +5. **Monitor renewal.** Let's Encrypt wildcards renew before expiry (~30 + days out). Confirm Traefik still has DNS credentials and that ACME + storage is writable (the DNS-01 resolver's path — coexistence: the + distinct second file). Optionally dry-run against the staging CA + (`caServer`) on a non-prod Traefik first. #### Managed Traefik notes (Coolify and similar) sprout does not configure Coolify's proxy. On a managed Traefik: - Find where that product exposes **certificates resolvers** / ACME storage - (static config, UI, or generated compose). You need a `dnsChallenge` - resolver there — Docker labels alone cannot add one. + (static config, UI, or generated compose). A `dnsChallenge` resolver is + needed there — Docker labels alone cannot add one. - Prefer converting the existing resolver when this Traefik only terminates - hosts your DNS creds cover; use the coexistence branch only when HTTP-01 + hosts the DNS creds cover; use the coexistence branch only when HTTP-01 must remain for other apps. -- Ensure every ACME JSON path is on **persistent storage**, not an ephemeral - container filesystem. Dual resolvers need **two** durable files. +- Ensure every ACME JSON path is on **persistent storage**, not an + ephemeral container filesystem. Dual resolvers need **two** durable files. - Provider credentials belong in the **proxy** environment, not in sprout gateway env. - Gateway knobs stay the same: `SPROUT_TRAEFIK_ENTRYPOINTS` + - `SPROUT_TRAEFIK_CERTRESOLVER` must match the managed proxy's entrypoint and - the resolver that holds the wildcard. If the managed proxy cannot add or - convert to DNS-01, you keep per-host issuance and rate limits. + `SPROUT_TRAEFIK_CERTRESOLVER` must match the managed proxy's entrypoint + and the resolver that holds the wildcard. If the managed proxy cannot add + or convert to DNS-01, per-host issuance and rate limits remain. #### Verification (zero LE orders per deploy) @@ -421,33 +435,37 @@ After the wildcard is in the store: | New preview → **zero new LE orders** | Note LE account order count or Traefik ACME log cursor. `sprout deploy` a fresh PR host. Confirm logs show a store hit / `No ACME certificate generation required` for that FQDN against the resolver that holds the wildcard — **no** `Obtain` / new order; `openssl s_client` (or browser) shows the wildcard cert (`CN`/`SAN` includes `*.previews.example.com`). | | Rate-limit safety | Optional: temporarily revoke network to the DNS API and deploy again — TLS should still work from the store (renewal would fail later; issuance on deploy must not be required). | -Acceptance from [#105](https://github.com/simpros/sprout/issues/105): new preview deploys order zero LE certificates (store hit); ACME storage survives Traefik restarts/upgrades; renewal observed or staging dry-run verified. +Acceptance from [#105](https://github.com/simpros/sprout/issues/105): new +preview deploys order zero LE certificates (store hit); ACME storage +survives Traefik restarts/upgrades; renewal observed or staging dry-run +verified. ## Preview mail (Mailpit) -The reference compose stack ships a Mailpit service for preview mail, and -the gateway joins its `mail` network (`SPROUT_MAIL_NETWORK`, default -`sprout-mail`). Preview app, companion-service, and seed containers -receive the canonical `MAIL*` env (see [Previews](previews.md#email-from-a-preview)) and join that -network, so they reach Mailpit by hostname. Mail is never provisioned per -preview: one shared Mailpit, one shared inbox for the whole gateway. +Give previews a shared mailbox. The reference compose stack ships a Mailpit +service for preview mail, and the gateway joins its `mail` network +(`SPROUT_MAIL_NETWORK`, default `sprout-mail`). Preview app, +companion-service, and seed containers receive the canonical `MAIL*` env +(see [Previews](previews.md#email-from-a-preview)) and join that network, +so they reach Mailpit by hostname. Mail is never provisioned per preview: +one shared Mailpit, one shared inbox for the whole gateway. -Pointing at an external Mailpit (a Mailpit running outside the preview +Point at an external Mailpit (a Mailpit running outside the preview networks, or a long-lived instance shared across environments): 1. Set `SPROUT_MAIL_HOST` to the hostname preview containers use to reach it on `SPROUT_MAIL_NETWORK` — a container-network DNS name, mirroring how `SPROUT_PG_HOST` names Postgres. That is often different from the host an operator's browser uses. -2. Set `SPROUT_MAIL_NETWORK` to the Docker network Mailpit is attached - to (create it first if needed). Preview app containers join it - alongside Traefik (+ Postgres for `postgres` previews); seed - containers join it whenever a seed can run. +2. Set `SPROUT_MAIL_NETWORK` to the Docker network Mailpit is attached to + (create it first if needed). Preview app containers join it alongside + Traefik (+ Postgres for `postgres` previews); seed containers join it + whenever a seed can run. 3. Set `SPROUT_MAIL_UI_URL` to the inbox URL humans open (surfaced in the MR note, `sprout list`, and deploy output). 4. Leave `SPROUT_MAIL_PORT` at `1025` unless the instance listens - elsewhere; leave `SPROUT_MAIL_USER` / `SPROUT_MAIL_PASSWORD` empty - when Mailpit accepts any credentials (the bundled service does — + elsewhere; leave `SPROUT_MAIL_USER` / `SPROUT_MAIL_PASSWORD` empty when + Mailpit accepts any credentials (the bundled service does — `MP_SMTP_AUTH_ACCEPT_ANY=1`), otherwise set both. Unset group = no mail: with no `SPROUT_MAIL_*` set, previews deploy @@ -456,15 +474,15 @@ gateway boot (`Incomplete mail configuration: missing SPROUT_MAIL_HOST`). ### How the mailbox is protected -The bundled compose publishes Mailpit's UI (`8025`) and SMTP (`1025`) on -host ports for local smoke. In production, treat the mailbox as internal: +Treat the mailbox as internal. The bundled compose publishes Mailpit's UI +(`8025`) and SMTP (`1025`) on host ports for local smoke; in production: - Put basic auth on the Mailpit UI and API with Mailpit's own knobs (`MP_UI_AUTH` as `user:password`, or `MP_UI_AUTH_FILE` pointing at a file holding it) and restrict serving with `MP_ALLOWED_HOSTS` — see the Mailpit docs for exact semantics. -- Do not publish the SMTP port beyond the preview networks; previews - reach it over `SPROUT_MAIL_NETWORK`, browsers need only the UI. +- Do not publish the SMTP port beyond the preview networks; previews reach + it over `SPROUT_MAIL_NETWORK`, browsers need only the UI. - Never use a preview to send real customer mail. The default `SPROUT_MAIL_FROM_DOMAIN=preview.invalid` is a reserved suffix that can never deliver for real — keep it unless the fleet has a dedicated, @@ -472,11 +490,11 @@ host ports for local smoke. In production, treat the mailbox as internal: ## Dashboard (read-only, opt-in) -The gateway can serve a single human-readable page of live previews at -`GET /dashboard` — one row per preview with repository slug, PR id linked to -the forge PR, status, clickable preview URL, database name and provider, -created and last-deploy timestamps, mail identity when mail is configured, -and the last deploy outcome or error code. It renders from the same state the +Serve a human-readable page of live previews at `GET /dashboard` — one row +per preview with repository slug, PR id linked to the forge PR, status, +clickable preview URL, database name and provider, created and last-deploy +timestamps, mail identity when mail is configured, and the last deploy +outcome or error code. It renders from the same state the `GET /v1/previews` read surface reads, costs one state query per request regardless of preview count (no per-preview container inspection), and ships as server-rendered HTML with inline styles only: no frontend toolchain, no @@ -506,11 +524,12 @@ credential material. ## Env var reference -Canonical compose keys live in [`compose.env.example`](../compose.env.example). -Host `bun run dev` uses [`.env.example`](../.env.example) (same gateway names; -different defaults for host networking). Keep `POSTGRES_*` and -`SPROUT_PREVIEW_POSTGRES_URL` in sync in `compose.env`; do not synthesize the -DSN from the raw password in YAML. +Set gateway env from this table. Canonical compose keys live in +[`compose.env.example`](../compose.env.example). Host `bun run dev` uses +[`.env.example`](../.env.example) (same gateway names; different defaults +for host networking). Keep `POSTGRES_*` and +`SPROUT_PREVIEW_POSTGRES_URL` in sync in `compose.env`; do not synthesize +the DSN from the raw password in YAML. ### Compose project (`compose.env`) @@ -560,16 +579,16 @@ DSN from the raw password in YAML. Forge kind is chosen **per repo** from the canonical URL (`github.com` / `gitlab.com`) or `SPROUT_FORGE_HOSTS` — not a gateway-wide forge switch. -Registry: the deploy request carries a fully-qualified `app_image`; there is no -separate registry-host env. Host `docker login` does not help gateway-initiated -Engine API pulls — set registry auth when images are private. Malformed -`SPROUT_REGISTRY_AUTHS_JSON` fails at boot. +Registry: the deploy request carries a fully-qualified `app_image`; there +is no separate registry-host env. Host `docker login` does not help +gateway-initiated Engine API pulls — set registry auth when images are +private. Malformed `SPROUT_REGISTRY_AUTHS_JSON` fails at boot. ### Optional gateway tuning (host `.env` / non-compose) -Compose pins `SPROUT_PORT=7331` inside the container. These tuning knobs are in -`.env.example` for host runs (and may be added to compose `environment:` if you -need non-defaults): +Compose pins `SPROUT_PORT=7331` inside the container. These tuning knobs +are in `.env.example` for host runs (and may be added to compose +`environment:` for non-defaults): | Variable | Default | |---|---| @@ -587,26 +606,26 @@ need non-defaults): | `SPROUT_POSTGRES_MAX_CONNECTIONS` | `off` (`100` in the compose example) | `SPROUT_SWEEP_CRON` is a cron expression (local gateway time); an invalid -value fails gateway boot. The first sweep pass lands on the next boundary — -never at boot. +value fails gateway boot. The first sweep pass lands on the next boundary +— never at boot. Unbounded by default: with TTL/idle/caps all `off` the gateway keeps every -running preview until PR close and logs a loud boot warning. Rows that never -complete a deploy are still collected by the legacy creation-age bound -(`SPROUT_TTL_HOURS`). The self-serve compose -stack sets `SPROUT_PREVIEW_TTL=7d` so a new operator is safe without -configuring anything; an existing deployment upgrades with no new required -env and identical behaviour apart from that warning. +running preview until PR close and logs a loud boot warning. Rows that +never complete a deploy are still collected by the legacy creation-age +bound (`SPROUT_TTL_HOURS`). The self-serve compose stack sets +`SPROUT_PREVIEW_TTL=7d` so a new operator is safe without configuring +anything; an existing deployment upgrades with no new required env and +identical behaviour apart from that warning. ### Preview caps and connection budget -Caps fail fast and loudly: exceeding `SPROUT_MAX_PREVIEWS` or -`SPROUT_MAX_PREVIEWS_PER_REPO` fails the deploy with `preview_limit_reached` -naming the cap, its value, and the current count — no container and no -database is created. The sweep logs over-cap state but never deletes to fix -it; close a PR or raise the cap. +Enforce caps instead of discovering overload afterwards. Exceeding +`SPROUT_MAX_PREVIEWS` or `SPROUT_MAX_PREVIEWS_PER_REPO` fails the deploy +with `preview_limit_reached` naming the cap, its value, and the current +count — no container and no database is created. The sweep logs over-cap +state but never deletes to fix it; close a PR or raise the cap. -Connection arithmetic the gateway enforces instead of discovering afterwards: +Connection arithmetic the gateway enforces: `projected = (active previews + 1) × SPROUT_PREVIEW_MAX_DB_CONNECTIONS` against `SPROUT_POSTGRES_MAX_CONNECTIONS`. A deploy beyond the projection @@ -617,18 +636,20 @@ silently enforcing nothing. Measured case: each preview app holds two long-lived pools at default size, so ~12 idle connections per preview; ~8 previews fill a 100-slot Postgres instance with zero traffic (`remaining connection slots are reserved for -SUPERUSER`, SQLSTATE 53300). Set `SPROUT_PREVIEW_MAX_DB_CONNECTIONS=12` and -`SPROUT_POSTGRES_MAX_CONNECTIONS=100` to enforce that shape. The figure is -only accurate for apps that honour the injected cap — a repo that raises its -pool sizes bypasses the budget, and the gateway must not assume otherwise. -Raising `max_connections` is headroom, not a bound. +SUPERUSER`, SQLSTATE 53300). Set `SPROUT_PREVIEW_MAX_DB_CONNECTIONS=12` +and `SPROUT_POSTGRES_MAX_CONNECTIONS=100` to enforce that shape. The figure +is only accurate for apps that honour the injected cap — a repo that raises +its pool sizes bypasses the budget, and the gateway must not assume +otherwise. Raising `max_connections` is headroom, not a bound. For HTTPS behind an external Traefik, set entrypoints (and optionally -certresolver) to that proxy's names. Example Coolify-shaped values (operator -choice, not sprout defaults): `SPROUT_TRAEFIK_ENTRYPOINTS=https` and -`SPROUT_TRAEFIK_CERTRESOLVER=letsencrypt` (HTTP-01, or the same name after a -DNS-01 convert — see [Wildcard preview certificate (DNS-01)](#wildcard-preview-certificate-dns-01); -use `letsencrypt-dns` only on the coexistence path). +certresolver) to that proxy's names. Example Coolify-shaped values +(operator choice, not sprout defaults): +`SPROUT_TRAEFIK_ENTRYPOINTS=https` and +`SPROUT_TRAEFIK_CERTRESOLVER=letsencrypt` (HTTP-01, or the same name after +a DNS-01 convert — see [Wildcard preview certificate +(DNS-01)](#wildcard-preview-certificate-dns-01); use `letsencrypt-dns` only +on the coexistence path). For SSO via Traefik forwardAuth (e.g. VoidAuth), set both `SPROUT_TRAEFIK_MIDDLEWARES=voidauth` and @@ -636,18 +657,19 @@ For SSO via Traefik forwardAuth (e.g. VoidAuth), set both ## Install telemetry -The published image reports anonymous installation facts and deploy -outcomes to a maintainer-run endpoint — on by default, one variable to stop -(`SPROUT_TELEMETRY=off`, or `DO_NOT_TRACK=1`). Full schema and off switches: +Leave the defaults unless the fleet opts out. The published image reports +anonymous installation facts and deploy outcomes to a maintainer-run +endpoint — on by default, one variable to stop (`SPROUT_TELEMETRY=off`, +or `DO_NOT_TRACK=1`). Schema, payloads, and off switches: [Telemetry](telemetry.md). -- `SPROUT_TELEMETRY` is the on/off knob (`on`/`off`, `1`/`0`, `true`/`false`, - `yes`/`no`, case-insensitive; anything else fails boot). +- `SPROUT_TELEMETRY` is the on/off knob (`on`/`off`, `1`/`0`, + `true`/`false`, `yes`/`no`, case-insensitive; anything else fails boot). - `SPROUT_TELEMETRY_ENDPOINT` + `SPROUT_TELEMETRY_AUTH` are the destination pair: both set means reporting is active, exactly one set fails boot, neither set means nothing is sent. - A from-source build carries no destination (the `Dockerfile` `ARG` - defaults are empty), so it reports nothing until you set the pair. To opt + defaults are empty), so it reports nothing until the pair is set. To opt a source build in, pass the same two variables as build args: ```bash @@ -669,38 +691,41 @@ separate opt-in feature (`SPROUT_OTLP_*`, #262) — never both in one knob. **Fresh Postgres, zero SQL.** Point `SPROUT_PREVIEW_POSTGRES_URL` at a new instance whose admin has `CREATEROLE` (or is superuser), set `SPROUT_PG_USER` / `SPROUT_PG_PASSWORD`, start the gateway — no -`CREATE ROLE`, no init scripts, no compose one-shot. Compose no longer ships -an `ensure-preview-role` service; the gateway owns role ensure on boot -(and again before each `CREATE DATABASE`). +`CREATE ROLE`, no init scripts, no compose one-shot. Compose no longer +ships an `ensure-preview-role` service; the gateway owns role ensure on +boot (and again before each `CREATE DATABASE`). -If `SPROUT_PG_USER` is missing the gateway runs -`CREATE ROLE … LOGIN PASSWORD …`; if present it -`ALTER ROLE … LOGIN PASSWORD …` so password rotation is -`change SPROUT_PG_PASSWORD` + restart. Role names must match the lowercase -`SAFE_ROLE` guard. Without `CREATEROLE` (or superuser), boot fails with a -clear error instead of a later `CREATE DATABASE … OWNER` failure. +If `SPROUT_PG_USER` is missing the gateway runs `CREATE ROLE … LOGIN +PASSWORD …`; if present it `ALTER ROLE … LOGIN PASSWORD …` so password +rotation is `change SPROUT_PG_PASSWORD` + restart. Role names must match +the lowercase `SAFE_ROLE` guard. Without `CREATEROLE` (or superuser), boot +fails with a clear error instead of a later `CREATE DATABASE … OWNER` +failure. An optional manual helper remains at -`deploy/postgres/ensure-preview-role.sh` for pre-provisioning without starting -the gateway. +`deploy/postgres/ensure-preview-role.sh` for pre-provisioning without +starting the gateway. -The gateway preview-db module grants that role ownership when it creates each -`sprout__pr` database, and — for `dual` previews (`db.roles`, -see [Adopting a repo](adopting-a-repo.md)) — also creates a per-DB restricted companion +The gateway preview-db module grants that role ownership when it creates +each `sprout__pr` database, and — for `dual` previews +(`db.roles`, see [Adopting a +repo](adopting-a-repo.md)) — also creates a per-DB restricted companion LOGIN (`_app`) with `CONNECT` + schema `USAGE`. Containers receive -owner credentials as `PGUSER`/`PGPASSWORD` and, in `dual` only, companion credentials as -`PGAPPUSER`/`PGAPPPASSWORD` (remappable via `preview.env` — see [Adopting a repo](adopting-a-repo.md)). Teardown drops the database then the companion role when one was provisioned. +owner credentials as `PGUSER`/`PGPASSWORD` and, in `dual` only, companion +credentials as `PGAPPUSER`/`PGAPPPASSWORD` (remappable via `preview.env` — +see [Adopting a repo](adopting-a-repo.md)). Teardown drops the database +then the companion role when one was provisioned. ## Bootstrap admin token -On every boot where the raw bearer is known (pinned `SPROUT_ADMIN_TOKEN`, or -first-time auto-generate), the gateway writes it to `SPROUT_ADMIN_TOKEN_PATH` -(mode `0600`). Compose and the gateway image publish -`SPROUT_ADMIN_TOKEN_PATH=/data/admin-token` so the embedded CLI does not guess -from the SQLite path. Later boots with a hashed-only admin require that file to -still be readable (missing/empty → boot fails). When `SPROUT_ADMIN_TOKEN` is -unset or blank on first generate, the raw token is also printed once in gateway -logs: +Persist the admin bearer across boots. On every boot where the raw bearer +is known (pinned `SPROUT_ADMIN_TOKEN`, or first-time auto-generate), the +gateway writes it to `SPROUT_ADMIN_TOKEN_PATH` (mode `0600`). Compose and +the gateway image publish `SPROUT_ADMIN_TOKEN_PATH=/data/admin-token` so +the embedded CLI does not guess from the SQLite path. Later boots with a +hashed-only admin require that file to still be readable (missing/empty → +boot fails). When `SPROUT_ADMIN_TOKEN` is unset or blank on first +generate, the raw token is also printed once in gateway logs: ```bash docker compose --env-file compose.env logs gateway | grep -i admin @@ -708,10 +733,10 @@ docker compose --env-file compose.env logs gateway | grep -i admin Create a **deploy token** for each adopting repo by exec'ing the CLI already in the gateway image (`SPROUT_URL` defaults to `http://127.0.0.1:7331`). -Against that loopback URL the CLI resolves a bearer as: -`SPROUT_TOKEN` → `SPROUT_ADMIN_TOKEN` → `SPROUT_ADMIN_TOKEN_PATH` (default -`admin-token`; `/data/admin-token` in-container), so bare `docker exec` works -for pinned and auto-generated admin tokens: +Against that loopback URL the CLI resolves a bearer as: `SPROUT_TOKEN` → +`SPROUT_ADMIN_TOKEN` → `SPROUT_ADMIN_TOKEN_PATH` (default `admin-token`; +`/data/admin-token` in-container), so bare `docker exec` works for pinned +and auto-generated admin tokens: ```bash # Primary path: CLI embedded in the gateway image (no SPROUT_TOKEN needed) @@ -729,9 +754,9 @@ Store the deploy token in the adopting repo's CI secrets as `SPROUT_TOKEN`. ## Worktree DB (local provisioner) -Parallel agents / herdr worktrees on one machine can clash on shared Postgres -credentials. The CLI provisions an isolated DB + LOGIN role per worktree -**without** talking to the gateway: +Isolate per-worktree Postgres credentials for parallel agents / herdr +worktrees on one machine. The CLI provisions an isolated DB + LOGIN role +per worktree **without** talking to the gateway: ```bash sprout worktree-db provision --slug --env-file --admin-url "$ADMIN_DSN" @@ -746,16 +771,18 @@ sprout worktree-db drop --slug --admin-url "$ADMIN_DSN" underscores). `drop` refuses any name outside the `sprout_wt_` prefix. - Worktree key rule (CLI `--slug`; distinct from adopting-repo **slug**): lowercase, `[^a-z0-9-]` → `-`, collapse runs, max 40 characters. -- `provision` is idempotent: a second run re-ensures the role/DB and rewrites - connection vars in `--env-file` (creates the file if missing; atomic - temp+rename). When `PGPASSWORD` is already present in the env file, that - password is reused so live connections are not rotated. Defaults write - `DATABASE_URL` plus canonical `PGHOST` / `PGPORT` / `PGUSER` / - `PGPASSWORD` / `PGDATABASE`; rename with - `--rename LOGICAL=NAME` (logical keys: `DATABASE_URL`, `PGHOST`, …). +- `provision` is idempotent: a second run re-ensures the role/DB and + rewrites connection vars in `--env-file` (creates the file if missing; + atomic temp+rename). When `PGPASSWORD` is already present in the env + file, that password is reused so live connections are not rotated. + Defaults write `DATABASE_URL` plus canonical `PGHOST` / `PGPORT` / + `PGUSER` / `PGPASSWORD` / `PGDATABASE`; rename with `--rename + LOGICAL=NAME` (logical keys: `DATABASE_URL`, `PGHOST`, …). ## Upgrade / redeploy +Pull latest operator files, then rebuild + recreate: + ```bash # Pull latest operator files, then rebuild + recreate git pull @@ -763,20 +790,23 @@ docker compose --env-file compose.env up -d --build curl -sf http://127.0.0.1:7331/healthz ``` -- Changing `SPROUT_PG_PASSWORD` (or other role env) → restart the gateway so - boot re-syncs the preview login. +- Changing `SPROUT_PG_PASSWORD` (or other role env) → restart the gateway + so boot re-syncs the preview login. - Changing network names or moving to the external overlay → recreate the - gateway against the new networks (`up -d --build gateway` with the overlay - files as needed). + gateway against the new networks (`up -d --build gateway` with the + overlay files as needed). - Upgrading from legacy `preview-buddy*` / `pb*` / `prev_*` defaults is a - **wipe-and-redeploy**: tear down volumes, networks, and control-plane state, - then bring the stack up again. There is no in-place migrator. Pin - `SPROUT_TRAEFIK_NETWORK` / `SPROUT_POSTGRES_NETWORK` / `SPROUT_STATE_DB_PATH` / - `SPROUT_ADMIN_TOKEN_PATH` (and role env vars) only when you intentionally use - non-default names (external Traefik, Coolify, etc.). + **wipe-and-redeploy**: tear down volumes, networks, and control-plane + state, then bring the stack up again. There is no in-place migrator. Pin + `SPROUT_TRAEFIK_NETWORK` / `SPROUT_POSTGRES_NETWORK` / + `SPROUT_STATE_DB_PATH` / `SPROUT_ADMIN_TOKEN_PATH` (and role env vars) + only when intentionally using non-default names (external Traefik, + Coolify, etc.). ## Teardown +Remove the stack: + ```bash docker compose --env-file compose.env down # docker compose --env-file compose.env down -v # also drops SQLite + Postgres volumes @@ -791,10 +821,10 @@ docker compose -f docker-compose.yml -f docker-compose.external.yml \ ## Smoke checklist -`POSTGRES_PASSWORD` and the password embedded in `SPROUT_PREVIEW_POSTGRES_URL` are -two spellings of one secret — keep them identical in `compose.env`. Drift is a -known risk of this dual-write; catch it with the login vs redacted-URL check -below. +`POSTGRES_PASSWORD` and the password embedded in +`SPROUT_PREVIEW_POSTGRES_URL` are two spellings of one secret — keep them +identical in `compose.env`. Drift is a known risk of this dual-write; catch +it with the login vs redacted-URL check below. | Check | Command | |---|---| diff --git a/docs/previews.md b/docs/previews.md index eaef42c9..92370efd 100644 --- a/docs/previews.md +++ b/docs/previews.md @@ -1,16 +1,18 @@ # Previews -One pull request gets at most one preview per adopting repo: its preview -database and, when configured, its preview app container plus optional -companion **service** containers that share the same preview database. -This page owns the lifecycle: database providers, seeding, services, mail. +Run one preview per pull request per adopting repo: its preview database +and, when configured, its preview app container plus optional companion +**service** containers sharing the same preview database. Each section +below is one task. -- Manifest keys? See [Adopting a repo](adopting-a-repo.md). -- CI wiring? See [CI integration](ci-integration.md). -- Operator side (networks, sweep config)? See [Operator deploy](operator-deploy.md). +- Manifest keys: [Adopting a repo](adopting-a-repo.md). +- CI wiring: [CI integration](ci-integration.md). +- Operator side (networks, sweep config): [Operator deploy](operator-deploy.md). ## Preview lifecycle +Follow one preview from open to close: + `create → migrate (app) → seed (optional) → hand over → drop`: - PR opened / synchronize → `sprout deploy`: `CREATE DATABASE` @@ -25,8 +27,8 @@ Synchronize re-deploys keep the same database; only a reset wipes it. ## Preview TTL and idle teardown -A preview that nobody closes still disappears on a bound. Two manifest keys, -both optional durations (`30m`, `2h`, `7d`) or `off`: +Bound a preview that nobody closes. Set two optional manifest keys, +durations (`30m`, `2h`, `7d`) or `off`: ```yaml preview: @@ -61,8 +63,8 @@ a deploy; a running preview with both bounds off has no deadline. ## Preview database roles (`db.roles`) -Postgres previews run with one (`single`) or two (`dual`) database -LOGINS. `single` is the owner only; `dual` adds the per-database +Pick one (`single`) or two (`dual`) database LOGINS for a Postgres +preview. `single` is the owner only; `dual` adds the per-database restricted companion (`_app`, injected as `PGAPPUSER` / `PGAPPPASSWORD`, remappable via `preview.env`). The default is derived — `dual` when `preview.env` remaps a companion key, else @@ -72,8 +74,9 @@ contradiction guard, and the RLS recipe live in ## Service images: merge, leave, clear, lifecycle -Static `image` in yaml pins the image; `--service name=image` overlays it -— every service needs an image after merge. Omitting `--service` leaves +Combine static manifest images with deploy-time `--service` flags. Static +`image` in yaml pins the image; `--service name=image` overlays it — +every service needs an image after merge. Omitting `--service` leaves companions in place; `--clear-services` removes all (cannot combine with `--service`). An empty list is rejected (omit the key, or `--clear-services`). Reseed bodies carry no service list, so companions @@ -93,12 +96,12 @@ services start. There is no per-service health poll in this release. ## Multi-image previews (app + services) -Routing examples only — merge, lifecycle, and health-gate rules live above. +Share one preview database across long-lived containers (API + worker, web ++ secondary service). Routing examples only — merge, lifecycle, and +health-gate rules live above. -Full-stack previews often need more than one long-lived container sharing the -same preview database (API + worker, web + secondary service, etc.). With -`sprout ci preview` pass repeatable `--service name=image`. Low-level deploys -use the same flag: +With `sprout ci preview` pass repeatable `--service name=image`. Low-level +deploys use the same flag: ```bash sprout deploy -i "$APP_IMAGE" \ @@ -108,11 +111,10 @@ sprout deploy -i "$APP_IMAGE" \ ### Per-MR service tags (consumer builds, manifest references) -A literal ref can't follow the merge request, so a pipeline that builds a -per-MR image for a second surface (a separate SPA besides the app) pins the -tag in the manifest and lets the CLI resolve it at deploy time. The -consumer's pipeline builds and pushes the image; nothing in the component -builds service images. +Pin a per-MR image for a second surface (a separate SPA besides the app) +in the manifest and let the CLI resolve it at deploy time. The consumer's +pipeline builds and pushes the image; nothing in the component builds +service images. ```yaml preview: @@ -137,9 +139,9 @@ literal refs only. ### Routing (optional) -Without routing metadata, a service is internal-only (reachable on the Docker -networks, no Traefik router). To expose a service, declare it under -`preview.services` in `.sprout.yaml`: +Expose a service by declaring it under `preview.services` in +`.sprout.yaml`. Without routing metadata, a service is internal-only +(reachable on the Docker networks, no Traefik router): ```yaml slug: myapp @@ -194,14 +196,13 @@ sprout deploy -i "$APP_IMAGE" \ #### Preview labels: adopter-supplied container labels - -`preview.labels` adds literal `key: value` string labels to the app -container **and** every service container; `preview.services[].labels` -adds to that service container only (same key at both levels resolves to -the per-service value). Labels are orthogonal to routing, so internal -(unrouted) services receive them too. The one-shot seed container never -carries adopter labels. Values are literal only — no `{pr_id}` -interpolation. +Label preview containers for tooling outside the gateway. `preview.labels` +adds literal `key: value` string labels to the app container **and** every +service container; `preview.services[].labels` adds to that service +container only (same key at both levels resolves to the per-service +value). Labels are orthogonal to routing, so internal (unrouted) services +receive them too. The one-shot seed container never carries adopter +labels. Values are literal only — no `{pr_id}` interpolation. Worked example — attach the `api` service to a Traefik middleware defined outside the gateway (file provider) and tag every preview container for @@ -233,7 +234,10 @@ errors in the existing style. ## Seed run order and resume -1. App container starts (entrypoint waits for Postgres, runs migrations, serves). +Run "migrate in the app, then seed" with no API-code changes: + +1. App container starts (entrypoint waits for Postgres, runs migrations, + serves). 2. Gateway polls `health.path` on the Postgres-network container IP until `health.expect` or `health.timeout`. 3. **After healthy:** if a seed image was provided and this PR has never @@ -250,41 +254,33 @@ On later synchronize deploys, seeding is skipped only when the incoming seed image matches the last successful one. Same image + hostname: seed-only (no app container replace). A changed seed image alone re-runs the seed without replacing the app container. Image or hostname change -still replaces the app, then runs seed after -healthy when `seeded_at` is unset, the seed image changed, or `--reseed` -was passed. Seed -wall-clock is the gateway env `SPROUT_SEED_TIMEOUT` (seconds, default -`180`, applied internally as `seedTimeoutMs`); health timeout is separate -and never starts the seed. Seed failure outcomes (exit non-zero, timeout, -Docker/ops error → `500 seed_failed`, app stays up and routable, -`seeded_at` unset; health timeout → `health_timeout`, app removed, seed -never started) and the resume rule (failed/crash-mid-seed with same image -+ hostname: redeploy with `-s` resumes the seed only; without -`seed_image`, resume returns `422 +still replaces the app, then runs seed after healthy when `seeded_at` is +unset, the seed image changed, or `--reseed` was passed. Seed wall-clock +is the gateway env `SPROUT_SEED_TIMEOUT` (seconds, default `180`, applied +internally as `seedTimeoutMs`); health timeout is separate and never +starts the seed. Seed failure outcomes (exit non-zero, timeout, Docker/ops +error → `500 seed_failed`, app stays up and routable, `seeded_at` unset; +health timeout → `health_timeout`, app removed, seed never started) and +the resume rule (failed/crash-mid-seed with same image + hostname: redeploy +with `-s` resumes the seed only; without `seed_image`, resume returns `422 seed_image_required_to_resume_seeding`) are covered under [Troubleshooting](troubleshooting.md). ## After-healthy hook (seed image) -Run-order flow and low-level examples only — ordering, resume, timeout, and -failure outcomes are contract above and in -[Troubleshooting](troubleshooting.md). - -The gateway's only post-startup timing hook is **after-healthy**: once the -preview app passes `health.expect`, an optional **seed image** runs. That is -how you sequence "migrate in the app, then seed" with zero API-code changes. - -With the component you declare it once in `.sprout.yaml` and -never pass `-s` in CI — `sprout ci preview` builds + pushes the seed image -and deploys with it. Without `seed.inputs` the tag is commit-scoped -(`-seed`) and the image is rebuilt on every run; with explicit -`seed.inputs` the tag is `seed-` content-addressed over those -inputs (list every COPY source the seed image depends on). When the -content-addressed tag already exists in the -registry the build + push is skipped (`seed image reused: ` in the job -log) and the existing image deploys; a failed or unsupported registry check -rebuilds instead of skipping. The low-level equivalent is -`sprout deploy -i … -s …` with +Declare the seed once in `.sprout.yaml` and never pass `-s` in CI — +`sprout ci preview` builds + pushes the seed image and deploys with it. +Ordering, resume, timeout, and failure outcomes are contract in the section +above and in [Troubleshooting](troubleshooting.md). + +Without `seed.inputs` the tag is commit-scoped (`-seed`) and the +image is rebuilt on every run; with explicit `seed.inputs` the tag is +`seed-` content-addressed over those inputs (list every COPY +source the seed image depends on). When the content-addressed tag already +exists in the registry the build + push is skipped +(`seed image reused: ` in the job log) and the existing image +deploys; a failed or unsupported registry check rebuilds instead of +skipping. The low-level equivalent is `sprout deploy -i … -s …` with `--seed-env` / `--seed-arg`: ```bash @@ -307,8 +303,8 @@ sprout deploy -i "$APP_IMAGE" -s "$SEED_IMAGE" --reseed ## SQLite previews -For stacks that run on SQLite instead of Postgres, set `db.provider` — -`sprout ci preview` picks it up from `.sprout.yaml` (no new flag): +Serve a SQLite stack from a preview. Set `db.provider` — `sprout ci +preview` picks it up from `.sprout.yaml` (no new flag): ```yaml slug: myapp @@ -323,39 +319,39 @@ db: ``` Env keys: the gateway injects exactly one connection variable, -`DATABASE_URL=file:/` (remap replaces the name, no -dual alias). `PG*` remaps are rejected for SQLite previews, and -`DATABASE_URL` is rejected for Postgres ones — both at manifest parse -and at the gateway deploy route. +`DATABASE_URL=file:/` (remap replaces the name, no dual +alias). `PG*` remaps are rejected for SQLite previews, and `DATABASE_URL` +is rejected for Postgres ones — both at manifest parse and at the gateway +deploy route. Volume and seed behaviour: bring-up creates one named Docker volume per preview (`sprout--pr--sqlite`) mounted at `db.path` in the app, companion-service, and seed containers (seed inputs still own the -fixtures; `--reseed` semantics are unchanged). An app-image replace -keeps the volume, so preview data survives; teardown removes the -containers and the volume on the same paths that drop a Postgres -database today. Health, TTL, sweep, and teardown are otherwise unchanged. +fixtures; `--reseed` semantics are unchanged). An app-image replace keeps +the volume, so preview data survives; teardown removes the containers and +the volume on the same paths that drop a Postgres database today. Health, +TTL, sweep, and teardown are otherwise unchanged. Hand-rolled migration notes (moving an app from a Postgres preview to a SQLite one): point the app at the injected `DATABASE_URL` instead of the `PG*` set (SQLite opens the file directly — no host, port, user, or password); run file-level migrations at container startup as before (the file persists on the volume across replaces); keep companion `PGAPP*` -assumptions out of the SQLite path (there is no restricted role — the -file is the database; likewise a `single` Postgres preview injects no -`PGAPP*`). There is no gateway tooling that copies a -Postgres preview into a SQLite volume in this release. +assumptions out of the SQLite path (there is no restricted role — the file +is the database; likewise a `single` Postgres preview injects no +`PGAPP*`). There is no gateway tooling that copies a Postgres preview into +a SQLite volume in this release. -Operators: a gateway that only serves SQLite previews needs no Postgres -env at all (`SPROUT_PREVIEW_POSTGRES_URL`, `SPROUT_PG_HOST/USER/PASSWORD`, +Operators: a gateway that only serves SQLite previews needs no Postgres env +at all (`SPROUT_PREVIEW_POSTGRES_URL`, `SPROUT_PG_HOST/USER/PASSWORD`, `SPROUT_POSTGRES_NETWORK` are required only for `postgres` deploys). A -`postgres` deploy on such a gateway fails fast with -`postgres_not_configured`, naming the repo and the missing variables. +`postgres` deploy on such a gateway fails fast with `postgres_not_configured`, +naming the repo and the missing variables. ## No-database previews -For apps with no database — static or SSR frontends, apps whose data -lives behind an external API, worker-only services — set `db.provider` to +Serve an app with no database — static or SSR frontends, apps whose data +lives behind an external API, worker-only services. Set `db.provider` to `none`. `sprout ci preview` picks it up from `.sprout.yaml` (no new flag): ```yaml @@ -378,8 +374,7 @@ manifest parse and at the gateway deploy route: a `seed:` block (`seed requires db.provider postgres or sqlite (db.provider is none)`) and any `preview.env` database-key remap (`preview.env.PGHOST requires db.provider postgres`). `sprout ci reseed`, `sprout deploy -s …`, and -`--reseed` fail fast with the seed error on a `none` repo instead of a -5xx. +`--reseed` fail fast with the seed error on a `none` repo instead of a 5xx. Operators: a gateway whose repos are all `none` boots with no `SPROUT_*PG*` / `SPROUT_POSTGRES_NETWORK` set; with any `postgres` repo @@ -390,12 +385,13 @@ before the row is rewritten, so no resource strands. ## Preview app-data volumes -A preview's writable container filesystem is thrown away on every -replace (any push) — but a per-PR Postgres database survives, because it -lives on the shared instance. Anything the app writes at runtime into -its container is lost while its database rows survive, so the app can -boot into an inconsistent state no CI signal shows. `preview.volumes` -opts container paths into the same lifetime the database already has: +Keep files the app writes at runtime across synchronize re-deploys. A +preview's writable container filesystem is thrown away on every replace +(any push) — but a per-PR Postgres database survives, because it lives on +the shared instance. Anything the app writes at runtime into its container +is lost while its database rows survive, so the app can boot into an +inconsistent state no CI signal shows. `preview.volumes` opts container +paths into the same lifetime the database already has: ```yaml slug: myapp @@ -406,9 +402,8 @@ preview: ``` Each entry gets one named per-preview Docker volume -(`sprout--pr--data-`, indexed in manifest order), mounted -at that path in the app, companion-service, and seed containers. The -contract: +(`sprout--pr--data-`, indexed in manifest order), mounted at +that path in the app, companion-service, and seed containers. The contract: - **Replace keeps.** Synchronize re-deploys mount the same volume, so files written by the app are still readable after a push. @@ -425,26 +420,26 @@ paths, `/` itself, `..` segments, duplicates, nested entries (one path equal to or inside another), and any entry equal to or nested inside the SQLite `db.path` (or vice versa) are each rejected. -Volume names are index-keyed, so treat the list as append-only: -appending an entry is safe, but reordering or removing one silently -re-points an existing volume's contents at a different mount path (and -strands the tail volume). Shrinking the list does not delete the -surplus volumes either — they linger until teardown or the sweep's -orphan pass removes them. +Volume names are index-keyed, so treat the list as append-only: appending +an entry is safe, but reordering or removing one silently re-points an +existing volume's contents at a different mount path (and strands the tail +volume). Shrinking the list does not delete the surplus volumes either — +they linger until teardown or the sweep's orphan pass removes them. Writability rule: a fresh named volume starts empty. Docker copies the -image's content and ownership at that path into the volume only when -the path exists in the image. An app running as a non-root user must -therefore create the path in the image with the right ownership (or -chown it in the entrypoint) — otherwise the runtime user cannot write -the root-owned directory it gets. +image's content and ownership at that path into the volume only when the +path exists in the image. An app running as a non-root user must therefore +create the path in the image with the right ownership (or chown it in the +entrypoint) — otherwise the runtime user cannot write the root-owned +directory it gets. ## Email from a preview -Previews can send mail through a gateway-configured Mailpit. The operator -owns the Mailpit instance and the `SPROUT_MAIL_*` gateway variables (see -[Operator deploy](operator-deploy.md#preview-mail-mailpit)); the adopter owns -only the `mail:` block and the `preview.env` remap. Mail works on any +Send mail from a preview through a gateway-configured Mailpit. The +operator owns the Mailpit instance and the `SPROUT_MAIL_*` gateway +variables (see +[Operator deploy](operator-deploy.md#preview-mail-mailpit)); the adopter +owns only the `mail:` block and the `preview.env` remap. Mail works on any `db.provider` — `postgres`, `sqlite`, and `none` — and needs no provisioning, health gate, or lifecycle: the gateway only injects env and joins a network. @@ -459,9 +454,9 @@ MAILSECURE (only when true, as the string "true") MAILUIURL (only when the operator configures an inbox URL) ``` -`preview.env` renames these exactly like the `PG*` set: unmapped keys -keep their canonical name; a remap replaces the name (no dual alias). -Gateway mail keys win over colliding `preview.app_env` keys — do not put +`preview.env` renames these exactly like the `PG*` set: unmapped keys keep +their canonical name; a remap replaces the name (no dual alias). Gateway +mail keys win over colliding `preview.app_env` keys — do not put `MAILHOST` or a remapped name into `SPROUT_APP_ENV`. Worked remap for an app that speaks `SMTP_*`: @@ -478,14 +473,14 @@ preview: The entrypoint must read the adopter names (`SMTP_HOST`, …). -The `mail:` block has three postures. Omitted is opportunistic: mail env -is injected when the gateway configures it and silently skipped when it -does not. Explicit `mail: enabled` requires mail: on a gateway without -`SPROUT_MAIL_*` the deploy fails fast with `mail_not_configured` -(`repo declares mail enabled but the gateway has no mail -configured: missing SPROUT_MAIL_HOST` — same wording from the CLI and -the gateway). `mail: none` opts out: no mail env is injected for that -repo, and the preview never shows the `Mailbox:` note line. +The `mail:` block has three postures. Omitted is opportunistic: mail env is +injected when the gateway configures it and silently skipped when it does +not. Explicit `mail: enabled` requires mail: on a gateway without +`SPROUT_MAIL_*` the deploy fails fast with `mail_not_configured` (`repo + declares mail enabled but the gateway has no mail configured: missing +SPROUT_MAIL_HOST` — same wording from the CLI and the gateway). `mail: none` +opts out: no mail env is injected for that repo, and the preview never shows +the `Mailbox:` note line. ```yaml mail: enabled # require mail; fail fast without a configured gateway @@ -495,15 +490,14 @@ mail: from: "noreply+{pr_id}@preview.invalid" # send-from override (below) ``` -An adopter can enable mail with this guide alone: keep the `mail:` block -omitted (or `enabled`), remap `preview.env` onto the app's SMTP names, -deploy, and look for the preview's From address in the inbox linked from -the MR note (`Mailbox:` line). +To enable mail: keep the `mail:` block omitted (or `enabled`), remap +`preview.env` onto the app's SMTP names, deploy, and look for the preview's +From address in the inbox linked from the MR note (`Mailbox:` line). ### Which preview did this mail come from? -Every preview sends from its own address so testers can tell deployments -apart in the shared inbox: +Tell deployments apart in the shared inbox by the per-preview From +address: - `MAILFROM` defaults to `-pr@`, e.g. `myapp-pr42@preview.invalid`. `MAILREPLYTO` carries the same address. @@ -512,9 +506,9 @@ apart in the shared inbox: - The from-domain is the operator's `SPROUT_MAIL_FROM_DOMAIN` (default `preview.invalid`, a reserved suffix that can never deliver real mail). -`mail.from` overrides the address with a `{pr_id}` template using the -same grammar as `preview.hostname` — it must contain `{pr_id}`, support -no other placeholder, contain no whitespace, and read as an address once +`mail.from` overrides the address with a `{pr_id}` template using the same +grammar as `preview.hostname` — it must contain `{pr_id}`, support no +other placeholder, contain no whitespace, and read as an address once `{pr_id}` is substituted. An app that must send from its own convention points that convention at the preview-identifying address: @@ -534,39 +528,38 @@ naming the problem, e.g. `must contain {pr_id}`). The MR note and `sprout list` show what to filter for: the note gains `- Mail from:
` alongside `- Mailbox: `, `sprout -deploy` / `sprout ci preview` print `mail_from=` (and `mailbox_url=`), -and `sprout list` includes `mail_from`, `mail_from_name`, and -`mailbox_url` for previews that received mail env. Inbox recipe: search -the Mailpit UI for the preview's From address, or query the Mailpit API +deploy` / `sprout ci preview` print `mail_from=` (and `mailbox_url=`), and +`sprout list` includes `mail_from`, `mail_from_name`, and `mailbox_url` for +previews that received mail env. Inbox recipe: search the Mailpit UI for +the preview's From address, or query the Mailpit API (`GET /api/v1/messages`) and keep messages whose `From.Address` equals that address. ### Shared inbox All previews on one gateway write into one mailbox — there is no -per-preview isolation. Testers tell mail apart by the recipient / From / -subject convention above, not by separate inboxes. If previews must not -see each other's mail at all, run a second Mailpit plus a second gateway -pointed at it; one gateway holds exactly one mail configuration. +per-preview isolation. Tell mail apart by the recipient / From / subject +convention above, not by separate inboxes. If previews must not see each +other's mail at all, run a second Mailpit plus a second gateway pointed at +it; one gateway holds exactly one mail configuration. ## Preview access -Previews are open by design: anyone who knows the hostname reaches them. -Two opt-in postures gate one preview without touching the app, declared +Gate one preview without touching the app. Previews are open by design: +anyone who knows the hostname reaches them. Two opt-in postures, declared with `preview.auth` (`none` default; `basic` or `link`; see [Adopting a repo](adopting-a-repo.md#manifest-keys-sproutyaml)): -- `basic` — one username/password per preview, generated by the gateway - at deploy. Anonymous requests get `401`; the documented credential - gets `200`. The credential is printed by `sprout access ` and in - the CI note, never in gateway logs. +- `basic` — one username/password per preview, generated by the gateway at + deploy. Anonymous requests get `401`; the documented credential gets + `200`. The credential is printed by `sprout access ` and in the CI + note, never in gateway logs. - `link` — shareable, revocable, host-scoped links. `sprout access ` - mints a link (default expiry `7d`, `--expires 12h` to override); - opening it on the preview's own host sets a cookie for that host only. - The same cookie on another preview's host is rejected. `sprout access - --revoke` rotates the preview's secret and invalidates - outstanding links with no redeploy, no container recreation, no - gateway restart. + mints a link (default expiry `7d`, `--expires 12h` to override); opening + it on the preview's own host sets a cookie for that host only. The same + cookie on another preview's host is rejected. `sprout access + --revoke` rotates the preview's secret and invalidates outstanding links + with no redeploy, no container recreation, no gateway restart. Share-this-preview-with-a-reviewer recipe: @@ -585,25 +578,25 @@ sprout access 42 --revoke --repo https://github.com/org/repo Wire contract: by default the app receives **no identity header at all**. The gate is Traefik middleware plus the gateway — anonymous traffic never -reaches the app container, and authorised traffic arrives unmodified. -(If a future posture propagates identity, it will be recorded alongside -the env grammar decision in the maintainer records.) +reaches the app container, and authorised traffic arrives unmodified. (If a +future posture propagates identity, it will be recorded alongside the env +grammar decision in the maintainer records.) Companion services keep their own protection and are never double-gated: -the gate attaches to the app router only. Reset, reseed, and teardown -keep working with a gate on — they act through the gateway and the forge, -not through the gated preview host. Expiry is checked on every request, -not only when the cookie is set. +the gate attaches to the app router only. Reset, reseed, and teardown keep +working with a gate on — they act through the gateway and the forge, not +through the gated preview host. Expiry is checked on every request, not +only when the cookie is set. Two structural traps, so nobody rebuilds the abandoned shape: 1. There is no Cloudflare-free path from the preview edge to an external auth service — every Cloudflare-mediated path rewrites `X-Forwarded-Host`, so an external gate rebuilt its own URL from the - header, took its self-request shortcut, and authorised everything. - The gate therefore lives in the gateway, which is already on the - preview network; the Traefik `forwardAuth` address is the gateway's - in-network name (`SPROUT_PREVIEW_AUTH_ADDRESS`), never a public URL. + header, took its self-request shortcut, and authorised everything. The + gate therefore lives in the gateway, which is already on the preview + network; the Traefik `forwardAuth` address is the gateway's in-network + name (`SPROUT_PREVIEW_AUTH_ADDRESS`), never a public URL. 2. The basic-auth password is baked into a Traefik label at deploy, so a rotated password applies on the next deploy — only link secrets rotate live. `sprout access --revoke` on a `basic` preview says so. diff --git a/docs/site/assemble.ts b/docs/site/assemble.ts index 26e3d87f..9368d4a0 100644 --- a/docs/site/assemble.ts +++ b/docs/site/assemble.ts @@ -46,7 +46,7 @@ export const docsGroups: PageGroup[] = [ { file: "docs/onboarding-prompt.md", title: "Onboarding prompt", description: "copy-paste agent block (entry point for agents).", role: "Agent", entry: true }, ] }, { name: "Adopting", pages: [ - { file: "docs/adopting-a-repo.md", title: "Adopting a repo", description: "sprout.yaml manifest reference and app entrypoints." }, + { file: "docs/adopting-a-repo.md", title: "Adopting a repo", description: "write .sprout.yaml and the app entrypoint." }, { file: "docs/ci-integration.md", title: "CI integration", description: "GitLab component, GitHub reusable workflow, variables, reset, notes." }, ] }, { name: "Previews", pages: [ @@ -55,8 +55,8 @@ export const docsGroups: PageGroup[] = [ { name: "Operating", pages: [ { file: "docs/operator-deploy.md", title: "Operator deploy", description: "gateway compose stack, Traefik, env reference, admin token." }, { file: "docs/cli-reference.md", title: "CLI reference", description: "every sprout command, debugging, tokens." }, - { file: "docs/troubleshooting.md", title: "Troubleshooting", description: "adopter and operator error catalogue." }, - { file: "docs/telemetry.md", title: "Telemetry", description: "anonymous install reporting and how to stop it." }, + { file: "docs/troubleshooting.md", title: "Troubleshooting", description: "symptom → fix catalogue for adopters and operators." }, + { file: "docs/telemetry.md", title: "Telemetry", description: "what is sent, what it costs, how to stop it." }, { file: "docs/herdr-integration.md", title: "Herdr integration", description: "operator-side review automation." }, ] }, ]; diff --git a/docs/telemetry.md b/docs/telemetry.md index e281c105..454104c0 100644 --- a/docs/telemetry.md +++ b/docs/telemetry.md @@ -1,9 +1,9 @@ # Telemetry Every gateway built as the published image reports anonymous installation -facts and deploy outcomes to a maintainer-run ingest endpoint. It is on by -default and stops with one variable. A build from source reports nothing -until the two destination variables are set. +facts and deploy outcomes to a maintainer-run ingest endpoint. Reporting is +on by default and stops with one variable. A build from source reports +nothing until the two destination variables are set. ```bash SPROUT_TELEMETRY=off # or DO_NOT_TRACK=1 — either one stops all reporting @@ -11,7 +11,7 @@ SPROUT_TELEMETRY=off # or DO_NOT_TRACK=1 — either one stops all reporting ## What is sent -Two events, one fixed schema. Every event carries the same envelope: +Two events share one fixed envelope: | Field | Meaning | |---|---| @@ -87,19 +87,23 @@ means the row never recorded a code: } ``` -Transport: one `POST` per event with a 3s timeout, fire-and-forget. A failed -export never changes a deploy outcome — at most one warning is logged. -There are no retries, no queue, and no batching: losing an anonymous event -is acceptable, changing a deploy's outcome is not. +Transport costs one `POST` per event with a 3s timeout, fire-and-forget. A +failed export never changes a deploy outcome — at most one warning is +logged. There are no retries, no queue, and no batching: losing an +anonymous event is acceptable, changing a deploy's outcome is not. That is +the whole cost: two small JSON posts per deploy plus one per day, with no +effect on the request path when the backend is slow or dead. ## What is never sent Repo id, URL or name; PR id; slug; preview name; hostname; database name; -image reference; container id; DSN; any credential; any value from `appEnv`, -the injected preview env, or the manifest; request or response headers and -bodies; error detail text; stack traces; log lines. The payload is assembled -from a fixed field list in one module — the only free values that travel -are the closed vocabularies `lastError`, `failureFamily` and `plan`. +image reference; container id; DSN; any credential; any value from +`appEnv`, the injected preview env, or the manifest; request or response +headers and bodies; error detail text; stack traces; log lines. The payload +is assembled from a fixed field list in one module — the only free values +that travel are the closed vocabularies `lastError`, `failureFamily` and +`plan`. This closed list is why the channel is safe to leave on: nothing +the adopter types into config, env, or code can reach the payload. ## Stopping it and forgetting @@ -121,8 +125,8 @@ is sent (`on (no destination)` — the usual case for a from-source build). The published image carries the maintainer destination baked in at image build time (`ARG` → `ENV` in the `Dockerfile`, supplied by the release workflow). That value is therefore visible in `docker inspect` for anyone -pulling the image — by construction, so the credential is ingest-only for a -single stream and the collector treats every record as untrusted input. +pulling the image — by construction, so the credential is ingest-only for +a single stream and the collector treats every record as untrusted input. Setting the two variables at runtime overrides what the image carries, so a fork or a company can redirect reporting without rebuilding. @@ -133,8 +137,8 @@ opt-in trace backend an operator points at their own collector (below). Point the gateway at any OTLP/HTTP traces backend you run (a collector, Tempo, Jaeger, OpenObserve — anything speaking OTLP/HTTP) and deploys show -up there as distributed traces. Nothing is exported unless you set the -endpoint; the image carries no trace destination. +up there as distributed traces. Nothing is exported unless the endpoint is +set; the image carries no trace destination. ```bash SPROUT_OTLP_ENDPOINT=https:///api//v1/traces @@ -151,8 +155,8 @@ SPROUT_OTLP_HEADERS=Authorization=Basic ,stream-name=defa on every export request. Each entry splits on the first `=` only, so a base64 `Authorization` value keeps its padding. An entry without `=`, or with an empty name, fails boot naming the entry. -- Tracing is active exactly when the endpoint is set. With no endpoint there - is no SDK, no export, and no boot failure — the boot line reports +- Tracing is active exactly when the endpoint is set. With no endpoint + there is no SDK, no export, and no boot failure — the boot line reports `traces: "off"`, otherwise `traces: "/"`. Header values never appear in the boot line or the config summary (`otlpHeaders` is `"[set]"` / `"[empty]"`). @@ -164,10 +168,10 @@ with `preview.db` (database provision/reset), `preview.app` (container replace plus health gate), and `preview.seed` (seed image run) children. The deploy is detached from the request that accepted it, so it is its own root trace correlated by attributes, not by trace id. Deploy attributes are -`sprout.repo`, `sprout.pr`, `sprout.slug`, `sprout.plan`, and `sprout.status`; -failures set status `ERROR` and record the exception. Export uses a batch -processor, never the request path, with no signal handling — a slow or dead -backend cannot block or crash the gateway. +`sprout.repo`, `sprout.pr`, `sprout.slug`, `sprout.plan`, and +`sprout.status`; failures set status `ERROR` and record the exception. +Export uses a batch processor, never the request path, with no signal +handling — a slow or dead backend cannot block or crash the gateway. Privacy: the operator's own values go to the operator's own backend — and still never `appEnv`, DSNs, tokens, request or response bodies, headers, or diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md index 60bd44ab..9277ebd4 100644 --- a/docs/troubleshooting.md +++ b/docs/troubleshooting.md @@ -1,13 +1,8 @@ # Troubleshooting -The error catalogue for adopting repos and operators. Every entry names the -exact error. None of these print env values — parse errors name the key or -file without echoing the value. - -- Manifest keys? See [Adopting a repo](adopting-a-repo.md). -- Lifecycle background? See [Previews](previews.md). -- Gateway setup? See [Operator deploy](operator-deploy.md). -- Commands? See [CLI reference](cli-reference.md). +Match the symptom, copy the fix. Every entry names the exact error. None +of these print env values — parse errors name the key or file without +echoing the value. ## Adopter errors @@ -19,7 +14,7 @@ file without echoing the value. | Downloaded release fails verification | `checksum entry missing for in /SHA256SUMS.txt …` or a `sha256sum -c` mismatch; `test "$(sprout --version)" = "…"` fails | Pin to a release that ships checksums (≥ the release that publishes `SHA256SUMS.txt`); do not hand-edit the install — the component verifies the downloaded asset. | | Hostname template rejected | `preview.hostname … must contain {pr_id}` / scheme/port/path/placeholder errors; service `preview.services[i].hostname` / `.path must start with /` | Keep the template a bare host with `{pr_id}` (`pr-{pr_id}.app.example.com`). Read the URL from `preview_url=` / `PREVIEW_URL` — never reconstruct it in CI. | | Unknown manifest key | `unknown key: ` (top-level, `preview.*`, `health.*`, `seed.*`, `preview.env.*`) | Rename to a key in the [manifest table](adopting-a-repo.md#manifest-keys-sproutyaml); check `preview.env` against the canonical `PG*` set and services against `name/image/hostname/path/port/env/labels`. | -| Seed configured without health | `health block required in .sprout.yaml when seed block is configured` (or `when -s is passed`) | Add the `health:` block (quickstart snippet). The gate runs before any `docker build`. | +| Seed configured without health | `health block required in .sprout.yaml when seed block is configured` (or `when -s is passed`) | Add the `health:` block ([Getting started](getting-started.md#1-sproutyaml-at-the-repo-root) has the shape). The gate runs before any `docker build`. | | Secret not supplied | deploy fails before the gateway call naming the key (declared `{ required: true }`, no file/flag provided it) | Provide it via the `SPROUT_APP_ENV` / `SPROUT_SEED_ENV` file-type variable or `--app-env[-file]` / `--seed-env[-file]`. Never commit the secret to the manifest. | | File-type CI variable passed via `app_env_file` / `seed_env_file` input (e.g. `inputs: { app_env_file: $MY_ENV_FILE }`) | `preview.app_env.: required value missing` (nothing points at the input) — had the flag been passed with a bad path, the CLI would say `cannot read --app-env-file: ` instead | Repo-relative dotenv paths only — never pass File vars via `inputs:`; map the blob at job runtime via `variables:` (`sprout-preview: { variables: { SPROUT_APP_ENV: $MY_ENV_FILE } }`, seed: `SPROUT_SEED_ENV: $MY_SEED_FILE`). Full diagnostic in [component Troubleshooting](../templates/README.md#troubleshooting). | | Invalid dotenv line or flag | names the offending key or file, value never echoed | Fix the `KEY=value` line (blank lines, `#` comments, optional `export ` prefix; values may contain `=`); check `--tail` is a positive integer (`--tail must be a positive integer`). | @@ -43,7 +38,7 @@ file without echoing the value. | Symptom | Likely cause | What to try | |---|---|---| -| `curl …/healthz` fails / connection refused | Gateway not up, or host port conflict on `SPROUT_GATEWAY_HOST_PORT` | `docker compose --env-file compose.env ps`; `ss -ltnp | grep 7331` (or your host port); check `docker compose … logs gateway` | +| `curl …/healthz` fails / connection refused | Gateway not up, or host port conflict on `SPROUT_GATEWAY_HOST_PORT` | `docker compose --env-file compose.env ps`; `ss -ltnp \| grep 7331` (or your host port); check `docker compose … logs gateway` | | Traefik curl on `:8880` fails | Bundled Traefik not published, or `TRAEFIK_HTTP_PORT` overridden | Confirm `TRAEFIK_HTTP_PORT` in `compose.env`; `docker compose … ps traefik` | | `network … not found` on external overlay | `SPROUT_*_NETWORK` names do not exist on the host | `docker network ls`; `docker network create "$SPROUT_TRAEFIK_NETWORK"` (and postgres) before `up` | | Gateway boot: missing env / CREATEROLE | Required vars blank, or admin DSN lacks role privileges | Diff `compose.env` against `compose.env.example`; confirm admin can `CREATE ROLE` | diff --git a/llms.txt b/llms.txt index 94af1cdd..5db44d0e 100644 --- a/llms.txt +++ b/llms.txt @@ -1,11 +1,11 @@ These docs are for agents. Start with the onboarding prompt. - [Onboarding prompt](https://simpros.github.io/sprout/docs/onboarding-prompt.md): copy-paste agent block (entry point for agents). - [Getting started](https://simpros.github.io/sprout/docs/getting-started.md): first preview in one sitting. -- [Adopting a repo](https://simpros.github.io/sprout/docs/adopting-a-repo.md): sprout.yaml manifest reference and app entrypoints. +- [Adopting a repo](https://simpros.github.io/sprout/docs/adopting-a-repo.md): write .sprout.yaml and the app entrypoint. - [CI integration](https://simpros.github.io/sprout/docs/ci-integration.md): GitLab component, GitHub reusable workflow, variables, reset, notes. - [Previews](https://simpros.github.io/sprout/docs/previews.md): lifecycle: database providers, seeding, services, mail. - [Operator deploy](https://simpros.github.io/sprout/docs/operator-deploy.md): gateway compose stack, Traefik, env reference, admin token. - [CLI reference](https://simpros.github.io/sprout/docs/cli-reference.md): every sprout command, debugging, tokens. -- [Troubleshooting](https://simpros.github.io/sprout/docs/troubleshooting.md): adopter and operator error catalogue. -- [Telemetry](https://simpros.github.io/sprout/docs/telemetry.md): anonymous install reporting and how to stop it. +- [Troubleshooting](https://simpros.github.io/sprout/docs/troubleshooting.md): symptom → fix catalogue for adopters and operators. +- [Telemetry](https://simpros.github.io/sprout/docs/telemetry.md): what is sent, what it costs, how to stop it. - [Herdr integration](https://simpros.github.io/sprout/docs/herdr-integration.md): operator-side review automation. From 554f168776af7b261c535aef10148992ff0863b3 Mon Sep 17 00:00:00 2001 From: simpros <38320875+simpros@users.noreply.github.com> Date: Fri, 2 Oct 2026 21:17:08 +0200 Subject: [PATCH 2/2] address review feedback on #274 - operator-deploy: worktree-db flag contract single-sourced to cli-reference - adopting-a-repo: restore concrete seeded manifest (health + seed) example - troubleshooting: repoint seed-without-health fix at adopting-a-repo - telemetry: one post per deploy plus boot/daily heartbeat, not two per deploy - cli-reference: restore e2e/mail.test.ts Mailpit pointer - getting-started: drop internal parseSproutYaml loader step for human readers --- docs/adopting-a-repo.md | 18 ++++++++++++++++-- docs/cli-reference.md | 1 + docs/getting-started.md | 5 ++--- docs/operator-deploy.md | 17 ++--------------- docs/telemetry.md | 3 ++- docs/troubleshooting.md | 2 +- 6 files changed, 24 insertions(+), 22 deletions(-) diff --git a/docs/adopting-a-repo.md b/docs/adopting-a-repo.md index b6c5c739..57542887 100644 --- a/docs/adopting-a-repo.md +++ b/docs/adopting-a-repo.md @@ -155,8 +155,22 @@ preview: hostname: "pr-{pr_id}.myapp.preview.example.com" ``` -Write a seeded manifest (as in the quickstart): add `health:` + `seed:`. -`seed: {}` alone enables seeding with the conventional `Dockerfile.seed`. +Write a seeded manifest: add `health:` + `seed:` beside the app-only +manifest above. `health:` is required when seeding; `seed: {}` alone +enables seeding with the conventional `Dockerfile.seed`. + +```yaml +slug: myapp +preview: + hostname: "pr-{pr_id}.myapp.preview.example.com" +health: + path: /health + interval: 2s + timeout: 120s + expect: 200 +seed: + dockerfile: Dockerfile.seed +``` ## App image: migrate at startup diff --git a/docs/cli-reference.md b/docs/cli-reference.md index ffb5d495..d756ad28 100644 --- a/docs/cli-reference.md +++ b/docs/cli-reference.md @@ -157,6 +157,7 @@ Repo-relative paths for the contract above: `apps/server/src/http/deploy-mail.test.ts` - Mailbox / From presentation (MR note lines, settled deploy output): `apps/cli/src/commands/forge-note.test.ts` +- Send-and-read-back through Mailpit's API: `e2e/mail.test.ts` - Substituted hostname on a live deploy: `e2e/lifecycle.test.ts` - Component inputs / dotenv / `on_stop` wiring: `templates/preview.test.ts` diff --git a/docs/getting-started.md b/docs/getting-started.md index d6e55ea5..5819117f 100644 --- a/docs/getting-started.md +++ b/docs/getting-started.md @@ -37,9 +37,8 @@ preview: hostname: "pr-{pr_id}.myapp.preview.example.com" ``` -Check the file loads with the CLI's own loader (`apps/cli/src/yaml.ts`, -`parseSproutYaml`). Unknown keys are rejected (`unknown key: `), so -typos fail here, not at deploy. +Unknown keys are rejected (`unknown key: `), so typos fail on the +first `sprout ci preview`. When this deploys, add seeding next: [Adopting a repo](adopting-a-repo.md#manifest-keys-sproutyaml). diff --git a/docs/operator-deploy.md b/docs/operator-deploy.md index 71a5d131..bdd845bf 100644 --- a/docs/operator-deploy.md +++ b/docs/operator-deploy.md @@ -763,21 +763,8 @@ sprout worktree-db provision --slug --env-file --admin-url "$ADMIN sprout worktree-db drop --slug --admin-url "$ADMIN_DSN" ``` -- **Admin DSN** must allow `CREATEROLE` (or be superuser) — same bar as - `SPROUT_PREVIEW_POSTGRES_URL`. Role ensure reuses the shared - `@sprout/preview-db` `#71` algorithm (`CREATE` if missing, else - `ALTER … PASSWORD`). -- Objects are named `sprout_wt_` (hyphens in the worktree key become - underscores). `drop` refuses any name outside the `sprout_wt_` prefix. -- Worktree key rule (CLI `--slug`; distinct from adopting-repo **slug**): - lowercase, `[^a-z0-9-]` → `-`, collapse runs, max 40 characters. -- `provision` is idempotent: a second run re-ensures the role/DB and - rewrites connection vars in `--env-file` (creates the file if missing; - atomic temp+rename). When `PGPASSWORD` is already present in the env - file, that password is reused so live connections are not rotated. - Defaults write `DATABASE_URL` plus canonical `PGHOST` / `PGPORT` / - `PGUSER` / `PGPASSWORD` / `PGDATABASE`; rename with `--rename - LOGICAL=NAME` (logical keys: `DATABASE_URL`, `PGHOST`, …). +Flag contract (slug normalization, object naming, env-file behavior, +`--rename`): [CLI reference](cli-reference.md#worktree-db-local-provisioner). ## Upgrade / redeploy diff --git a/docs/telemetry.md b/docs/telemetry.md index 454104c0..1d591258 100644 --- a/docs/telemetry.md +++ b/docs/telemetry.md @@ -91,7 +91,8 @@ Transport costs one `POST` per event with a 3s timeout, fire-and-forget. A failed export never changes a deploy outcome — at most one warning is logged. There are no retries, no queue, and no batching: losing an anonymous event is acceptable, changing a deploy's outcome is not. That is -the whole cost: two small JSON posts per deploy plus one per day, with no +the whole cost: one small JSON post per deploy, plus an install +heartbeat at boot and every 24h, with no effect on the request path when the backend is slow or dead. ## What is never sent diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md index 9277ebd4..0813ccf7 100644 --- a/docs/troubleshooting.md +++ b/docs/troubleshooting.md @@ -14,7 +14,7 @@ echoing the value. | Downloaded release fails verification | `checksum entry missing for in /SHA256SUMS.txt …` or a `sha256sum -c` mismatch; `test "$(sprout --version)" = "…"` fails | Pin to a release that ships checksums (≥ the release that publishes `SHA256SUMS.txt`); do not hand-edit the install — the component verifies the downloaded asset. | | Hostname template rejected | `preview.hostname … must contain {pr_id}` / scheme/port/path/placeholder errors; service `preview.services[i].hostname` / `.path must start with /` | Keep the template a bare host with `{pr_id}` (`pr-{pr_id}.app.example.com`). Read the URL from `preview_url=` / `PREVIEW_URL` — never reconstruct it in CI. | | Unknown manifest key | `unknown key: ` (top-level, `preview.*`, `health.*`, `seed.*`, `preview.env.*`) | Rename to a key in the [manifest table](adopting-a-repo.md#manifest-keys-sproutyaml); check `preview.env` against the canonical `PG*` set and services against `name/image/hostname/path/port/env/labels`. | -| Seed configured without health | `health block required in .sprout.yaml when seed block is configured` (or `when -s is passed`) | Add the `health:` block ([Getting started](getting-started.md#1-sproutyaml-at-the-repo-root) has the shape). The gate runs before any `docker build`. | +| Seed configured without health | `health block required in .sprout.yaml when seed block is configured` (or `when -s is passed`) | Add the `health:` block ([Adopting a repo](adopting-a-repo.md#manifest-keys-sproutyaml) shows the shape). The gate runs before any `docker build`. | | Secret not supplied | deploy fails before the gateway call naming the key (declared `{ required: true }`, no file/flag provided it) | Provide it via the `SPROUT_APP_ENV` / `SPROUT_SEED_ENV` file-type variable or `--app-env[-file]` / `--seed-env[-file]`. Never commit the secret to the manifest. | | File-type CI variable passed via `app_env_file` / `seed_env_file` input (e.g. `inputs: { app_env_file: $MY_ENV_FILE }`) | `preview.app_env.: required value missing` (nothing points at the input) — had the flag been passed with a bad path, the CLI would say `cannot read --app-env-file: ` instead | Repo-relative dotenv paths only — never pass File vars via `inputs:`; map the blob at job runtime via `variables:` (`sprout-preview: { variables: { SPROUT_APP_ENV: $MY_ENV_FILE } }`, seed: `SPROUT_SEED_ENV: $MY_SEED_FILE`). Full diagnostic in [component Troubleshooting](../templates/README.md#troubleshooting). | | Invalid dotenv line or flag | names the offending key or file, value never echoed | Fix the `KEY=value` line (blank lines, `#` comments, optional `export ` prefix; values may contain `=`); check `--tail` is a positive integer (`--tail must be a positive integer`). |