Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .agents/checks/review.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ the reviewer's distillation.
- Core packages: every loop, queue, retry, and wait must be bounded. An unbounded anything in
a core package is a review-blocking defect.
- Dangerous APIs accept proof types (`statement.Classified`, `PreflightedTable`,
`AbsentTarget`, `VerifiedShadow`, `CleanWatermark`, `TableLock`) with package-private
`AbsentTarget`, `CreationRole`, `VerifiedShadow`, `CleanWatermark`, `TableLock`) with package-private
constructors — never a
raw string or bool that a caller could fabricate. Core code re-verifies its own
preconditions; it never trusts that the planner or CLI checked.
Expand Down
24 changes: 24 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,30 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).

### Changed — observable outcomes for automation callers

- **Desired-state execution now creates a table that does not exist yet**
instead of refusing the plan. `migrate.RunDesired` on a greenfield plan
verifies the target name is free and the role holds `CREATE` on the
schema, then runs the `CREATE TABLE` and the index builds as brief
bounded steps; a rerun converges to an empty plan. An occupied name is a
new typed refusal reason, **`create-collision`** (added to
`verdict.Reasons()`); `PARTITION OF` and `IF NOT EXISTS` shapes refuse
with `unsupported-statement` before anything runs. A caller that relied
on the previous greenfield `unsupported-statement` refusal now sees the
create execute. Desired-file statements are additionally ordered for
execution at parse — the `CREATE TABLE` first, indexes keeping their
input order after it — everywhere the file replays: the greenfield plan,
the create path's steps, and the scratch-schema introspection that
derives a diff once the table exists. The plan states execution order, a
greenfield plan's fingerprint changes when the desired file listed an
index before its table, and an index-first file converges on rerun.
- **Alter attempts now run with `search_path` pinned to the target
schema** (then `public`) whenever the statement is schema-qualified —
the same resolution the create path and introspection use. A statement's
unqualified secondary names — a column's type, an expression's
function — resolve in the target schema, where previously they resolved
via the session's ambient `search_path` and could silently bind a
same-named object in `public`. A caller that relied on ambient
resolution for secondary names must qualify them.
- **`diff` now exits 2 when the derived plan contains a statement execution
would refuse**, in all three output modes (default report, `--sql`,
`--json`) — the same CI-gate contract as `migrate --dry-run`. Previously
Expand Down
8 changes: 8 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,10 @@ routes the statement, then executes the routed SQL — the planner's safer nativ
default when the submitted form blocks (reported in the verdict's `executed_sql`), a bounded
optimistic native attempt otherwise. A gated `--force` runs the submitted form as-is under the
same budgets. Changes without an available backend get a structured refusal (exit code 2).
Desired-state execution — converging a live table onto a `CREATE TABLE` file, including
creating the table when it does not exist yet — is a Go API today: `migrate.RunDesired`
in [`pkg/migrate`](pkg/migrate/desired.go); the CLI's `migrate` verb takes one imperative
statement.
The design docs and the phased
build plan live in [docs/](docs/) — start with
[docs/README.md](docs/README.md); the vision — what pg-sprite is and is not —
Expand Down Expand Up @@ -68,6 +72,10 @@ refusal — never a silently wrong or incomplete result:
- **Unlogged tables and explicit column collations** are outside the
declarative model: converging either is a table (or column) rewrite, so
export and diff refuse rather than plan one.
- **Desired-state execution has no CLI verb yet** — `migrate.RunDesired`
(including the greenfield `CREATE TABLE` path for a table that does not
exist) is library-only; the CLI's `migrate` takes one imperative
statement.
- **Non-table objects** — views, standalone sequences, enums, domains,
extensions, functions, triggers — are outside the declarative model,
which covers one ordinary table plus its indexes per file.
Expand Down
4 changes: 2 additions & 2 deletions SAFETY.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,8 +67,8 @@ The short version — the full rules live in [docs/tcb-model.md](docs/tcb-model.
- **Never trust callers.** Every dangerous operation re-verifies its preconditions, whoever the
requester is (CLI, planner, orchestrator). The periphery may request; the core enforces.
- **Domain types make illegal states unrepresentable.** Validating passages return proof types
with package-private constructors (today `preflight.PreflightedTable` and
`preflight.AbsentTarget`; later phases add
with package-private constructors (today `preflight.PreflightedTable`,
`preflight.AbsentTarget`, and `preflight.CreationRole`; later phases add
`VerifiedShadow`, `CleanWatermark`, and `TableLock`); dangerous APIs accept only proof types —
e.g. the planned cutover swap will accept only a `VerifiedShadow`.
- **Put a limit on everything.** Every loop bounded, every queue bounded, every retry counted,
Expand Down
4 changes: 2 additions & 2 deletions docs/capabilities.md
Original file line number Diff line number Diff line change
Expand Up @@ -156,7 +156,7 @@ Status legend: ✅ T1 (supported today) · 🟡 T2 (planned; typed refusal today

| Operation | Status | Online-safety problem? | Behavior and why |
| --- | --- | --- | --- |
| `CREATE TABLE ... PARTITION OF` | | Yes | Executed, with a typed warning: creating a partition takes a brief `ACCESS EXCLUSIVE` on the **parent** and queues behind long-running queries |
| `CREATE TABLE ... PARTITION OF` | 🟡 | Yes | Typed refusal at both doors: the imperative door does not take `CREATE TABLE`, and the declarative create path refuses the form — attaching a partition takes a brief `ACCESS EXCLUSIVE` on the **parent**, which the greenfield absence proof does not cover. The partition-aware flow is planned |
| `ATTACH PARTITION` | ✅ | Yes | Executed; the safer idiom (pre-prove the bound with a validated `CHECK` so the attach skips its scan) is surfaced as guidance. A classify-first flow that constructs the proof itself is planned |
| `DETACH PARTITION [CONCURRENTLY]` | ✅ | Yes | `CONCURRENTLY` is the idiom; the blocking form is rewritten to it |
| Partitioned parents in the **declarative model** | 🟡 | Yes | Typed refusal: the model does not yet carry partition keys, and rendering a partitioned parent as a plain `CREATE TABLE` would be silently wrong |
Expand All @@ -171,7 +171,7 @@ Status legend: ✅ T1 (supported today) · 🟡 T2 (planned; typed refusal today
| Unlogged tables | 🟡 | Yes | Typed refusal: persistence is not modeled, converging it (`SET LOGGED`) is a full rewrite, and rendering the table as plain `CREATE TABLE` would silently change crash-safety |
| Explicit column collations | 🟡 | Yes | Typed refusal: dropping a `COLLATE` clause from a rendered baseline silently changes sort order and index semantics; a collation delta cannot converge without a rewrite |
| Columns whose default uses a sequence the column does not own | 🟡 | Yes | Typed refusal: in a desired-state model that sequence exists only inside the scratch transaction, so no derived plan can reference it. Column-owned (`serial`-style) sequences are fine |
| Greenfield `CREATE TABLE` apply (the table does not exist yet — a fresh database or a new table in a live one) | 🟡 | Yes — a `REFERENCES` clause takes a brief `SHARE ROW EXCLUSIVE` on each **referenced** live table | Planned as an owned operation. The new table itself has no readers or writers to protect; the online-safety problem is the `REFERENCES` clause, whose lock on each referenced live table queues behind long-running queries and blocks writers behind it — exactly the run-it-under-a-bounded-`lock_timeout` job this engine owns and owner tooling does not do. The absence preflight (`CheckTableAbsent`) is in place; the executor create path and front-door admission build on it. `diff --sql` already emits the statement |
| Greenfield `CREATE TABLE` apply (the table does not exist yet — a fresh database or a new table in a live one) | | Yes — a `REFERENCES` clause would take a brief `SHARE ROW EXCLUSIVE` on each **referenced** live table, but desired files refuse foreign keys today, so no live table is locked | Desired-state execution creates the table: the absence preflight (`CheckTableAbsent`) verifies the name is free, `CheckCreatePrivileges` verifies the role can create in the schema, and the executor runs the `CREATE TABLE` and the index builds as brief bounded steps under the engine's `lock_timeout` / `statement_timeout` budgets. An occupied name is a typed `create-collision` refusal; `PARTITION OF`, `INHERITS`, `LIKE`, `OF`, `IF NOT EXISTS`, and in-set duplicate names are typed refusals before anything runs, while `REFERENCES` and `CONCURRENTLY` are refused upstream at desired-file parse and re-checked at admission as defense in depth |

### Types and non-table objects

Expand Down
3 changes: 2 additions & 1 deletion docs/cli-output-examples.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ in `detail`. The set is closed and pinned by test (`verdict.Reasons()`).

| Reason | Meaning |
|---|---|
| `unsupported-statement` | No safe path is known for the statement — only `ALTER TABLE` and `CREATE INDEX` reach classification — or a desired-state plan needs a table that does not exist yet. |
| `unsupported-statement` | No safe path is known for the statement — only `ALTER TABLE` and `CREATE INDEX` reach classification — or a greenfield create plan carries a shape the create path refuses (`PARTITION OF`, `IF NOT EXISTS`). |
| `index-statement` | Index maintenance (`DROP INDEX`, `REINDEX`) has a native safe idiom (`CONCURRENTLY`) and is never attempted; the verdict's `safer_idiom` names it. |
| `not-native-safe-table-too-large` | The size guard skipped the optimistic attempt: the table exceeds the configured bound and the change is not provably metadata-only. |
| `insufficient-privileges` | The connected role lacks the access the change needs; `detail` names the exact missing GRANT (see [engine-role.md](engine-role.md)). |
Expand All @@ -87,6 +87,7 @@ in `detail`. The set is closed and pinned by test (`verdict.Reasons()`).
| `backend-unavailable` | The change routes to an execution strategy this build does not implement (copy-and-swap). |
| `destructive-change` | The desired-state plan discards live structure — a dropped column, constraint, index, or `NOT NULL` — and desired-state execution runs no destructive statement; run the drop deliberately instead ([execution model](execution-model.md)). |
| `plan-fingerprint-mismatch` | The plan recomputed at execution time does not carry the pinned fingerprint: the plan a reviewer approved is not the plan that would execute, so nothing runs ([execution model](execution-model.md)). |
| `create-collision` | The greenfield create plan's target name is already occupied — a relation or standalone type took it after the plan was derived. Nothing runs; re-derive the plan against the live catalog and review what it says now. |

## Migrate

Expand Down
9 changes: 9 additions & 0 deletions docs/engine-role.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,15 @@ same preflight: `wal_level = logical` (`rds.logical_replication = 1` on Aurora/R
static parameter requiring a reboot), and free `max_replication_slots` /
`max_wal_senders` headroom.

### Off-ladder: greenfield `CREATE TABLE`

Creating a new table sits outside the ladder: the table does not exist yet, so there is no
owning role to be a member of — the table is born owned by the role that creates it. The
create path's preflight (`CheckCreatePrivileges`) therefore proves exactly `CONNECT` on the
database plus `USAGE` and `CREATE` on the target schema, deliberately not the Tier 1–3
ownership membership. A missing grant is refused with the exact `GRANT` statement, whose
grantee is the engine role itself.

## Provisioning

For a target whose tables are owned by `app_owner` in schema `app`:
Expand Down
48 changes: 44 additions & 4 deletions docs/execution-model.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ and [suggest-report.md](suggest-report.md#caveats-caveats).
- [The committed prefix](#the-committed-prefix)
- [How a failure is reported](#how-a-failure-is-reported)
- [Why the prefix is safe to leave](#why-the-prefix-is-safe-to-leave)
- [Outcome codes](#outcome-codes)

## Why there is no wrapping transaction

Expand Down Expand Up @@ -55,10 +56,15 @@ autocommit-each-step has two shapes in the executor:
- **Brief catalog steps (step kind `brief`) and `VALIDATE CONSTRAINT` (step
kind `validate-constraint`)** each run as one short *explicit* transaction:
`BEGIN` → `SET LOCAL lock_timeout` / `statement_timeout` → the statement →
`COMMIT` (`pkg/executor`'s bounded runner). The explicit `BEGIN` exists
only because the budgets are applied with `SET LOCAL`, which is scoped to
that transaction — functionally it is still one statement, one
transaction, committed immediately, rolled back atomically on failure.
`COMMIT` (`pkg/executor`'s bounded runner). When the preflight proof
carries a schema, the same `SET LOCAL` pins `search_path` to that schema
then `public`, so the statement's unqualified secondary names — a
column's type, an expression's function — resolve in the target schema,
exactly as the introspection read path resolves them. The explicit
`BEGIN` exists only because the settings are applied with `SET LOCAL`,
which is scoped to that transaction — functionally it is still one
statement, one transaction, committed immediately, rolled back
atomically on failure.
- **`CREATE INDEX CONCURRENTLY` (step kind `concurrent-index-build`)** is
true autocommit on a dedicated budgeted session: it refuses to run inside
any transaction block and internally manages multiple transactions of its
Expand Down Expand Up @@ -241,3 +247,37 @@ statement — stopping at the first refusal or failure. Its result carries the
plan, one verdict per attempted statement, and a detail naming exactly which
planned statements committed and remain in effect: the committed prefix at
the plan level, statements instead of steps.

## Outcome codes

`executor.Codes()` enumerates the closed vocabulary below, and
`executor.OutcomeCode` maps any executor error to its entry — the same code
that reaches the JSON verdict's `code` field. Adapters render three facts
per failure — the outcome code, the failing step's position
(`SequenceStepError.Step` of `.Total`), and the failing step's SQL — and
log the raw error, whose text interpolates server prose and is not a
branching surface.

| Code | Meaning |
| --- | --- |
| `budget-lock-exceeded` | The lock was not granted within `lock_timeout`; nothing executed |
| `budget-statement-exceeded` | The statement ran past `statement_timeout` and was cancelled |
| `cancelled-externally` | The statement was cancelled from outside the executor before its budget elapsed |
| `invalid-index-own-leftover` | The failed build's own INVALID index remains; the [recovery runbook](invalid-index-recovery.md) applies |
| `invalid-index-preexisting` | An INVALID index under the requested name predates this run |
| `invalid-index-unproven` | An INVALID index may remain but the catalog state could not be proven |
| `empty-sequence` | The sequence had no steps to run |
| `unsupported-sequence-step` | A step is not a shape the sequence executor can run safely |
| `unsupported-partitioned-parent` | Partitioned-parent admission refusal |
| `not-concurrent-index-build` | The statement handed to the concurrent build executor is not a `CREATE INDEX CONCURRENTLY` |
| `unnamed-index` | The concurrent build does not name its index, so its outcome could not be verified |
| `unqualified-table` | The target table is not schema-qualified at the library boundary |
| `if-not-exists-unsupported` | `CREATE ... IF NOT EXISTS` cannot prove what its no-op would mean |
| `create-collision` | A name the create path needs is already taken on the server; re-diff the live catalog |
| `duplicate-create-name` | The desired set claims the same relation name twice; refused at admission |
| `partition-of-unsupported` | `CREATE TABLE PARTITION OF` locks the partitioned parent, which the absence proof does not cover |
| `unsupported-create-step` | A desired statement is not a shape the create path can run |
| `pool-too-small` | The pool cannot hold the build session and the verdict connection at once |
| `table-not-found` | The statement's qualified table does not exist |
| `invariant-violation` | A breach of the invariant registry; never a retry candidate |
| `execution-failed` | Fallback for a failure outside the typed set — an operational error to investigate, not a refusal to branch on |
20 changes: 18 additions & 2 deletions docs/invariants.md
Original file line number Diff line number Diff line change
Expand Up @@ -244,10 +244,25 @@ executes, any statement whose target table does not match the preflight proof it
A proof for one table can never smuggle SQL against another, and a multi-statement string can
never reach the database through the executor (pgx's simple protocol would happily run all of
it). *Enforced:* `pkg/executor` (`ExecuteNative`; `RunSequence` admission re-proves every step's
target against the preflight proof before the first step executes), `pkg/statement` (proof
construction).
target against the preflight proof before the first step executes; `ExecuteCreate` re-proves
every desired statement's target against the absence proof the same way), `pkg/statement`
(proof construction).
*Source:* adversarial review of the optimistic front door.

### ST-8 — A desired schema's statements carry execution order in the proof

A `statement.DesiredSchema` orders its statements for execution at construction — the
`CREATE TABLE` first, the indexes keeping their input order after it — so every replay of
the file states the same order and the position mapping between a greenfield plan's
statements and the create path's step verdicts holds by construction, not by each replay
site re-deriving the rule. A set that does not lead with a `CREATE TABLE` means the proof
was forged or mutated, and every consumer refuses it fail-closed rather than reordering.
*Enforced:* `pkg/statement` (`ParseDesired` establishes the order), `pkg/diffplan`
(`qualifiedDesired` asserts it when rendering the greenfield plan), `pkg/executor`
(`admitCreateSteps` asserts it before anything runs); `pkg/schemadiff`'s scratch
materialization relies on it to run the `CREATE TABLE` before its indexes.
*Source:* adversarial review of the declarative front door.

## Refusals and preflight (RF)

Each refusal is a preflight **error with a stated reason** — never a warning, never attempted.
Expand Down Expand Up @@ -345,4 +360,5 @@ about **how we write and review the code**.
| ST-1, ST-2, ST-3, ST-4 | 8 | kill/resume, cross-version refuse, orphan-slot reap, failover reconcile |
| ST-6 | 1 onward, complete by 8 | preflight matrix |
| ST-7 | 1 | target-mismatch refusal + single-statement-by-construction tests |
| ST-8 | 2 (declarative model) | parse-time ordering + forged-proof refusal tests at every replay site |
| OC-1..OC-6 | shape APIs from 2; bind at 11 | engine-contract tests |
Loading
Loading