fix(bootstrap): build migrate from maintained PostgreSQL-only source

.claude/skills/bootstrap-image-lock/SKILL.md

@@ -1,74 +1,56 @@
 ---
 name: bootstrap-image-lock
-description: Move bootstrap-image.json forward — the lock the bootstrap image builds from (alpine base digest, pinned apk package versions, golang-migrate archive checksums). Use when a bootstrap image build fails "unable to select packages", when the weekly bootstrap-lock-freshness check reports drift, when bumping the alpine branch or the migrate version, or when a bootstrap_image_test.go checksum assertion fails. Explains why a failing build is the intended signal and why values are never authored by hand.
+description: Resolve bootstrap-image.json from upstream — Alpine base/package pins and the source-built migrate Go toolchain. Use for bootstrap package drift, migrate/module or compiler updates, and bootstrap freshness failures. Never guess lock values.
 ---
 
 # Moving the bootstrap image lock
 
-`bootstrap-image.json` pins what the bootstrap image installs: the alpine base
-by digest, exact apk package versions, and the golang-migrate archive sha256 per
-architecture. `templates/builder/Dockerfile.tmpl` renders entirely from it.
+`bootstrap-image.json` pins the Alpine base, apk packages, migrate source version
+and Go builder image. The binary is built from the agent's embedded `go.mod` and
+`go.sum`, with only the PostgreSQL and file drivers. The module files' hashes,
+compiler image, source version and drivers are exposed in recipe/image provenance.
+See `docs/bootstrap-migrate.md` for the dependency and image verification contract.
 
-**Never edit this file by hand.** Every value is resolved from upstream by
-`.github/scripts/update-bootstrap-lock.sh` — the base digest from the registry,
-package versions from inside the digest-pinned base on *both* architectures, and
-the migrate checksums from the release's published `sha256sum.txt`.
+**Never edit lock values by hand.** The resolver verifies the registry manifest
+bytes against its digest and requires both linux/amd64 and linux/arm64 support.
 
 ```bash
-.github/scripts/update-bootstrap-lock.sh            # re-resolve what the lock targets
-.github/scripts/update-bootstrap-lock.sh --check    # report drift, write nothing, exit 1 if stale
+# Source/toolchain-only update: no engine. Keep the base and apk pins unchanged.
+python3 .github/scripts/update-bootstrap-migrate.py
+MIGRATE_GO_IMAGE=golang:1.27.3-alpine3.24 python3 .github/scripts/update-bootstrap-migrate.py
+
+# Full refresh: requires Docker and both architectures.
+.github/scripts/update-bootstrap-lock.sh
+.github/scripts/update-bootstrap-lock.sh --check
 ALPINE_BRANCH=v3.22 .github/scripts/update-bootstrap-lock.sh
-MIGRATE_VERSION=v4.20.0 .github/scripts/update-bootstrap-lock.sh
-BOOTSTRAP_PACKAGES="postgresql17-client" .github/scripts/update-bootstrap-lock.sh
-go test ./... && git commit bootstrap-image.json
 ```
 
-A plain rerun re-resolves the lock's *own* intent: the branch, migrate version
-and package set are read back out of the current file, so it cannot silently
-revert someone's deliberate bump. The environment variables are the only way to
-change what is targeted, the package set included (`BOOTSTRAP_PACKAGES`).
-
-Pin only what the image runs. The migrate archive is fetched with the base's
-busybox `wget` and verified against the lock's sha256, so no download tool is
-pinned: every pin is one more package a rolling repository can withdraw, and
-Alpine's per-architecture builders publish a replacement at different times, so
-for that lag no lock is satisfiable on both architectures (#189).
-
-## "unable to select packages" is the feature
-
-Alpine publishes no immutable package snapshot — a release branch keeps only the
-newest `-rN` of each package. Pinning exact versions trades availability for
-integrity: once upstream replaces a pinned package (`postgresql17-client`), the build
-fails loudly instead of quietly installing different bytes.
-
-That failure is the signal to rerun the script, and it is also how a security
-fix reaches the image. Do not respond to it by loosening a version constraint or
-dropping a pin — that converts a build failure into an unrecorded change of
-what ships.
-
-The signal covers pinned packages only. A base digest the branch has moved past,
-and a newer migrate release, fail nothing on their own, which is why
-`.github/workflows/bootstrap-lock-freshness.yml` runs `--check` on a schedule
-instead of gating pull requests on unrelated upstream releases.
-
-## What the checksums do and do not prove
-
-The `sha256sum.txt` ships in the same GitHub release as the archives it covers.
-It pins **identity** — the bytes cannot change under a fixed version, and a
-mirror or CDN cannot substitute them — but it shares a trust root with the
-artifact, so it is not proof of authorship. Locking the version is what bounds
-that trust. Say so if a PR changes `MIGRATE_VERSION`.
-
-## Tests that hold the lock honest
-
-Under `go test ./...`, and several need docker with arm64 emulation:
-
-- `TestBootstrapImageLockMatchesCommittedDocument` — the parsed lock equals the file.
-- `TestBootstrapImageLockChecksumsMatchPublishedArchives` — the recorded sha256
-  is what the release actually publishes.
-- `TestBootstrapImageBuildsLockedInputsPerArchitecture` — the locked inputs
-  install on both architectures.
-- `TestBootstrapImageRejectsTamperedMigrateArchive` — a wrong checksum fails the build.
-
-Without an arm64 emulator these skip locally and fail under `CI`. If you ran
-them only on amd64, the arm64 archive checksum is unverified — say that.
+To upgrade migrate or its dependencies, update the parent module through normal
+Go dependency maintenance first, then resolve the lock. Do not use `go install
+...@version` in the image: that ignores the maintained parent graph. The recipe
+uses `go build -mod=readonly`, `GOTOOLCHAIN=local` and the exact builder digest.
+The Go builder must be at least the module's Go version. Updating the parent
+module changes the embedded inputs and their provenance hashes automatically.
+
+## Package drift
+
+Alpine's rolling repository replaces `-rN` packages. A missing pinned package
+must fail the build; resolve the lock rather than loosening a constraint. The
+full resolver checks package versions on both architectures. The freshness
+workflow checks for registry/package drift and a newer migrate source release.
+An exact Go patch pin requires an explicit compiler update when a new patch ships.
+
+## Verification
+
+- `TestBootstrapMigrateSourceBuildGraph` cross-compiles the rendered build command
+  for both target architectures and inspects the actual binary's module graph.
+- `TestBootstrapMigrateRejectsTamperedModuleInputs` refuses changed module files
+  before compiling, so provenance cannot describe different input bytes.
+- `TestBuildRecipeLocksBootstrapInputs` checks the staged source files and recipe
+  hashes; the existing deterministic-emission tests cover the aggregate digest.
+- `TestBootstrapImageBuildsLockedInputsPerArchitecture` builds and runs both
+  images; the existing bootstrap parity tests exercise database migrations.
+
+Host cross-compilation does not verify the container build or image scan. If no
+engine slot is allocated, run host tests only and state that the Docker checks,
+registry freshness and actual image vulnerability scan remain unverified.

.github/scripts/update-bootstrap-lock.sh

@@ -5,24 +5,19 @@
 #   .github/scripts/update-bootstrap-lock.sh                 # re-resolve what the lock already targets
 #   .github/scripts/update-bootstrap-lock.sh --check         # report drift, write nothing, exit 1 if stale
 #   ALPINE_BRANCH=v3.22 .github/scripts/update-bootstrap-lock.sh
-#   MIGRATE_VERSION=v4.20.0 .github/scripts/update-bootstrap-lock.sh
+#   MIGRATE_GO_IMAGE=golang:1.27.3-alpine3.24 .github/scripts/update-bootstrap-lock.sh
 #   BOOTSTRAP_PACKAGES="postgresql17-client" .github/scripts/update-bootstrap-lock.sh
 #
 # Every value is resolved from upstream, never authored here: the base digest comes
 # from the registry, package versions from inside the digest-pinned base on both
-# architectures, and the migrate archive checksums from the release's published
-# sha256sum.txt. What the run targets — Alpine branch, migrate version, package set
+# architectures, and the Go toolchain manifest digest from the registry.
+# Migrate's version comes from go.mod. What the run targets — Alpine branch, package set
 # — is read out of the current lock, so a plain rerun re-resolves the lock's own
 # intent instead of a default that could silently revert someone's bump. The
 # environment variables are the only way to change what is targeted, the package
 # set (BOOTSTRAP_PACKAGES, space-separated) included. Run
 # `go test ./...` afterwards and commit the diff.
 #
-# The checksum file ships in the same GitHub release as the archives it covers, so
-# it pins identity — the bytes cannot change under a fixed version, and a mirror or
-# CDN cannot substitute them — but it shares a trust root with the artifact and is
-# therefore not proof of authorship. Locking the version is what bounds that trust.
-#
 # Why pinning packages this way, and what it costs:
 #
 # Alpine publishes no immutable package snapshot — a release branch keeps only the
@@ -52,13 +47,14 @@ LOCK="$(git rev-parse --show-toplevel)/bootstrap-image.json"
 # overrides it deliberately.
 lock_branch=$(jq -er '.packages.repositories[0] | capture("/(?<branch>v[0-9]+\\.[0-9]+)/").branch' "$LOCK")
 ALPINE_BRANCH=${ALPINE_BRANCH:-$lock_branch}
-MIGRATE_VERSION=${MIGRATE_VERSION:-$(jq -er '.migrate.version' "$LOCK")}
+migrate=$(python3 "$(dirname "$0")/update-bootstrap-migrate.py" --resolve)
+MIGRATE_VERSION=$(jq -er '.version' <<<"$migrate")
 if [[ -n "${BOOTSTRAP_PACKAGES:-}" ]]; then
   read -r -a PACKAGES <<<"$BOOTSTRAP_PACKAGES"
 else
   mapfile -t PACKAGES < <(jq -er '.packages.pinned[].name' "$LOCK")
 fi
-mapfile -t ARCHITECTURES < <(jq -er '.migrate.archives[].architecture' "$LOCK")
+ARCHITECTURES=(amd64 arm64)
 # A package name reaches a shell inside the resolver container, so it is held to
 # apk's own name shape here rather than trusted as typed.
 for package in "${PACKAGES[@]}"; do
@@ -128,28 +124,8 @@ resolve_package_versions() {
   printf '%s\n' "$resolved"
 }
 
-resolve_archive_checksums() {
-  local checksums
-  checksums=$(curl -fsSL \
-    "https://github.com/golang-migrate/migrate/releases/download/${MIGRATE_VERSION}/sha256sum.txt") || {
-    echo "could not fetch published checksums for ${MIGRATE_VERSION}" >&2
-    return 1
-  }
-  local architecture
-  for architecture in "${ARCHITECTURES[@]}"; do
-    local checksum
-    checksum=$(awk -v archive="migrate.linux-${architecture}.tar.gz" \
-      '$2 == archive { print $1; exit }' <<<"$checksums")
-    if [[ ! "$checksum" =~ ^[0-9a-f]{64}$ ]]; then
-      echo "${MIGRATE_VERSION} publishes no checksum for linux-${architecture}" >&2
-      return 1
-    fi
-    printf '%s\n' "$checksum"
-  done
-}
-
 # Reports a newer upstream migrate release. Nothing else signals it: the pinned
-# archive keeps verifying and installing forever, so staleness here is invisible
+# source version keeps compiling forever, so staleness here is invisible
 # until someone looks.
 report_migrate_staleness() {
   local latest
@@ -160,7 +136,7 @@ report_migrate_staleness() {
   fi
   if [[ "$latest" != "$MIGRATE_VERSION" ]]; then
     echo "migrate ${MIGRATE_VERSION} is behind upstream ${latest}"
-    echo "  to take it: MIGRATE_VERSION=${latest} ${0##*/}"
+    echo "  to take it: update the migrate dependency in go.mod, then run ${0##*/}"
     return 1
   fi
   echo "migrate ${MIGRATE_VERSION} is the current upstream release"
@@ -181,43 +157,27 @@ if ! versions_output=$(resolve_package_versions "$base"); then
   exit 1
 fi
 mapfile -t versions <<<"$versions_output"
-if ! checksums_output=$(resolve_archive_checksums); then
-  exit 1
-fi
-mapfile -t checksums <<<"$checksums_output"
-
 if [[ "${#versions[@]}" -ne "${#PACKAGES[@]}" ]]; then
   echo "could not resolve every package version from ${REPOSITORIES[*]}" >&2
   exit 1
 fi
-if [[ "${#checksums[@]}" -ne "${#ARCHITECTURES[@]}" ]]; then
-  echo "could not resolve every migrate archive checksum for ${MIGRATE_VERSION}" >&2
-  exit 1
-fi
 
 pinned=$(jq -n '[]')
 for index in "${!PACKAGES[@]}"; do
   pinned=$(jq --arg name "${PACKAGES[$index]}" --arg version "${versions[$index]}" \
     '. + [{name: $name, version: $version}]' <<<"$pinned")
 done
 
-archives=$(jq -n '[]')
-for index in "${!ARCHITECTURES[@]}"; do
-  archives=$(jq --arg architecture "${ARCHITECTURES[$index]}" --arg sha256 "${checksums[$index]}" \
-    '. + [{architecture: $architecture, sha256: $sha256}]' <<<"$archives")
-done
-
 resolved=$(jq -n \
   --arg version "$release" \
   --arg digest "$digest" \
-  --arg migrate "$MIGRATE_VERSION" \
+  --argjson migrate "$migrate" \
   --argjson repositories "$(jq -n --args '$ARGS.positional' "${REPOSITORIES[@]}")" \
   --argjson pinned "$pinned" \
-  --argjson archives "$archives" \
   '{
     base: {image: "alpine", version: $version, digest: $digest},
     packages: {repositories: $repositories, pinned: $pinned},
-    migrate: {version: $migrate, archives: $archives}
+    migrate: $migrate
   }')
 
 stale=false

.github/scripts/update-bootstrap-migrate.py

@@ -0,0 +1,54 @@
+#!/usr/bin/env python3
+"""Resolve migrate's maintained module version and Go builder digest, without Docker.
+
+Default: update only the migrate portion of bootstrap-image.json atomically.
+--resolve: emit that portion for update-bootstrap-lock.sh's full refresh.
+MIGRATE_GO_IMAGE selects an exact golang:<patch>-alpine<release> tag deliberately;
+otherwise retain the current lock's tag. Module upgrades belong in go.mod.
+"""
+import hashlib
+import json
+import os
+from pathlib import Path
+import re
+import subprocess
+import sys
+import urllib.request
+
+if sys.argv[1:] not in ([], ["--resolve"]):
+    sys.exit("usage: update-bootstrap-migrate.py [--resolve]")
+
+root = Path(__file__).resolve().parents[2]
+lock_path = root / "bootstrap-image.json"
+lock = json.loads(lock_path.read_text())
+module = json.loads(subprocess.check_output(["go", "mod", "edit", "-json"], cwd=root))
+version = next(r["Version"] for r in module["Require"]
+               if r["Path"] == "github.com/golang-migrate/migrate/v4")
+image = os.environ.get("MIGRATE_GO_IMAGE", lock["migrate"].get("toolchain", "").split("@")[0])
+if not re.fullmatch(r"golang:[0-9]+\.[0-9]+\.[0-9]+-alpine[0-9]+\.[0-9]+", image):
+    sys.exit("MIGRATE_GO_IMAGE must name an exact golang:<patch>-alpine<release> tag")
+with urllib.request.urlopen("https://auth.docker.io/token?service=registry.docker.io&scope=repository:library/golang:pull", timeout=30) as response:
+    token = json.load(response)["token"]
+request = urllib.request.Request(
+    "https://registry-1.docker.io/v2/library/golang/manifests/" + image.split(":", 1)[1],
+    headers={"Authorization": "Bearer " + token,
+             "Accept": "application/vnd.oci.image.index.v1+json, application/vnd.docker.distribution.manifest.list.v2+json"})
+with urllib.request.urlopen(request, timeout=30) as response:
+    body = response.read()
+    digest = response.headers["Docker-Content-Digest"]
+if digest != "sha256:" + hashlib.sha256(body).hexdigest():
+    sys.exit("registry toolchain manifest digest mismatch")
+platforms = {(m["platform"].get("os"), m["platform"].get("architecture"))
+             for m in json.loads(body)["manifests"]}
+if not {("linux", "amd64"), ("linux", "arm64")} <= platforms:
+    sys.exit("toolchain must support linux/amd64 and linux/arm64")
+resolved = {"version": version, "toolchain": image + "@" + digest}
+if sys.argv[1:] == ["--resolve"]:
+    print(json.dumps(resolved))
+elif not sys.argv[1:]:
+    lock["migrate"] = resolved
+    staged = lock_path.with_suffix(".json.staged")
+    staged.write_text(json.dumps(lock, indent=2) + "\n")
+    staged.replace(lock_path)
+else:
+    sys.exit("usage: update-bootstrap-migrate.py [--resolve]")

AGENTS.md

@@ -142,6 +142,8 @@ variables only the qualify script supplies.
   reported, never re-applied.
 - **`libs/go` is consumed by published version, never `replace`.** The qualify
   script fails on a `replace` directive or any core/parent import.
+- Bootstrap migrate is source-built from the maintained parent module; see
+  `docs/bootstrap-migrate.md`. Inspect the binary graph, not just `go.mod`.
 - **The two image locks move by different mechanisms.** `bootstrap-image.json`
   is written by a script; `runtime-image.json` is hand-edited and only verified.
   Neither is a value to guess — see the skills below.

bootstrap-image.json

@@ -17,15 +17,6 @@
   },
   "migrate": {
     "version": "v4.19.1",
-    "archives": [
-      {
-        "architecture": "amd64",
-        "sha256": "2ac648fbd1b127b69ab5a7b33cf96212178f71e22379fc50573630c6f4c7ce18"
-      },
-      {
-        "architecture": "arm64",
-        "sha256": "2fea2455c0f3f07cc3f4b98471c951ad1a716059574b20b6416bd1e9058751c5"
-      }
-    ]
+    "toolchain": "golang:1.27.3-alpine3.24@sha256:d200d2dd76b03a99b52651fd1325aff7dcd35422c35c01bac56571b0507f3deb"
   }
 }