From a1a8181c22cd5054bdb5bccb8e56e570e2a933d0 Mon Sep 17 00:00:00 2001 From: Antoine Toussaint <312536060+antoinetoussaint-byte@users.noreply.github.com> Date: Wed, 28 Oct 2026 18:34:37 -0400 Subject: [PATCH] fix(bootstrap): build migrate from maintained PostgreSQL-only source --- .claude/skills/bootstrap-image-lock/SKILL.md | 106 ++++----- .github/scripts/update-bootstrap-lock.sh | 60 +---- .github/scripts/update-bootstrap-migrate.py | 54 +++++ AGENTS.md | 2 + bootstrap-image.json | 11 +- bootstrap_image.go | 99 ++++----- bootstrap_image_test.go | 219 +++++++------------ bootstrap_template_test.go | 19 +- build_test.go | 4 + builder.go | 3 + docs/bootstrap-migrate.md | 59 +++++ templates/builder/Dockerfile.tmpl | 43 ++-- 12 files changed, 336 insertions(+), 343 deletions(-) create mode 100644 .github/scripts/update-bootstrap-migrate.py create mode 100644 docs/bootstrap-migrate.md diff --git a/.claude/skills/bootstrap-image-lock/SKILL.md b/.claude/skills/bootstrap-image-lock/SKILL.md index e3a075a..7a1cf54 100644 --- a/.claude/skills/bootstrap-image-lock/SKILL.md +++ b/.claude/skills/bootstrap-image-lock/SKILL.md @@ -1,74 +1,56 @@ --- name: bootstrap-image-lock -description: Move bootstrap-image.json forward — the lock the bootstrap image builds from (alpine base digest, pinned apk package versions, golang-migrate archive checksums). Use when a bootstrap image build fails "unable to select packages", when the weekly bootstrap-lock-freshness check reports drift, when bumping the alpine branch or the migrate version, or when a bootstrap_image_test.go checksum assertion fails. Explains why a failing build is the intended signal and why values are never authored by hand. +description: Resolve bootstrap-image.json from upstream — Alpine base/package pins and the source-built migrate Go toolchain. Use for bootstrap package drift, migrate/module or compiler updates, and bootstrap freshness failures. Never guess lock values. --- # Moving the bootstrap image lock -`bootstrap-image.json` pins what the bootstrap image installs: the alpine base -by digest, exact apk package versions, and the golang-migrate archive sha256 per -architecture. `templates/builder/Dockerfile.tmpl` renders entirely from it. +`bootstrap-image.json` pins the Alpine base, apk packages, migrate source version +and Go builder image. The binary is built from the agent's embedded `go.mod` and +`go.sum`, with only the PostgreSQL and file drivers. The module files' hashes, +compiler image, source version and drivers are exposed in recipe/image provenance. +See `docs/bootstrap-migrate.md` for the dependency and image verification contract. -**Never edit this file by hand.** Every value is resolved from upstream by -`.github/scripts/update-bootstrap-lock.sh` — the base digest from the registry, -package versions from inside the digest-pinned base on *both* architectures, and -the migrate checksums from the release's published `sha256sum.txt`. +**Never edit lock values by hand.** The resolver verifies the registry manifest +bytes against its digest and requires both linux/amd64 and linux/arm64 support. ```bash -.github/scripts/update-bootstrap-lock.sh # re-resolve what the lock targets -.github/scripts/update-bootstrap-lock.sh --check # report drift, write nothing, exit 1 if stale +# Source/toolchain-only update: no engine. Keep the base and apk pins unchanged. +python3 .github/scripts/update-bootstrap-migrate.py +MIGRATE_GO_IMAGE=golang:1.27.3-alpine3.24 python3 .github/scripts/update-bootstrap-migrate.py + +# Full refresh: requires Docker and both architectures. +.github/scripts/update-bootstrap-lock.sh +.github/scripts/update-bootstrap-lock.sh --check ALPINE_BRANCH=v3.22 .github/scripts/update-bootstrap-lock.sh -MIGRATE_VERSION=v4.20.0 .github/scripts/update-bootstrap-lock.sh -BOOTSTRAP_PACKAGES="postgresql17-client" .github/scripts/update-bootstrap-lock.sh -go test ./... && git commit bootstrap-image.json ``` -A plain rerun re-resolves the lock's *own* intent: the branch, migrate version -and package set are read back out of the current file, so it cannot silently -revert someone's deliberate bump. The environment variables are the only way to -change what is targeted, the package set included (`BOOTSTRAP_PACKAGES`). - -Pin only what the image runs. The migrate archive is fetched with the base's -busybox `wget` and verified against the lock's sha256, so no download tool is -pinned: every pin is one more package a rolling repository can withdraw, and -Alpine's per-architecture builders publish a replacement at different times, so -for that lag no lock is satisfiable on both architectures (#189). - -## "unable to select packages" is the feature - -Alpine publishes no immutable package snapshot — a release branch keeps only the -newest `-rN` of each package. Pinning exact versions trades availability for -integrity: once upstream replaces a pinned package (`postgresql17-client`), the build -fails loudly instead of quietly installing different bytes. - -That failure is the signal to rerun the script, and it is also how a security -fix reaches the image. Do not respond to it by loosening a version constraint or -dropping a pin — that converts a build failure into an unrecorded change of -what ships. - -The signal covers pinned packages only. A base digest the branch has moved past, -and a newer migrate release, fail nothing on their own, which is why -`.github/workflows/bootstrap-lock-freshness.yml` runs `--check` on a schedule -instead of gating pull requests on unrelated upstream releases. - -## What the checksums do and do not prove - -The `sha256sum.txt` ships in the same GitHub release as the archives it covers. -It pins **identity** — the bytes cannot change under a fixed version, and a -mirror or CDN cannot substitute them — but it shares a trust root with the -artifact, so it is not proof of authorship. Locking the version is what bounds -that trust. Say so if a PR changes `MIGRATE_VERSION`. - -## Tests that hold the lock honest - -Under `go test ./...`, and several need docker with arm64 emulation: - -- `TestBootstrapImageLockMatchesCommittedDocument` — the parsed lock equals the file. -- `TestBootstrapImageLockChecksumsMatchPublishedArchives` — the recorded sha256 - is what the release actually publishes. -- `TestBootstrapImageBuildsLockedInputsPerArchitecture` — the locked inputs - install on both architectures. -- `TestBootstrapImageRejectsTamperedMigrateArchive` — a wrong checksum fails the build. - -Without an arm64 emulator these skip locally and fail under `CI`. If you ran -them only on amd64, the arm64 archive checksum is unverified — say that. +To upgrade migrate or its dependencies, update the parent module through normal +Go dependency maintenance first, then resolve the lock. Do not use `go install +...@version` in the image: that ignores the maintained parent graph. The recipe +uses `go build -mod=readonly`, `GOTOOLCHAIN=local` and the exact builder digest. +The Go builder must be at least the module's Go version. Updating the parent +module changes the embedded inputs and their provenance hashes automatically. + +## Package drift + +Alpine's rolling repository replaces `-rN` packages. A missing pinned package +must fail the build; resolve the lock rather than loosening a constraint. The +full resolver checks package versions on both architectures. The freshness +workflow checks for registry/package drift and a newer migrate source release. +An exact Go patch pin requires an explicit compiler update when a new patch ships. + +## Verification + +- `TestBootstrapMigrateSourceBuildGraph` cross-compiles the rendered build command + for both target architectures and inspects the actual binary's module graph. +- `TestBootstrapMigrateRejectsTamperedModuleInputs` refuses changed module files + before compiling, so provenance cannot describe different input bytes. +- `TestBuildRecipeLocksBootstrapInputs` checks the staged source files and recipe + hashes; the existing deterministic-emission tests cover the aggregate digest. +- `TestBootstrapImageBuildsLockedInputsPerArchitecture` builds and runs both + images; the existing bootstrap parity tests exercise database migrations. + +Host cross-compilation does not verify the container build or image scan. If no +engine slot is allocated, run host tests only and state that the Docker checks, +registry freshness and actual image vulnerability scan remain unverified. diff --git a/.github/scripts/update-bootstrap-lock.sh b/.github/scripts/update-bootstrap-lock.sh index 61d89ed..b87774f 100755 --- a/.github/scripts/update-bootstrap-lock.sh +++ b/.github/scripts/update-bootstrap-lock.sh @@ -5,24 +5,19 @@ # .github/scripts/update-bootstrap-lock.sh # re-resolve what the lock already targets # .github/scripts/update-bootstrap-lock.sh --check # report drift, write nothing, exit 1 if stale # ALPINE_BRANCH=v3.22 .github/scripts/update-bootstrap-lock.sh -# MIGRATE_VERSION=v4.20.0 .github/scripts/update-bootstrap-lock.sh +# MIGRATE_GO_IMAGE=golang:1.27.3-alpine3.24 .github/scripts/update-bootstrap-lock.sh # BOOTSTRAP_PACKAGES="postgresql17-client" .github/scripts/update-bootstrap-lock.sh # # Every value is resolved from upstream, never authored here: the base digest comes # from the registry, package versions from inside the digest-pinned base on both -# architectures, and the migrate archive checksums from the release's published -# sha256sum.txt. What the run targets — Alpine branch, migrate version, package set +# architectures, and the Go toolchain manifest digest from the registry. +# Migrate's version comes from go.mod. What the run targets — Alpine branch, package set # — is read out of the current lock, so a plain rerun re-resolves the lock's own # intent instead of a default that could silently revert someone's bump. The # environment variables are the only way to change what is targeted, the package # set (BOOTSTRAP_PACKAGES, space-separated) included. Run # `go test ./...` afterwards and commit the diff. # -# The checksum file ships in the same GitHub release as the archives it covers, so -# it pins identity — the bytes cannot change under a fixed version, and a mirror or -# CDN cannot substitute them — but it shares a trust root with the artifact and is -# therefore not proof of authorship. Locking the version is what bounds that trust. -# # Why pinning packages this way, and what it costs: # # Alpine publishes no immutable package snapshot — a release branch keeps only the @@ -52,13 +47,14 @@ LOCK="$(git rev-parse --show-toplevel)/bootstrap-image.json" # overrides it deliberately. lock_branch=$(jq -er '.packages.repositories[0] | capture("/(?v[0-9]+\\.[0-9]+)/").branch' "$LOCK") ALPINE_BRANCH=${ALPINE_BRANCH:-$lock_branch} -MIGRATE_VERSION=${MIGRATE_VERSION:-$(jq -er '.migrate.version' "$LOCK")} +migrate=$(python3 "$(dirname "$0")/update-bootstrap-migrate.py" --resolve) +MIGRATE_VERSION=$(jq -er '.version' <<<"$migrate") if [[ -n "${BOOTSTRAP_PACKAGES:-}" ]]; then read -r -a PACKAGES <<<"$BOOTSTRAP_PACKAGES" else mapfile -t PACKAGES < <(jq -er '.packages.pinned[].name' "$LOCK") fi -mapfile -t ARCHITECTURES < <(jq -er '.migrate.archives[].architecture' "$LOCK") +ARCHITECTURES=(amd64 arm64) # A package name reaches a shell inside the resolver container, so it is held to # apk's own name shape here rather than trusted as typed. for package in "${PACKAGES[@]}"; do @@ -128,28 +124,8 @@ resolve_package_versions() { printf '%s\n' "$resolved" } -resolve_archive_checksums() { - local checksums - checksums=$(curl -fsSL \ - "https://github.com/golang-migrate/migrate/releases/download/${MIGRATE_VERSION}/sha256sum.txt") || { - echo "could not fetch published checksums for ${MIGRATE_VERSION}" >&2 - return 1 - } - local architecture - for architecture in "${ARCHITECTURES[@]}"; do - local checksum - checksum=$(awk -v archive="migrate.linux-${architecture}.tar.gz" \ - '$2 == archive { print $1; exit }' <<<"$checksums") - if [[ ! "$checksum" =~ ^[0-9a-f]{64}$ ]]; then - echo "${MIGRATE_VERSION} publishes no checksum for linux-${architecture}" >&2 - return 1 - fi - printf '%s\n' "$checksum" - done -} - # Reports a newer upstream migrate release. Nothing else signals it: the pinned -# archive keeps verifying and installing forever, so staleness here is invisible +# source version keeps compiling forever, so staleness here is invisible # until someone looks. report_migrate_staleness() { local latest @@ -160,7 +136,7 @@ report_migrate_staleness() { fi if [[ "$latest" != "$MIGRATE_VERSION" ]]; then echo "migrate ${MIGRATE_VERSION} is behind upstream ${latest}" - echo " to take it: MIGRATE_VERSION=${latest} ${0##*/}" + echo " to take it: update the migrate dependency in go.mod, then run ${0##*/}" return 1 fi echo "migrate ${MIGRATE_VERSION} is the current upstream release" @@ -181,19 +157,10 @@ if ! versions_output=$(resolve_package_versions "$base"); then exit 1 fi mapfile -t versions <<<"$versions_output" -if ! checksums_output=$(resolve_archive_checksums); then - exit 1 -fi -mapfile -t checksums <<<"$checksums_output" - if [[ "${#versions[@]}" -ne "${#PACKAGES[@]}" ]]; then echo "could not resolve every package version from ${REPOSITORIES[*]}" >&2 exit 1 fi -if [[ "${#checksums[@]}" -ne "${#ARCHITECTURES[@]}" ]]; then - echo "could not resolve every migrate archive checksum for ${MIGRATE_VERSION}" >&2 - exit 1 -fi pinned=$(jq -n '[]') for index in "${!PACKAGES[@]}"; do @@ -201,23 +168,16 @@ for index in "${!PACKAGES[@]}"; do '. + [{name: $name, version: $version}]' <<<"$pinned") done -archives=$(jq -n '[]') -for index in "${!ARCHITECTURES[@]}"; do - archives=$(jq --arg architecture "${ARCHITECTURES[$index]}" --arg sha256 "${checksums[$index]}" \ - '. + [{architecture: $architecture, sha256: $sha256}]' <<<"$archives") -done - resolved=$(jq -n \ --arg version "$release" \ --arg digest "$digest" \ - --arg migrate "$MIGRATE_VERSION" \ + --argjson migrate "$migrate" \ --argjson repositories "$(jq -n --args '$ARGS.positional' "${REPOSITORIES[@]}")" \ --argjson pinned "$pinned" \ - --argjson archives "$archives" \ '{ base: {image: "alpine", version: $version, digest: $digest}, packages: {repositories: $repositories, pinned: $pinned}, - migrate: {version: $migrate, archives: $archives} + migrate: $migrate }') stale=false diff --git a/.github/scripts/update-bootstrap-migrate.py b/.github/scripts/update-bootstrap-migrate.py new file mode 100644 index 0000000..3a72ad2 --- /dev/null +++ b/.github/scripts/update-bootstrap-migrate.py @@ -0,0 +1,54 @@ +#!/usr/bin/env python3 +"""Resolve migrate's maintained module version and Go builder digest, without Docker. + +Default: update only the migrate portion of bootstrap-image.json atomically. +--resolve: emit that portion for update-bootstrap-lock.sh's full refresh. +MIGRATE_GO_IMAGE selects an exact golang:-alpine tag deliberately; +otherwise retain the current lock's tag. Module upgrades belong in go.mod. +""" +import hashlib +import json +import os +from pathlib import Path +import re +import subprocess +import sys +import urllib.request + +if sys.argv[1:] not in ([], ["--resolve"]): + sys.exit("usage: update-bootstrap-migrate.py [--resolve]") + +root = Path(__file__).resolve().parents[2] +lock_path = root / "bootstrap-image.json" +lock = json.loads(lock_path.read_text()) +module = json.loads(subprocess.check_output(["go", "mod", "edit", "-json"], cwd=root)) +version = next(r["Version"] for r in module["Require"] + if r["Path"] == "github.com/golang-migrate/migrate/v4") +image = os.environ.get("MIGRATE_GO_IMAGE", lock["migrate"].get("toolchain", "").split("@")[0]) +if not re.fullmatch(r"golang:[0-9]+\.[0-9]+\.[0-9]+-alpine[0-9]+\.[0-9]+", image): + sys.exit("MIGRATE_GO_IMAGE must name an exact golang:-alpine tag") +with urllib.request.urlopen("https://auth.docker.io/token?service=registry.docker.io&scope=repository:library/golang:pull", timeout=30) as response: + token = json.load(response)["token"] +request = urllib.request.Request( + "https://registry-1.docker.io/v2/library/golang/manifests/" + image.split(":", 1)[1], + headers={"Authorization": "Bearer " + token, + "Accept": "application/vnd.oci.image.index.v1+json, application/vnd.docker.distribution.manifest.list.v2+json"}) +with urllib.request.urlopen(request, timeout=30) as response: + body = response.read() + digest = response.headers["Docker-Content-Digest"] +if digest != "sha256:" + hashlib.sha256(body).hexdigest(): + sys.exit("registry toolchain manifest digest mismatch") +platforms = {(m["platform"].get("os"), m["platform"].get("architecture")) + for m in json.loads(body)["manifests"]} +if not {("linux", "amd64"), ("linux", "arm64")} <= platforms: + sys.exit("toolchain must support linux/amd64 and linux/arm64") +resolved = {"version": version, "toolchain": image + "@" + digest} +if sys.argv[1:] == ["--resolve"]: + print(json.dumps(resolved)) +elif not sys.argv[1:]: + lock["migrate"] = resolved + staged = lock_path.with_suffix(".json.staged") + staged.write_text(json.dumps(lock, indent=2) + "\n") + staged.replace(lock_path) +else: + sys.exit("usage: update-bootstrap-migrate.py [--resolve]") diff --git a/AGENTS.md b/AGENTS.md index 77d993c..cf63872 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -142,6 +142,8 @@ variables only the qualify script supplies. reported, never re-applied. - **`libs/go` is consumed by published version, never `replace`.** The qualify script fails on a `replace` directive or any core/parent import. +- Bootstrap migrate is source-built from the maintained parent module; see + `docs/bootstrap-migrate.md`. Inspect the binary graph, not just `go.mod`. - **The two image locks move by different mechanisms.** `bootstrap-image.json` is written by a script; `runtime-image.json` is hand-edited and only verified. Neither is a value to guess — see the skills below. diff --git a/bootstrap-image.json b/bootstrap-image.json index 2594c79..2e61bab 100644 --- a/bootstrap-image.json +++ b/bootstrap-image.json @@ -17,15 +17,6 @@ }, "migrate": { "version": "v4.19.1", - "archives": [ - { - "architecture": "amd64", - "sha256": "2ac648fbd1b127b69ab5a7b33cf96212178f71e22379fc50573630c6f4c7ce18" - }, - { - "architecture": "arm64", - "sha256": "2fea2455c0f3f07cc3f4b98471c951ad1a716059574b20b6416bd1e9058751c5" - } - ] + "toolchain": "golang:1.27.3-alpine3.24@sha256:d200d2dd76b03a99b52651fd1325aff7dcd35422c35c01bac56571b0507f3deb" } } diff --git a/bootstrap_image.go b/bootstrap_image.go index 1637ed7..ec1f2c6 100644 --- a/bootstrap_image.go +++ b/bootstrap_image.go @@ -7,7 +7,9 @@ import ( "encoding/hex" "encoding/json" "fmt" - "slices" + "os" + "path/filepath" + "regexp" "strings" "github.com/codefly-dev/core/shared" @@ -16,14 +18,20 @@ import ( //go:embed bootstrap-image.json var bootstrapImageLockJSON []byte +//go:embed go.mod +var bootstrapMigrateGoMod []byte + +//go:embed go.sum +var bootstrapMigrateGoSum []byte + // bootstrapArchitectures are the architectures the bootstrap image is built for, // matching the recipe's linux/amd64 + linux/arm64 platform list. The lock must -// resolve a migrate archive for each of them, and the image rejects any other. +// build a migrate binary for each of them, and the image rejects any other. var bootstrapArchitectures = []string{"amd64", "arm64"} // bootstrapRecipePlatforms are the buildx platforms the emitted recipe targets, // derived from the architectures the lock resolves inputs for so a platform can -// never be advertised without a verified migrate archive behind it. +// never be advertised without a source-built migrate binary behind it. func bootstrapRecipePlatforms() []string { platforms := make([]string, 0, len(bootstrapArchitectures)) for _, architecture := range bootstrapArchitectures { @@ -37,25 +45,18 @@ func bootstrapRecipePlatforms() []string { // its own source of truth. var bootstrapLock = shared.Must(parseBootstrapImageLock(bootstrapImageLockJSON)) -// bootstrapImageLock is the checked-in identity of every input the bootstrap -// image consumes: the base image by digest, the apk repository with exact package -// versions, and the migrate release archive by checksum per architecture. Nothing -// the image builds from resolves against a mutable tag, and the migrate archive is -// verified before any of its bytes are extracted or executed. -// -// .github/scripts/update-bootstrap-lock.sh is the supported way to change it; it -// resolves each value from upstream (including migrate's published checksums) -// rather than accepting hand-written digests. +// bootstrapImageLock pins the base, apk packages and source-build toolchain. +// The maintained module files are embedded separately; their hashes travel in +// provenance and are verified before compilation, so callers cannot substitute +// a different module graph behind the image labels. type bootstrapImageLock struct { Base bootstrapBaseImage `json:"base"` Packages bootstrapPackages `json:"packages"` Migrate bootstrapMigrateBinary `json:"migrate"` - // LockDigest is the sha256 of the lock's canonical encoding, derived when it is - // parsed. It is the single value two builds compare to prove they resolved the - // same inputs, and it travels with the image as a label. Canonical, not the - // document's own bytes: reindenting or reordering keys in the file changes no - // input, so it must not change the identity the provenance reports. + // LockDigest identifies the JSON inputs; the embedded module-file hashes are + // separate provenance entries. Together they enter the aggregate recipe digest. + // Canonical encoding ignores JSON whitespace and key order. LockDigest string `json:"-"` } @@ -76,13 +77,30 @@ type bootstrapPackage struct { } type bootstrapMigrateBinary struct { - Version string `json:"version"` - Archives []bootstrapMigrateArchive `json:"archives"` + Version string `json:"version"` + Toolchain string `json:"toolchain"` +} + +func (bootstrapMigrateBinary) GoModSHA256() string { + return fmt.Sprintf("%x", sha256.Sum256(bootstrapMigrateGoMod)) +} + +func (bootstrapMigrateBinary) GoSumSHA256() string { + return fmt.Sprintf("%x", sha256.Sum256(bootstrapMigrateGoSum)) } -type bootstrapMigrateArchive struct { - Architecture string `json:"architecture"` - SHA256 string `json:"sha256"` +// Stage only agent-owned module inputs, never a consumer's module or credentials. +func stageBootstrapMigrate(root string) error { + dir := filepath.Join(root, "migrate") + if err := os.MkdirAll(dir, 0o755); err != nil { + return err + } + for name, data := range map[string][]byte{"go.mod": bootstrapMigrateGoMod, "go.sum": bootstrapMigrateGoSum} { + if err := os.WriteFile(filepath.Join(dir, name), data, 0o644); err != nil { + return err + } + } + return nil } // Reference is the base image the Dockerfile builds FROM. The locked digest is the @@ -124,10 +142,6 @@ type bootstrapProvenance struct { // record, so a consumer can read what a built artifact was assembled from without // re-resolving anything. func (l *bootstrapImageLock) Provenance() []bootstrapProvenance { - archives := make([]string, 0, len(l.Migrate.Archives)) - for _, archive := range l.Migrate.Archives { - archives = append(archives, archive.Architecture+"="+archive.SHA256) - } return []bootstrapProvenance{ {Arg: "CODEFLY_BOOTSTRAP_LOCK_DIGEST", Label: "dev.codefly.bootstrap.lock-digest", Value: l.LockDigest}, {Arg: "CODEFLY_BOOTSTRAP_BASE", Label: "dev.codefly.bootstrap.base", Value: l.Base.Reference()}, @@ -135,7 +149,10 @@ func (l *bootstrapImageLock) Provenance() []bootstrapProvenance { {Arg: "CODEFLY_BOOTSTRAP_APK_REPOSITORIES", Label: "dev.codefly.bootstrap.apk-repositories", Value: strings.Join(l.Packages.Repositories, " ")}, {Arg: "CODEFLY_BOOTSTRAP_APK_PACKAGES", Label: "dev.codefly.bootstrap.apk-packages", Value: l.Packages.Specs()}, {Arg: "CODEFLY_BOOTSTRAP_MIGRATE_VERSION", Label: "dev.codefly.bootstrap.migrate-version", Value: l.Migrate.Version}, - {Arg: "CODEFLY_BOOTSTRAP_MIGRATE_ARCHIVES", Label: "dev.codefly.bootstrap.migrate-archives", Value: strings.Join(archives, ",")}, + {Arg: "CODEFLY_BOOTSTRAP_MIGRATE_TOOLCHAIN", Label: "dev.codefly.bootstrap.migrate-toolchain", Value: l.Migrate.Toolchain}, + {Arg: "CODEFLY_BOOTSTRAP_MIGRATE_GO_MOD", Label: "dev.codefly.bootstrap.migrate-go-mod", Value: l.Migrate.GoModSHA256()}, + {Arg: "CODEFLY_BOOTSTRAP_MIGRATE_GO_SUM", Label: "dev.codefly.bootstrap.migrate-go-sum", Value: l.Migrate.GoSumSHA256()}, + {Arg: "CODEFLY_BOOTSTRAP_MIGRATE_DRIVERS", Label: "dev.codefly.bootstrap.migrate-drivers", Value: "postgres,file"}, } } @@ -225,33 +242,11 @@ func (l *bootstrapImageLock) validatePackages() error { } func (l *bootstrapImageLock) validateMigrate() error { - if !strings.HasPrefix(l.Migrate.Version, "v") { + if !regexp.MustCompile(`^v[0-9]+\.[0-9]+\.[0-9]+$`).MatchString(l.Migrate.Version) { return fmt.Errorf("bootstrap migrate version must be a release tag, got %q", l.Migrate.Version) } - locked := make(map[string]bool, len(l.Migrate.Archives)) - for _, archive := range l.Migrate.Archives { - if !slices.Contains(bootstrapArchitectures, archive.Architecture) { - return fmt.Errorf( - "bootstrap migrate archive architecture %q is not one of %s", - archive.Architecture, - strings.Join(bootstrapArchitectures, ", "), - ) - } - if locked[archive.Architecture] { - return fmt.Errorf("bootstrap migrate archive for %s is locked twice", archive.Architecture) - } - locked[archive.Architecture] = true - if err := validateSHA256Checksum( - fmt.Sprintf("bootstrap migrate %s archive checksum", archive.Architecture), - archive.SHA256, - ); err != nil { - return err - } - } - for _, architecture := range bootstrapArchitectures { - if !locked[architecture] { - return fmt.Errorf("bootstrap migrate archive for %s is required", architecture) - } + if !regexp.MustCompile(`^golang:[0-9]+\.[0-9]+\.[0-9]+-alpine[0-9]+\.[0-9]+@sha256:[0-9a-f]{64}$`).MatchString(l.Migrate.Toolchain) { + return fmt.Errorf("bootstrap migrate toolchain must pin an exact Go/Alpine tag and sha256 digest") } return nil } diff --git a/bootstrap_image_test.go b/bootstrap_image_test.go index 48ba8aa..5b77e89 100644 --- a/bootstrap_image_test.go +++ b/bootstrap_image_test.go @@ -1,14 +1,11 @@ package main import ( - "archive/tar" - "compress/gzip" "crypto/sha256" + "debug/buildinfo" "encoding/hex" "encoding/json" "fmt" - "io" - "net/http" "os" "os/exec" "path/filepath" @@ -53,46 +50,15 @@ func TestBootstrapImageLockDigestIgnoresDocumentFormatting(t *testing.T) { require.Equal(t, bootstrapLock.LockDigest, fromIndented.LockDigest) } -// TestBootstrapImageLockChecksumsMatchPublishedArchives is the check that makes a -// locked checksum falsifiable: it downloads each archive the lock pins and hashes -// it. Nothing else in the suite can tell a correct checksum from a plausible one — -// the template test only asserts the value is present, and the per-architecture -// build test needs emulation the host may not have. -func TestBootstrapImageLockChecksumsMatchPublishedArchives(t *testing.T) { - for _, archive := range bootstrapLock.Migrate.Archives { - t.Run(archive.Architecture, func(t *testing.T) { - url := fmt.Sprintf( - "https://github.com/golang-migrate/migrate/releases/download/%s/migrate.linux-%s.tar.gz", - bootstrapLock.Migrate.Version, archive.Architecture, - ) - response, err := http.Get(url) - require.NoError(t, err, "cannot reach %s to verify the locked checksum", url) - defer func() { require.NoError(t, response.Body.Close()) }() - require.Equal(t, http.StatusOK, response.StatusCode, url) - - hasher := sha256.New() - _, err = io.Copy(hasher, response.Body) - require.NoError(t, err) - require.Equal(t, archive.SHA256, hex.EncodeToString(hasher.Sum(nil)), - "locked checksum does not match the archive published at %s", url) - }) - } -} - -// The bootstrap command reconciles runtime access with psql, so the client stays -// pinned in the lock rather than pulled from whatever the repository serves. No -// download tool is: the migrate archive is fetched with the base's busybox wget -// and verified against the lock's sha256, so a package installed only to download -// it would add one more pin a rolling repository can withdraw (#189) and one more -// binary in a production Job image, for no integrity it does not already have. +// The runtime needs psql, but no download or compiler package. func TestBootstrapImageLockPinsBuildAndCommandTooling(t *testing.T) { pinned := map[string]string{} for _, entry := range bootstrapLock.Packages.Pinned { pinned[entry.Name] = entry.Version } require.Contains(t, pinned, "postgresql17-client") - require.NotContains(t, pinned, "curl", "the archive download needs no installed tool") - require.NotContains(t, pinned, "wget", "the archive download needs no installed tool") + require.NotContains(t, pinned, "curl", "source build needs no runtime download tool") + require.NotContains(t, pinned, "wget", "source build needs no runtime download tool") } func TestBootstrapProvenanceExposesEveryResolvedInput(t *testing.T) { @@ -119,8 +85,10 @@ func TestBootstrapProvenanceExposesEveryResolvedInput(t *testing.T) { require.Equal(t, bootstrapLock.Base.Reference(), values["dev.codefly.bootstrap.base"]) require.Equal(t, bootstrapLock.Packages.Specs(), values["dev.codefly.bootstrap.apk-packages"]) require.Equal(t, strings.Join(bootstrapLock.Packages.Repositories, " "), values["dev.codefly.bootstrap.apk-repositories"]) - require.Contains(t, values["dev.codefly.bootstrap.migrate-archives"], "amd64=") - require.Contains(t, values["dev.codefly.bootstrap.migrate-archives"], "arm64=") + require.Equal(t, bootstrapLock.Migrate.Toolchain, values["dev.codefly.bootstrap.migrate-toolchain"]) + require.Equal(t, bootstrapLock.Migrate.GoModSHA256(), values["dev.codefly.bootstrap.migrate-go-mod"]) + require.Equal(t, bootstrapLock.Migrate.GoSumSHA256(), values["dev.codefly.bootstrap.migrate-go-sum"]) + require.Equal(t, "postgres,file", values["dev.codefly.bootstrap.migrate-drivers"]) buildArgs := bootstrapLock.RecipeBuildArgs() require.Len(t, buildArgs, len(provenance)) @@ -190,32 +158,16 @@ func TestParseBootstrapImageLockRejectsUnverifiableInput(t *testing.T) { error: `bootstrap migrate version must be a release tag, got "latest"`, }, { - name: "archive without a checksum", - amend: func(document map[string]any) { - lockArchives(document)[0].(map[string]any)["sha256"] = "" - }, - error: "bootstrap migrate amd64 archive checksum must be a sha256 checksum", - }, - { - name: "architecture locked twice", - amend: func(document map[string]any) { - lockArchives(document)[1].(map[string]any)["architecture"] = "amd64" - }, - error: "bootstrap migrate archive for amd64 is locked twice", + name: "toolchain without digest", + amend: func(document map[string]any) { lockMigrate(document)["toolchain"] = "golang:1.27.3-alpine3.24" }, + error: "bootstrap migrate toolchain must pin an exact Go/Alpine tag and sha256 digest", }, { - name: "unsupported architecture", + name: "toolchain shell injection", amend: func(document map[string]any) { - lockArchives(document)[1].(map[string]any)["architecture"] = "riscv64" + lockMigrate(document)["toolchain"] = "golang:1.27.3-alpine3.24@sha256:$(id)" }, - error: `bootstrap migrate archive architecture "riscv64" is not one of amd64, arm64`, - }, - { - name: "architecture missing from the lock", - amend: func(document map[string]any) { - lockMigrate(document)["archives"] = lockArchives(document)[:1] - }, - error: "bootstrap migrate archive for arm64 is required", + error: "bootstrap migrate toolchain must pin an exact Go/Alpine tag and sha256 digest", }, { name: "unknown field", @@ -259,10 +211,6 @@ func lockMigrate(document map[string]any) map[string]any { return document["migrate"].(map[string]any) } -func lockArchives(document map[string]any) []any { - return lockMigrate(document)["archives"].([]any) -} - func TestBootstrapDockerfileLocksEveryInput(t *testing.T) { dockerfile := renderBuilderTemplate(t, "templates/builder/Dockerfile.tmpl", DockerTemplating{ MigrationConnectionEnvironment: migrationConnectionEnvironmentKey, @@ -279,25 +227,16 @@ func TestBootstrapDockerfileLocksEveryInput(t *testing.T) { // appended to it would ship whatever the repository serves that day. require.Regexp(t, `(?m)apk add --no-cache `+regexp.QuoteMeta(bootstrapLock.Packages.Specs())+`$`, dockerfile) - script := bootstrapMigrateInstallScript(t, dockerfile) - for _, archive := range bootstrapLock.Migrate.Archives { - require.Contains(t, script, archive.Architecture+") checksum="+archive.SHA256) - } - require.Contains(t, script, "/releases/download/"+bootstrapLock.Migrate.Version+"/migrate.linux-${architecture}.tar.gz") - - // The archive lands in a file that is checksum-verified before anything reads - // it, so unverified bytes never reach tar — the shape a download-into-tar pipe - // cannot have. - require.NotContains(t, script, "| tar") - require.NotContains(t, script, "curl", "the download uses the base's busybox wget") - download := strings.Index(script, "wget -q -O /tmp/migrate.tar.gz") - verification := strings.Index(script, "sha256sum -c -") - extraction := strings.Index(script, "tar -xzf") - installation := strings.Index(script, "install -m 0755") - require.Positive(t, download) - require.Greater(t, verification, download) - require.Greater(t, extraction, verification) - require.Greater(t, installation, extraction) + require.Contains(t, dockerfile, "FROM --platform=$BUILDPLATFORM "+bootstrapLock.Migrate.Toolchain+" AS migrate-builder") + require.NotContains(t, dockerfile, "releases/download") + require.Contains(t, dockerfile, "COPY --from=migrate-builder /migrate /usr/local/bin/migrate") + require.Contains(t, dockerfile, "COPY --from=migrate-builder /bootstrap /app/bootstrap") + require.Contains(t, dockerfile, "RUN rm -rf /bootstrap/migrate") + script := bootstrapMigrateBuildScript(t, dockerfile) + require.Contains(t, script, "GOTOOLCHAIN=local") + require.Contains(t, script, "CGO_ENABLED=0 GOOS=linux GOARCH=\"$architecture\"") + require.Contains(t, script, "go build -mod=readonly -trimpath -tags postgres,file") + require.Less(t, strings.Index(script, "sha256sum -c -"), strings.Index(script, "go build")) // Literal label values, and no ARG behind them: a label that resolves a build // argument reports whatever the caller passed, not what built the image. @@ -393,33 +332,60 @@ func TestBootstrapImageLabelsCannotBeOverriddenAtBuildTime(t *testing.T) { } } -// TestBootstrapImageRejectsTamperedMigrateArchive runs the Dockerfile's own install -// step against a substituted archive: only the download is replaced, so the -// verification under test is the template's. The archive carries a payload that -// records its own execution, so the test can prove the tampered bytes were never -// extracted or run. -func TestBootstrapImageRejectsTamperedMigrateArchive(t *testing.T) { - requireDocker(t) - dockerfile := renderBuilderTemplate(t, "templates/builder/Dockerfile.tmpl", DockerTemplating{ - MigrationConnectionEnvironment: migrationConnectionEnvironmentKey, - }) - script := bootstrapMigrateInstallScript(t, dockerfile) - - download := regexp.MustCompile(`wget -q -O /tmp/migrate\.tar\.gz "[^"]+"`) - require.Len(t, download.FindAllString(script, -1), 1, "install step no longer downloads the archive to a file") - script = download.ReplaceAllString(script, "cp /fixture/migrate.tar.gz /tmp/migrate.tar.gz") - - fixture := t.TempDir() - writeTamperedMigrateArchive(t, filepath.Join(fixture, "migrate.tar.gz")) - - output, err := exec.Command("docker", "run", "--rm", - "--volume", fixture+":/fixture", - bootstrapLock.Base.Reference(), "sh", "-c", script, - ).CombinedOutput() - require.Error(t, err, "tampered archive was accepted: %s", output) - require.Contains(t, string(output), "FAILED") - require.NoFileExists(t, filepath.Join(fixture, "executed"), - "tampered archive was extracted and executed") +// Verify the rendered recipe's commands on the host, with no Docker prerequisite. +// The output graph is the evidence: unrelated DB/cloud drivers must not sneak +// back into the bootstrap binary even when the parent go.mod needs them. +func TestBootstrapMigrateSourceBuildGraph(t *testing.T) { + for _, arch := range bootstrapArchitectures { + t.Run(arch, func(t *testing.T) { + dir := t.TempDir() + require.NoError(t, stageBootstrapMigrate(dir)) + output := filepath.Join(dir, "migrate-"+arch) + script := bootstrapMigrateBuildScript(t, renderBuilderTemplate(t, "templates/builder/Dockerfile.tmpl", DockerTemplating{})) + script = strings.ReplaceAll(script, "-o /migrate", "-o "+output) + command := exec.Command("sh", "-c", script) + command.Dir = filepath.Join(dir, "migrate") + command.Env = append(os.Environ(), "TARGETARCH="+arch) + log, err := command.CombinedOutput() + require.NoError(t, err, string(log)) + info, err := buildinfo.ReadFile(output) + require.NoError(t, err) + require.Equal(t, "github.com/golang-migrate/migrate/v4", info.Main.Path) + require.Equal(t, bootstrapLock.Migrate.Version, info.Main.Version) + require.NotEmpty(t, info.Main.Sum, "upstream source must be checksum pinned") + require.Len(t, info.Deps, 1, "only the PostgreSQL driver dependency belongs in this binary: %+v", info.Deps) + require.Equal(t, "github.com/lib/pq", info.Deps[0].Path) + require.Equal(t, "v1.12.3", info.Deps[0].Version) + settings := map[string]string{} + for _, setting := range info.Settings { + settings[setting.Key] = setting.Value + } + require.Equal(t, "linux", settings["GOOS"]) + require.Equal(t, arch, settings["GOARCH"]) + require.Equal(t, "0", settings["CGO_ENABLED"]) + require.Equal(t, "postgres,file", settings["-tags"]) + }) + } +} + +func TestBootstrapMigrateRejectsTamperedModuleInputs(t *testing.T) { + for _, name := range []string{"go.mod", "go.sum"} { + t.Run(name, func(t *testing.T) { + dir := t.TempDir() + require.NoError(t, stageBootstrapMigrate(dir)) + require.NoError(t, os.WriteFile(filepath.Join(dir, "migrate", name), []byte("tampered"), 0o644)) + script := bootstrapMigrateBuildScript(t, renderBuilderTemplate(t, "templates/builder/Dockerfile.tmpl", DockerTemplating{})) + output := filepath.Join(dir, "output") + script = strings.ReplaceAll(script, "-o /migrate", "-o "+output) + command := exec.Command("sh", "-c", script) + command.Dir = filepath.Join(dir, "migrate") + command.Env = append(os.Environ(), "TARGETARCH=amd64") + log, err := command.CombinedOutput() + require.Error(t, err, string(log)) + require.Contains(t, string(log), name+": FAILED") + require.NoFileExists(t, output) + }) + } } // bootstrapBuildContext stages a build context for the bootstrap Dockerfile: the @@ -439,15 +405,16 @@ func bootstrapBuildContext(t *testing.T) string { []byte(renderBootstrapTemplate(t, parameters)), 0o644)) staged := filepath.Join(context, bootstrapDirectory) require.NoError(t, os.MkdirAll(staged, 0o755)) + require.NoError(t, stageBootstrapMigrate(staged)) require.NoError(t, os.WriteFile(filepath.Join(staged, "bootstrap.sh"), []byte(renderStagedTemplate(t, "templates/bootstrap/bootstrap.sh.tmpl", parameters)), 0o644)) return context } -// bootstrapMigrateInstallScript lifts the migrate install step out of the rendered +// bootstrapMigrateBuildScript lifts the migrate build step out of the rendered // Dockerfile as a runnable shell script, so a test exercises the template's own // commands instead of a re-typed copy of them. -func bootstrapMigrateInstallScript(t *testing.T, dockerfile string) string { +func bootstrapMigrateBuildScript(t *testing.T, dockerfile string) string { t.Helper() var scripts []string lines := strings.Split(dockerfile, "\n") @@ -464,32 +431,10 @@ func bootstrapMigrateInstallScript(t *testing.T, dockerfile string) string { scripts = append(scripts, script) } } - require.Len(t, scripts, 1, "exactly one build step must verify the migrate archive") + require.Len(t, scripts, 1, "exactly one build step must verify the maintained module inputs") return scripts[0] } -// writeTamperedMigrateArchive writes an archive shaped like the migrate release but -// carrying a payload that records being run, so an accepted archive is observable. -func writeTamperedMigrateArchive(t *testing.T, path string) { - t.Helper() - payload := []byte("#!/bin/sh\ntouch /fixture/executed\n") - - file, err := os.Create(path) - require.NoError(t, err) - compressor := gzip.NewWriter(file) - archive := tar.NewWriter(compressor) - require.NoError(t, archive.WriteHeader(&tar.Header{ - Name: "migrate", - Mode: 0o755, - Size: int64(len(payload)), - })) - _, err = archive.Write(payload) - require.NoError(t, err) - require.NoError(t, archive.Close()) - require.NoError(t, compressor.Close()) - require.NoError(t, file.Close()) -} - func bootstrapPinnedVersion(t *testing.T, name string) string { t.Helper() for _, pinned := range bootstrapLock.Packages.Pinned { diff --git a/bootstrap_template_test.go b/bootstrap_template_test.go index 4c7fd10..32f2772 100644 --- a/bootstrap_template_test.go +++ b/bootstrap_template_test.go @@ -286,7 +286,7 @@ func TestBootstrapImageCopiesOnlyTheStagedArtifact(t *testing.T) { t.Fatal("bootstrap image copies the whole service directory, including local secrets") } for _, required := range []string{ - "COPY bootstrap /app/bootstrap", + "COPY --from=migrate-builder /bootstrap /app/bootstrap", "COPY runtime-access.sql /app/runtime-access.sql", `CMD ["/bin/sh", "/app/bootstrap/bootstrap.sh"]`, } { @@ -299,15 +299,13 @@ func TestBootstrapImageCopiesOnlyTheStagedArtifact(t *testing.T) { func TestBootstrapImageIsTargetArchitecturePortable(t *testing.T) { dockerfile := renderBuilderTemplate(t, "templates/builder/Dockerfile.tmpl", testBootstrapTemplating()) for _, required := range []string{ + "FROM --platform=$BUILDPLATFORM " + bootstrapLock.Migrate.Toolchain, "ARG TARGETARCH", - `architecture="${TARGETARCH:-$(apk --print-arch)}"`, - "x86_64) architecture=amd64", - "aarch64) architecture=arm64", - `case "${architecture}" in`, - "amd64) checksum=", - "arm64) checksum=", - `*) echo "unsupported target architecture: ${architecture}" >&2; exit 1 ;;`, - "/releases/download/" + bootstrapLock.Migrate.Version + "/migrate.linux-${architecture}.tar.gz", + `architecture="${TARGETARCH:-$(go env GOARCH)}"`, + `case "$architecture" in amd64|arm64)`, + `GOARCH="$architecture"`, + "CGO_ENABLED=0 GOOS=linux", + `*) echo "unsupported target architecture: $architecture" >&2; exit 1 ;;`, } { if !strings.Contains(dockerfile, required) { t.Fatalf("bootstrap image is not target-architecture portable: missing %q", required) @@ -338,6 +336,9 @@ func TestBootstrapImageBuildsWhenDockerOmitsTargetArchitecture(t *testing.T) { if err := os.MkdirAll(filepath.Join(root, "bootstrap"), 0o755); err != nil { t.Fatal(err) } + if err := stageBootstrapMigrate(filepath.Join(root, "bootstrap")); err != nil { + t.Fatal(err) + } if err := os.WriteFile(filepath.Join(root, "bootstrap", "bootstrap.sh"), []byte("set -eu\n"), 0o644); err != nil { t.Fatal(err) } diff --git a/build_test.go b/build_test.go index 1095a94..bd3b052 100644 --- a/build_test.go +++ b/build_test.go @@ -569,6 +569,10 @@ func TestBuildRecipeLocksBootstrapInputs(t *testing.T) { recipe := response.GetResult().GetDockerBuildPlan().GetRecipes()[0] require.Equal(t, bootstrapLock.RecipeBuildArgs(), recipe.GetBuildArgs()) require.Equal(t, bootstrapLock.LockDigest, recipe.GetBuildArgs()["CODEFLY_BOOTSTRAP_LOCK_DIGEST"]) + require.Equal(t, bootstrapMigrateGoMod, mustReadFile(t, filepath.Join(outputDirectory, "bootstrap", "migrate", "go.mod"))) + require.Equal(t, bootstrapMigrateGoSum, mustReadFile(t, filepath.Join(outputDirectory, "bootstrap", "migrate", "go.sum"))) + require.Equal(t, "sha256:"+recipe.GetBuildArgs()["CODEFLY_BOOTSTRAP_MIGRATE_GO_MOD"], mustFileDigest(t, filepath.Join(outputDirectory, "bootstrap", "migrate", "go.mod"))) + require.Equal(t, "sha256:"+recipe.GetBuildArgs()["CODEFLY_BOOTSTRAP_MIGRATE_GO_SUM"], mustFileDigest(t, filepath.Join(outputDirectory, "bootstrap", "migrate", "go.sum"))) dockerfile, err := os.ReadFile(filepath.Join(outputDirectory, "builder", "Dockerfile")) require.NoError(t, err) diff --git a/builder.go b/builder.go index 49f0386..ae39fa3 100644 --- a/builder.go +++ b/builder.go @@ -475,6 +475,9 @@ func (s *Builder) stageBootstrapContext( if err = s.Templates(ctx, docker, services.WithTemplate(bootstrapFS, "bootstrap", "").WithDestination("%s", staging)); err != nil { return err } + if err = stageBootstrapMigrate(staging); err != nil { + return err + } // Every resolved lineage is staged, including one that carries no migration // yet, so the plan artifact's staged paths always describe real directories. for _, lineage := range plan.lineages { diff --git a/docs/bootstrap-migrate.md b/docs/bootstrap-migrate.md new file mode 100644 index 0000000..c660b86 --- /dev/null +++ b/docs/bootstrap-migrate.md @@ -0,0 +1,59 @@ +# Bootstrap migrate source and security maintenance + +The Builder emits a bootstrap image for each database. Its `migrate` executable +is compiled from the same maintained `go.mod` and `go.sum` embedded in the agent, +using the Go builder image pinned in `bootstrap-image.json`. Only the PostgreSQL +and file drivers are enabled. Cross-compilation runs on the builder's native +platform with CGO disabled; the final image contains neither the Go toolchain nor +module manifests. No core companion supplies this executable. + +This replaces the upstream all-driver v4.19.1 archives used through agent +v0.0.147. Their locked amd64 archive (`2ac648fbd1b127b69ab5a7b33cf96212178f71e22379fc50573630c6f4c7ce18`) +contains Go 1.25.4, x/crypto 0.45.0, x/net 0.47.0, pgx/v5 5.5.4, grpc 1.74.2, +go-jose/v4 4.0.5, and OpenTelemetry 1.37.0 (SDK 1.36.0). Rebuilding that recipe or +upgrading the agent's parent dependencies did not replace those binary bytes. + +The initial source build uses migrate v4.19.1, lib/pq v1.12.3 and Go 1.27.3. +Actual compiled-module inspection records only migrate and lib/pq; the six +vulnerable dependency families above are absent. This is a reduction to the +required drivers, not a claim that unused modules in the parent's full module +graph have been patched by rebuilding. The upstream CLI's built-in stub driver +is also present; no additional external database or remote-source driver is +selected. Existing migration SQL, ledgers, locks, budgets and dirty-state +behavior are unchanged. + +The source-stage checksum check verifies both staged module files before +`go build -mod=readonly`. The checksummed Go modules authenticate downloaded +source. `GOTOOLCHAIN=local` prevents an implicit compiler download. Literal image +labels and typed recipe arguments record the Go image digest, migrate version, +driver set and both module-file SHA256 values alongside the existing base/apk +provenance. Normal context hashing includes the staged source files. The build +copies only the compiled executable and cleaned bootstrap artifacts into the +runtime, so an SBOM describes shipped code rather than unused parent manifests. + +Use `.github/scripts/update-bootstrap-migrate.py` to resolve the migrate portion +of the lock without Docker. It reads migrate's version from `go.mod`, retains the +existing exact compiler tag, verifies the registry manifest digest and checks +both target platforms. `MIGRATE_GO_IMAGE=golang:-alpine` deliberately +selects a newer compiler. The existing full `update-bootstrap-lock.sh` calls the +same resolver while also refreshing Alpine/package pins on both architectures. +No lock digest is manually authored. Dependency and toolchain changes require the +same source-build tests; the strict binary dependency inventory intentionally +fails if another module enters the binary and requires review. + +Host verification: + +```sh +go test . -run '^(TestBootstrapMigrate|TestBootstrapDockerfile|TestBootstrapProvenance|TestParseBootstrapImageLock|TestBootstrapImageLock|TestBuildRecipeLocksBootstrapInputs)' -count=1 +go build ./... +go vet ./... +go mod tidy -diff +``` + +Before rollout, run the Docker bootstrap tests for both architectures and the +existing bootstrap parity tests in an allocated engine slot. Then build through +the normal released/local agent recipe, record each resulting migration image's +digest, and scan the actual images. A host binary inspection is source evidence; +it does not establish scanner clearance for the Alpine packages or deployed +images. Old agent releases continue to emit the old archive recipe, so consumers +must adopt the corrected agent and rebuild all migration images before deployment. diff --git a/templates/builder/Dockerfile.tmpl b/templates/builder/Dockerfile.tmpl index 1c81195..a419cfa 100644 --- a/templates/builder/Dockerfile.tmpl +++ b/templates/builder/Dockerfile.tmpl @@ -1,3 +1,21 @@ +# Compile on the builder's native platform; only the output targets TARGETARCH. +FROM --platform=$BUILDPLATFORM {{ .Bootstrap.Migrate.Toolchain }} AS migrate-builder +ARG TARGETARCH +WORKDIR /src +COPY bootstrap/migrate/go.mod bootstrap/migrate/go.sum ./ +RUN set -eu; \ + printf '%s go.mod\n%s go.sum\n' '{{ .Bootstrap.Migrate.GoModSHA256 }}' '{{ .Bootstrap.Migrate.GoSumSHA256 }}' | sha256sum -c -; \ + architecture="${TARGETARCH:-$(go env GOARCH)}"; \ + case "$architecture" in amd64|arm64) ;; *) echo "unsupported target architecture: $architecture" >&2; exit 1 ;; esac; \ + CGO_ENABLED=0 GOOS=linux GOARCH="$architecture" GOTOOLCHAIN=local \ + go build -mod=readonly -trimpath -tags postgres,file \ + -ldflags='-s -w -X main.Version={{ .Bootstrap.Migrate.Version }}' \ + -o /migrate github.com/golang-migrate/migrate/v4/cmd/migrate +# The runtime contains neither Go nor module manifests: its SBOM describes the +# compiled binary, not the parent's unused module graph. +COPY bootstrap /bootstrap +RUN rm -rf /bootstrap/migrate + FROM {{ .Bootstrap.Base.Reference }} ARG TARGETARCH @@ -16,35 +34,14 @@ WORKDIR /app RUN printf '%s\n' {{ .Bootstrap.Packages.RepositoryArguments }} >/etc/apk/repositories \ && apk add --no-cache {{ .Bootstrap.Packages.Specs }} -# The archive is downloaded to a file and checksum-verified against the lock before -# anything unpacks or runs it, so tampered release bytes never reach tar. The -# base's busybox wget fetches it: the integrity is the lock's sha256, not the -# transport, so no download tool is installed from the rolling package -# repository just for this step. -RUN set -eu; \ - architecture="${TARGETARCH:-$(apk --print-arch)}"; \ - case "${architecture}" in \ - x86_64) architecture=amd64 ;; \ - aarch64) architecture=arm64 ;; \ - esac; \ - case "${architecture}" in \ -{{- range .Bootstrap.Migrate.Archives }} - {{ .Architecture }}) checksum={{ .SHA256 }} ;; \ -{{- end }} - *) echo "unsupported target architecture: ${architecture}" >&2; exit 1 ;; \ - esac; \ - wget -q -O /tmp/migrate.tar.gz "https://github.com/golang-migrate/migrate/releases/download/{{ .Bootstrap.Migrate.Version }}/migrate.linux-${architecture}.tar.gz"; \ - printf '%s %s\n' "${checksum}" /tmp/migrate.tar.gz | sha256sum -c -; \ - tar -xzf /tmp/migrate.tar.gz -C /tmp migrate; \ - install -m 0755 /tmp/migrate /usr/local/bin/migrate; \ - rm -f /tmp/migrate.tar.gz /tmp/migrate +COPY --from=migrate-builder /migrate /usr/local/bin/migrate # Only the staged bootstrap artifact and the runtime-access program enter the # image. The build context is the whole service directory, which also holds # local configuration and secrets, so nothing is copied wholesale. The staged # tree carries only files matching the one migration-filename definition, so an # editor backup left beside a migration never reaches the image to be pruned. -COPY bootstrap /app/bootstrap +COPY --from=migrate-builder /bootstrap /app/bootstrap COPY bootstrap.sql /app/bootstrap.sql COPY runtime-access.sql /app/runtime-access.sql