[antoinetoussaint-byte]

What leaked, and where

The bootstrap Job this agent renders for a ClickHouse store logged the store's password whenever it failed to connect. Seen on a hosted cell: the Job for a ClickHouse store logged, on each failed attempt before it eventually completed,

error: failed to open database, "clickhouse://<svc>:80?username=<user>&password=<THE PASSWORD>&database=<db>&x-multi-statement=true": driver: bad connection in line 0: SHOW TABLES FROM ...

Root cause, in this repo:

• templates/builder/Dockerfile.tmpl downloaded the golang-migrate CLI v4.18.1 and its entrypoint hands it a URL with the credentials in the query (the only place clickhouse-go v1 reads them). v4.18.1's migrate.New wraps a failed database.Open as failed to open database, %q with the whole URL. The agent's own in-process migrations already linked v4.19.1 (go.mod), so the CLI had drifted from the library.
• v4.19.1 dropped the URL from that message, but its clickhouse driver (database/clickhouse/clickhouse.go, Open: url.Parse(dsn) then the tcp rewrite handed to clickhouse-go v1) still returns net/url's parse error, which quotes the whole URL, password included, for an address that does not parse. Reproduced locally with the v4.19.1 CLI: parse "clickhouse://bad host:9000?username=…&password=<fake>&…": invalid character " " in host name.
• In-process (codefly run), nativeDSN / adminDSN composed the DSN by Sprintf with unescaped credentials; clickhouse-go v1's open returns the same url.Parse error, which the agent wraps into its logs.

The fix

• Migrate release has one source. The Dockerfile downloads {{ .MigrateVersion }}, which Build reads from this agent's build information (the github.com/golang-migrate/migrate/v4 it links), so the image and the in-process path run the same release, now v4.19.1. A pseudo-version or local replacement is refused by name. curl -fsSL so a missing release fails the build instead of feeding tar an error page.
• Redaction filter in the entrypoint. Everything migrate prints goes through awk, which replaces the password value literally (read from ENVIRON, no regex or escape processing) and any password= query value with <redacted>. migrate's exit status is carried past the pipe on fd 3 (Ubuntu's dash and POSIX sh have no pipefail), so a failed migration still fails the Job. The migrate process is no longer execed, so sh is PID 1 in the Job's pod.
• Deploy refuses an unparsable address (checkAddress) naming only the address, so a render cannot hand the Job an address whose failure would quote the URL.
• In-process DSN always parses: nativeDSN(address, user, password, database) escapes the query with url.Values and checks the address first; adminDSN uses it too. A password containing #, % or & is now carried correctly instead of corrupting the DSN.
• Agent README (templates/agent/README.md.tmpl) describes the behaviour.

Classification: a fix at the owner. This agent composes the URL and chooses the CLI that prints it. The residual upstream behaviour (golang-migrate's clickhouse Open returning a *url.Error that quotes the DSN) is contained here by the address check plus the filter. No upstream issue was filed.

Reuse: the existing migrate CLI, the existing entrypoint and its variable carriers, the image's busybox awk, and the Go build information. No new binary, dependency or mechanism.

Tests

entrypoint_redaction_test.go:

• TestBootstrapEntrypointNeverPrintsThePassword renders the Dockerfile through Build, extracts its shell-form CMD, builds the migrate CLI at the linked release, and runs the entrypoint with sh -c against an unreachable server, an unparsable address, and a password that is a substring of the marker. It checks the output never contains the password, that the failure is still reported, and that the exit status is non-zero. It fails on main (the unparsable-address case prints the fake password).
• TestMigrationImageRunsTheLinkedMigrateRelease: the Dockerfile downloads the release go.mod links. It fails on main (v4.18.1 vs v4.19.1).
• TestDeployRefusesAnAddressTheMigrateURLCannotCarry: it fails with the check removed.
• TestNativeDSNNeverQuotesThePassword: an unparsable address is refused without the password, a password with URL metacharacters round-trips, and a ping error to an unreachable server does not carry it.

Local: go build ./..., go vet ./..., go mod tidy -diff, go test ./... all pass.

End-to-end, by hand: I built the rendered recipe with docker build (alpine 3.21, busybox sh/awk, migrate 4.19.1) and ran it as uid 65534 with a read-only root filesystem:

• unreachable host or unresolvable name: the error is printed without the URL, exit 1
• unparsable address: password=<redacted>, exit 1
• real clickhouse/clickhouse-server:26.3 with a wrong password: Authentication failed, no password, exit 1
• the right password: 1/u create_table, exit 0

What I did not verify

• I did not cut a release, so no cell runs this yet. The leaking Job on the cell keeps the old image until a release of this agent is pinned and the store is re-rendered. The new image and the new MigrateVersion change the Job's pod template, so the render names a new Job, which re-applies the (idempotent) migrations.
• The filter catches the password as raw text and as a password= query value. A password with " or \ would reach a %q-quoted error escaped, and only the password= rule would catch it. Codefly-generated passwords do not contain these characters.
• The entrypoint still puts the password unescaped into the migrate URL (as before), so a password containing & or # would not authenticate in the Job. This is pre-existing and out of scope here.
• Rotating the password that already leaked is an operator step and is not part of this PR.

🤖 Generated with Claude Code