Repository navigation
fix: adopt core v0.18.1 shared service plumbing - #90
Conversation
antoinetoussaint-byte
left a comment
There was a problem hiding this comment.
Deep review (fresh context, Opus) of head 5e3bc8b
Base fab4eac (merge-base with main). Plain clone, pr-90 checked out. I ran go build ./..., go vet ./..., go mod tidy -diff and go test -race -count=1 ./... with GOWORK=off. All pass, and 11 packages report ok. I also ran govulncheck ./.... Core v0.18.1 was read at tag ea49e7a: callertoken/, resources/image_lock.go, agents/services/sbom/evidence.go, .github/workflows/publish-service-image.yml, docs/ci-credentials.md and docs/service-image-plumbing.md.
Verdict: no blockers. It can merge. One major risk remains open: the new publisher's registry credential has never worked in a live run. The release can't proceed until that is proven, and the first chance to prove it is the dispatch after merge.
Findings, ranked
Worth reconsidering (major): the reusable publisher's push credential has never been exercised, and its permissions are ambiguous
.github/workflows/publish-gateway-image.yml:22-31 → core publish-service-image.yml job publish, which sets permissions: contents: read.
The caller job grants packages: write and passes registry-token: ${{ secrets.GITHUB_TOKEN }}. The called publish job narrows its own token to contents: read. GitHub mints one GITHUB_TOKEN per job, and a called workflow can only downgrade permissions. The open question is which token secrets.GITHUB_TOKEN resolves to when the caller passes it as a named secret:
- the caller job's grant, which includes
packages: write, or - the called job's narrowed token, which has
contents: readonly.
Core's docs say the token "travels as a value". That claim is backed only by static internal/ciguard tests.
No run of this workflow exists anywhere:
gh run listreturns nothing for core or service-warehouse.- Code search finds no caller except core's own docs and tests.
The PR also says a live dispatch was not verified. If it resolves to the narrowed token, the failure looks like this:
docker/login-actionsucceeds.build-push-actionbuilds for several minutes, then fails withdenied: permission_denied: write_package.- No lock artifact is produced, and the release is blocked.
This PR replaces a publisher that worked (it published the image for 0.0.14) with one that has never run. The failure is reversible: nothing gets pushed and nothing gets tagged. The way back is to revert this one file, or to fix core, release it and re-pin here.
Two smaller things also change without having run: setup-buildx moves v3 → v4.4.1 and build-push-action moves v6 → v7.4.0. The release.yml:77-95 label check depends on how those versions write org.opencontainers.image.version into each platform's config.
Remediation: make the post-merge dispatch the very first release step and watch it. If the push is denied, fix it in core. Either the called job declares packages: write, which conflicts with ciguard's "reusable declares no write" rule, or core documents a different credential. Do not fix it here.
Worth reconsidering (minor): the release caller still pins core 113 commits behind v0.18.1
.github/workflows/release.yml:146-147
This is the only secrets: inherit call. It still pins go-service-release.yml@08e37db, which is untagged and 113 commits behind ea49e7a. v0.18.1's version adds:
- a gate that runs goreleaser only on a pushed tag
- a refusal of tags that are not on the default branch
contents: readon both jobs
The PR bumps ci.yml "alongside the go.mod core version", but ci.yml carries no secret. The caller that does carry one stays behind, while the PR describes itself as adopting v0.18.1's shared plumbing.
To be honest about the size of this: on a tag push the workflow file comes from the tagged commit, so someone who can push tags can rewrite the pin anyway. The real security gain is small, so this is a consistency issue, not a hole. Bumping looks mechanical: the only input removed is goreleaser-version, and this caller doesn't pass it. Either bump it here, or say in the PR body why it is left alone.
Worth reconsidering (minor): version- and history-specific prose, and stale test messages
README.md:177andREADME.md:337,AGENTS.md:27: "Core v0.18.1'scallertoken…" and "core v0.18.1'spublish-service-image.yml". These go stale at the next core bump, and nothing checks them.README.md:180andAGENTS.md:132: "Caller tokens are now 64…" and "previously 48". This is changelog wording inside reference docs. It belongs in the PR or release notes, and the docs should just state the behaviour.gateway_image_test.go:16andgateway_image_test.go:60: the failure messages still sayparseGatewayImageLock:, a function this PR deletes.
Lenses the coordinator asked about
- Were the copies deleted? Yes.
internal/authis deleted, and so areparseGatewayImageLock/isSHA256Digest(main.go),randomSecret(runtime.go),checkListenerIsAuthenticated(config.go), the subject, inspector,Collect,WriteIndexandDocumentcode (internal/imageevidence), and the inline publisher steps. A grep finds no import of an old path and no remaining symbol. The only leftovers are the two test message strings above. The MinIO lock parser stays, correctly: its schema has release/revision keys that the strictParseImageLockwould refuse. It now uses core'sIsSHA256Digest. - Do the token interceptors behave the same? Yes, apart from the stated changes. The metadata key, constant-time compare, refusal of an empty presented token, and
RequireTransportSecurity=falseare unchanged.- The empty server token now fails closed. That can't happen here, because
cmd/service-object-storage/main.go:73installs interceptors only whencfg.AuthToken != "". - Nothing in the repo calls
Health/List. The Kubernetes startup, readiness and liveness probes (deployment.yaml.tmpl:119-134) are gRPCCheck, and Runtime readiness uses the token-bearing client. The new test covers List both refused without a token and admitted with one. - The longer 64-character token is used in three places:
SOS_AUTH_TOKEN, which has no length check.- The MinIO root password. MinIO enforces only a minimum length, and 48 characters already worked.
- Custody IDs.
minio_custody.go:247only checksID != "", so existing 48-character custody records stay valid.
- The listener error no longer names
x-codefly-tokenor the NetworkPolicy/mTLS hint. That is stated in the PR and moved to the README, and the error is still actionable. - The gateway binary now links only
core/callertoken(checked withgo list -deps ./cmd/service-object-storage). No agent runtime and no docker are linked, which keeps the old file's "don't link the agent runtime" property.
- The empty server token now fails closed. That can't happen here, because
- Does the image lock parse under the strict parser?
gateway-image.jsonholdsghcr.io/codefly-dev/service-object-storageand a lowercase 64-hex sha256, with no extra keys, so it parses.TestGatewayImageMatchesTheLockOnDiskpasses.FullName()is unchanged (ghcr.io/codefly-dev/service-object-storage@sha256:…) even though theDockerImageis now split into Repository and Name.- If the digest were empty,
ParseImageLockreturnsErrImageUnpublished, andshared.Must(main.go:54) panics at agent init. That is the same fail-closed result as before. - In
release.yml,jq -er '.digest'passes an empty string through, andimagetools inspect name@then fails, also closed. - The lock isn't empty today. It holds the image labelled 0.0.14, so
release.yml:77-95will refuse tagv0.0.15until a new image is locked.
- If the digest were empty,
- Does the caller pass any other credential? Only
registry-token: ${{ secrets.GITHUB_TOKEN }}, with nosecrets: inherit. The other checks pass:lock-file: gateway-image.jsonmatches core's<name>-image.jsonregex.- The image name defaults to the repo name, giving
ghcr.io/codefly-dev/service-object-storage. platformsmatchesimageevidence.Platforms, whichsbom_test.goasserts.- The artifact is named
gateway-image.json-<version>. Nothing in the cli, core or warehouse mirrors reads the old artifact name.
- Did
codefly agent deps --pinleave any lock stale? No. The repo has exactly one Go module, the root one: nobase/*module and no factorygo.mod(templates/factoryholds only an env file and a markdown template).go.sumchanges only the core lines, andtidy -diffis clean.agent-ci.ymlstays on core50cfbcf(326 commits behind), but it is dispatch-only and outside #89. - Are the two Moby advisories a real exposure? No. GO-2026-4883 (CVE-2026-33997, an off-by-one in plugin privilege validation) and GO-2026-4887 (CVE-2026-34040, an AuthZ plugin bypass via oversized request bodies) are bugs in the Docker daemon. They list no symbols, so govulncheck flags every
github.com/docker/dockerpackage as "called". This repo uses only the client SDK and API types, in the agent's local-MinIO path. The gateway image doesn't link docker at all, anddocker/docker v28.5.2is the same version on main, so nothing new is introduced. The real exposure sits with the operator's Docker Engine version. - Sprawl: none in the code. The AGENTS.md paragraph on
codefly agent civersuscodefly ci runrelates to step 3 of #89 and is worth keeping. The doc wording is covered by the third finding above.
Merge → release sequence the image lock requires
The reusable publisher refuses a GITHUB_SHA that isn't reachable from the default branch, and it runs only on workflow_dispatch. So it can't be proven from the PR branch, and the image has to be built from code that already contains #90. The current lock is labelled 0.0.14, and that image still carries the old health-prefix exemption.
- Merge #90.
- Run
codefly publish --dry-runto confirm the next version (expected: 0.0.15;agent.codefly.yamlis at 0.0.14 and tagv0.0.14exists). - Dispatch
publish-gateway-image.ymlonmainwithversion=0.0.15. This is the first live run of the new publisher; check the push step (finding 1). - Download the
gateway-image.json-0.0.15artifact and commit it asgateway-image.jsonin a reviewed PR. Merge it. This matches #87 in the 0.0.14 cycle. - Run
codefly publish patch. This creates the release PR that bumpsagent.codefly.yamlto 0.0.15, then the tag.release.ymlthen checks that the image label equals 0.0.15, creates the image tags, writes the evidence, then runs goreleaser.
Do not skip steps 3–4. Running codefly publish patch straight after merge, as #89's step 4 literally says, pushes v0.0.15, and only then does release.yml refuse the stale lock. That leaves a published tag with no release, and the only fix would be a hand-tag, which the rules forbid. Also, no other change to gateway code should merge between step 3 and step 5.
Completeness self-critique
- I didn't run the
integrationore2etags locally (Docker, MinIO, syft). The PR's CI run 38140921014 shows those jobs green at this head. That is acceptable because the code they exercise only changed call sites. - I didn't dispatch the publisher. That is finding 1, and the reviewer can't do it.
- I didn't run
codefly agent ci. - I read core's evidence code only at the API level.
CollectImageEvidenceinternals belong to core's own review. - The GitHub permission semantics in finding 1 are an open question I could not settle, not a confirmed bug.
Single highest risk to keep watching: the first publish-gateway-image.yml dispatch after merge, specifically whether the push is denied because of the narrowed token.
|
Addressed the r22 review in 65315e9:
The focused gateway-lock, image-preparation, shipped-platform and release-order/guard tests passed. All six CI jobs passed on this head: https://github.com/codefly-dev/service-object-storage/actions/runs/38144844678. The successful agent-CI result at implementation head 5e3bc8b remains recorded separately; this follow-up changes only workflow, docs and test diagnostics. The first live publisher dispatch remains a coordinator release prerequisite: verify its push credential and version label. If that fails, fix it in core and re-pin here. No publication, tag or merge was performed. |
Adopts core v0.18.1 so object-storage uses the shared caller-token, gateway image-lock, image evidence and image publisher implementations. Refs #89 and codefly-dev/core#759. Publishing, tagging and merging remain coordinator-owned.
Changes and classification
All changes are fixes at the owning boundary; no hacks or weakened gates.
codefly agent deps --pin v0.18.1, including its standalone build and owned-lock tidy checks. This checkout owns only the root Go module; no base/factory locks exist. NoVisibilityExternalreference remains in this checkout, and build/vet/race validate the other API boundaries. The shared Go CI caller now uses the v0.18.1 commit,ea49e7a968ef5f2fd325b1c41f823a47eb20141c.internal/authand the local listener/token-generation implementations; use corecallertokenon server, client, configuration and secret-generation paths. Move the service-level auth regressions beside the gateway command.resources.ParseImageLock,resources.IsSHA256Digest,sbom.ImageSubjects,CollectImageEvidenceandWriteImageEvidenceIndex.internal/imageevidenceretains only the service's role/platform declaration and delegates to core. MinIO retains its distinct upstream source/release/revision schema, sharing core's digest validation.publish-service-image.yml, passing version, lock filename, platforms and registry secret. Pingo-service-release.ymlto the same v0.18.1 commit: its agent-release job accepts only pushed tags reachable from the default branch, and both jobs declarecontents: read. Retain the service's release ordering and version/digest guard. Tests now check caller wiring and the declared/published platform agreement; core owns the publisher implementation tests.Every behavior change from #89
/grpc.health.v1.Health/Checkand/grpc.health.v1.Health/Watchbypass authentication.Health/Listnow requires the token; the former prefix exemption is deleted. Real gateway RPC tests check both unauthenticated refusal and authenticated admission of List.if cfg.AuthToken != "". The existing explicit anonymous profile still installs none, so its practical behavior is unchanged. Configuration still refuses both or neither auth settings.ghcr.io/codefly-dev/service-object-storage, lowercase SHA-256 hex, only known keys, and one JSON document; an extratagis refused. An empty digest returnsErrImageUnpublished. MinIO's separate metadata-bearing schema retains its parser but uses the lowercase digest validator.ExpectedFromImageReference: registry bare tags are refused before scanning; published linux/amd64 and linux/arm64 each have one subject. Local declarations omit platforms and record the actual daemon platform plus immutable image ID. The shared scan validates expected digests and scans the resolved local ID even if a tag moves.persist-credentials: falseand refuses a triggering commit not reachable from the caller's default branch before registry login/build.Additional final-core behavior: listener errors now carry configuration/exposure rather than deployment instructions; evidence filenames are repository/platform/digest based (
service-object-storage-…, formerlyobject-storage-…), and an unstated index platform occupies-. The inventory checksum remains a canonical BOM checksum, not a formatted-file checksum. The lock artifact is nowgateway-image.json-<version>, formerlygateway-image-lock-<version>; the shared publisher has a 30-minute timeout and derives the image name from this repository.Validation
Review follow-up
65315e9: fresh focused gateway-lock, image-preparation, platform, release-guard and release-order tests passed; all six repo CI jobs passed on this head. No runtime implementation or dependency change was made in this follow-up.codefly agent deps --pin v0.18.1: passed, standalone build and tidy verification included.go build ./...,go vet ./...,go mod tidy -diff, firstgo test -race ./...: passed.Final
go build ./...,go vet ./...,go test -race ./...: passed on the committed implementation.Repo CI at 65315e9: all six jobs passed — build/vet/tidy, unit tests, race, MinIO/GCS integration, runtime e2e and image SBOM. Both backend endpoints/buckets and the built gateway override are set in the run logs. The image command wrote one local document and both linux/amd64 and linux/arm64 registry documents.
Local
go test -tags integration -count=1 -v ./internal/integration/...: six top-level PASS, zero SKIP, covering MinIO and fake-gcs-server. The initial GCS full-stack failure was the emulator's default public host; setting it to Docker's allocated host address made the unchanged suite pass. The redundant local gateway build was stopped after the real-image CI jobs passed.codefly agent ciat implementation head5e3bc8b: passed, 6 passed / 0 failed / 0 skipped with CLI 0.1.181: manifest, source, release build, audit, generated-service conformance and source drift. Produced darwin/arm64 and linux/amd64 agent binaries and CycloneDX SBOMs. Local reports:.codefly/agent-ci/report.jsonand.codefly/agent-ci/workspace/report.json.The requested root
codefly ci runwas executed and correctly refusedno workspace foundbecause this is an agent repository. The successful agent gate generated a real workspace through this agent and rancodefly --local-agents ci run --all --format json --temporary-ports --disposablethere. Its workspace report is passed: 6 tasks passed, 0 failed, 1 skipped; onlysync-driftis skipped withagent_no_sync_capability, as advertised. The test phase runs the agent lifecycle and its explicit zero-native-tests response; the repo's separate integration/e2e jobs provide the real gateway assertions. The agent gate was invoked without skip flags; the CLI's built-in conformance policy leaves actionable vulnerability gating to the preceding release audit.The release audit records two unpatched upstream HIGH Moby advisories, GO-2026-4883 and GO-2026-4887, against
github.com/docker/docker@v28.5.2+incompatible; no upstream fix is available. They are reported under the CLI's existing audit policy, with no policy change here.One clean-tree agent-gate attempt failed downloading Go 1.27.2 inside the Linux builder (
net/http: TLS handshake timeout) and timed out during Docker cleanup. After this run's test containers were removed, the unchanged full gate passed. An earlier run was deliberately interrupted while final edits were still underway. That successful gate result is from the clean implementation head5e3bc8b; the review follow-up changes only the release-workflow pin, reference docs and test diagnostics.Release notes and coordinator handoff
The current lock names the image built for 0.0.14. Do not run
codefly publish patchstraight after merging #90: that pushesv0.0.15beforerelease.ymlrefuses the stale image lock. The coordinator must follow this order:codefly publish --dry-runand confirm the next version (expected 0.0.15).publish-gateway-image.ymlon main withversion=0.0.15. Watch this first live reusable-publisher run, especially registry push permission and the version label. If the credential handoff fails, fix it in core and re-pin here; do not weaken the caller.gateway-image.json-0.0.15, commit its contents asgateway-image.jsonin a reviewed PR, and merge that PR.codefly publish patch. Do not hand-tag.If the dry run reports another version, use that same version consistently in the dispatch and artifact. Keep gateway code stable between image preparation and publication so the locked image describes the release. This worker does not run
codefly publish, dispatch image publication, tag or merge.No live S3, Azure or GCS IAM/Workload Identity validation is claimed. A GCS emulator can prove the SDK/prefix path only. Live reusable workflow publication remains unverified until the coordinator dispatches it from merged history.