[antoinetoussaint-byte]

Fixes #1171.

Client and contract generation could merge green because CI never ran the published proto companion tests. This fix adds Linux/Docker qualification of linux-amd64-docker-generate on every pull request, every merge-queue entry, nightly runs and manual dispatch.

What runs

• changes runs scripts/generate-qualification-scope.sh to diff the PR (or queue entry) against its base. Rename detection is off, so a moved file counts by its old path too. The shards are skipped only when every changed path is documentation (docs/ or Markdown outside testdata/). TestDocsOnlyScopeSeesBothSidesOfARename covers code moved into docs/, a test renamed to .md, a testdata .md moved into docs/, and a testdata .md edited. All of these run the shards, and a prose-only change skips them. There is no workflow-level path filter, so a change to any Go package the generators compile (pkg/runnables, pkg/gitops, pkg/composition, …), to go.mod/go.sum, to the scripts or to the workflows runs the shards.
• generate (0..2): three runners split the discovered tests round-robin. Each pulls the published image once through Core's existing proto.CompanionImage resolution and extracts its pinned buf. Each package is compiled once under the row's build tag. Every selected top-level test runs in its own process and Codefly home with CODEFLY_GENERATE_QUALIFY=1. Logs, timings and receipts are kept as artifacts. Each shard has a 15-minute limit.
• docker-generate always reports, and is the one status that can be required. It fails unless changes succeeded and every shard passed. It then downloads every shard's artifact and runs verify-conformance-receipts.sh on the receipts of all shards together. A shard that happens to draw no receipt-bearing test therefore can't go falsely red. It passes without the shards only when changes found nothing but documentation. Making it required is the owner's ruleset call. This PR only makes it possible.

Derived, not listed

scripts/test-generate-isolated.sh takes its row from CODEFLY_CONFORMANCE_REQUIRED, set once at workflow level. It takes its packages and build tag from pkg/conformance/matrix.json and the tagged sources, through scripts/conformance-packages → Row.Packages. The packages are the row's gates plus every package with a file whose //go:build names the row's build_tag. Today that is cmd/generate, cmd/publish and cmd/sync.

pkg/generators drops out of this job: it has no gate and no tagged file. Its tests, including the TypeScript compile check that needs node, already run in go.yml coverage and race, which set up node. No coverage is lost.

The script builds its go test command once. --list prints that command, then walks the run's own loop and prints each package. TestGenerateQualificationScriptRunsTheMatrixRow checks the printed command against the row's build_tag, and the packages against an independent scan of the matrix gates and of every file built under the tag. It also refuses a script that:

• spells the tag or a package itself;
• runs go test other than through the printed command.

It also checks that the row names generate-qualification.yml#generate and that the generate job runs the script.

Mutations verified at 90ab4780, each one failing the test:

• a hard-coded -tags proto_companion_required on the compile command;
• a loop compiling with a bare go test -c;
• rename detection turned back on in the classifier. This fails the three rename cases.

(At 26cf9fd6 the tag claim was not true. --list echoed the derived tag and never read the compile line. r158 g2.)

Reuse

The existing conformance gate, receipt verifier, matrix (gates, build_tag, which nothing read before), proto.CompanionImage and the artifact actions. There is no duplicated image pin, test-name allowlist, generated-client edit or alternative image resolver. actions/download-artifact is pinned to v8.0.2 (9000827c). It errors on a digest mismatch by default.

The Python import test helper now uses a read-only mount and -B. Root-owned bytecode files had been preventing Linux temp-directory cleanup.

qualified keeps its "every change" meaning in docs/supported-matrix.md and in the StatusQualified comment, which now agree. Both name the one exemption, a documentation-only change.

Known gap: image pinned by tag, not digest (owner: core)

The companion resolves to ghcr.io/codefly-dev/proto:<semver> from core's embedded info.codefly.yaml, a mutable tag. A repush of that version changes what this job qualifies, and the extracted buf goes on PATH. The workflow logs RepoDigests but cannot verify them, because there is nothing to compare against. The fix belongs in core: pin the digest next to the version in companions/proto. This job and the tests would then pull by digest through the same resolver. That is out of scope for this PR (r145 f5).

Validation

• Earlier run at de659390: Docker qualification run, all three shards green. 314 test invocations split 105/105/104, 14 receipts, all 9 tagged companion tests passed.
• At the new head: go test ./pkg/conformance/... (plus -race), go vet, golangci-lint, actionlint with shellcheck. One thin local shard ran the script end to end on Docker, including a tagged companion test.
• At 90ab4780: go test ./pkg/conformance/... (plus -race), golangci-lint, actionlint and shellcheck. CI results are in the PR checks.
• At 26cf9fd6, every check is green. In the Docker qualification run, changes found relevant=true. The shards ran 95/94/94 of the 283 discovered tests. docker-generate downloaded all three artifacts and verified 14 receipts across the shards. Not exercised in CI: the documentation-only branch, where the shards are skipped and docker-generate passes, because no docs-only PR has run against this workflow yet.

🤖 Generated with Claude Code