release: attest reproducible binaries with signed provenance

[seonghobae]

Refs #51, #58.

Writer-safe child of #90 exact 069e11e170cd60b321e28ef4499c88d690a45956. This increment closes the next release-evidence gap only: GitHub OIDC/Sigstore-backed signed build provenance for the already byte-identical release binaries. Production Rust, routing/TLS/HTTP policy, product auth/business logic, Keyverse/Wardnet/EgressWeave authority, supplier source, deployment state, version/tag/package identity, and dedicated baseline lane #61 remain unchanged.

RED → identity-model repair

RED-first exact 7517c3691be1b4437914bb7498f95f105bf98d88 introduced the executable provenance contract before changing the workflow. Its hosted runs were superseded before semantic execution, so it is source-level RED intent only.

The first complete semantic attempt was exact 93b6b3fefc3981fd6dbf687316a9d7ebe6a19fcc, Release Reproducibility run 34633027944. Both Rust 1.98.1 clean builds were byte-identical and actions/attest successfully created GitHub/Sigstore Build Provenance, but verification failed closed because the initial verifier incorrectly expected the explicitly checked-out PR-head SHA to be the GitHub attestation signer/source digest. On a pull_request run, github.event.pull_request.head.sha is the explicitly built checkout while github.sha is the synthetic PR-merge execution commit used by the workflow/OIDC identity.

The causal repair separates those authorities instead of pretending they are the same. EXPECTED_SHA=${{ github.event.pull_request.head.sha || github.sha }} remains exact checkout/build authority; SIGNER_SHA=${{ github.sha }} is the attested source/signer authority. Verification uses an exact certificate SAN, not a path-pattern signer check:

https://github.com/${{ github.repository }}/.github/workflows/release-reproducibility.yml@${{ github.ref }}

Both binaries are verified with --cert-identity, --source-digest=$SIGNER_SHA, --signer-digest=$SIGNER_SHA, repository binding, and --deny-self-hosted-runners. actions/attest is pinned to exact commit 1e69f48acb82d1966a394da916b4c1698aa569d6. Attestation occurs only after the existing #90 byte-for-byte equality gate passes.

Exact-current GREEN

Current exact is bc85547ebed648c03142edad661c57b1b42a2dc4, based directly on #90 069e11e170cd60b321e28ef4499c88d690a45956. Effective child scope is exactly:

• .github/workflows/release-reproducibility.yml;
• tests/release_provenance_workflow_contract.rs;
• docs/doctoring/RELEASE_PROVENANCE_TRACEABILITY.md.

All writer-owned exact-current gates are terminal GREEN:

• CI 34634535392: load-contract, least-privilege OCI runtime, formatting, complete compile/test, Clippy, warnings-denied rustdoc, complete owned-production coverage, and resolved-lock verification all succeeded;
• Supply Chain 34634535334: committed dependency audit, candidate images, SPDX SBOM, both image scans, exact-source binding, and evidence upload succeeded;
• PgErd bounded-origin capacity 34634535434 succeeded;
• TLS H2 Performance 34634535401 succeeded. Exact artifact k6-tls-h2-performance-bc85547ebed648c03142edad661c57b1b42a2dc4 has digest sha256:6e057ad3da593d543af6bc1099af66c8f7ea885e017784cb81a302bd04a514e6. Fresh TLS/H2 buyer-path p95 was 6.46129615 ms, fresh TLS handshake p95 4.34887095 ms, reused buyer-path p95 0.74798925 ms; both phases completed 400/400 HTTP/2 requests with zero request failures, and 396/400 reuse-phase requests were actual connection reuse;
• Release Reproducibility 34634535515 succeeded end-to-end: exact checkout, Rust 1.98.1, compiler/Cargo-authority fail-close, locked fetch, both isolated clean builds, byte comparison, OIDC/Sigstore attestation, separate in-run provenance verification for both binaries, and evidence upload. Exact artifact release-reproducibility-bc85547ebed648c03142edad661c57b1b42a2dc4 has digest sha256:c78e35ab452ee8c7e143b49eedae76bef244a080f83b247cfe7713311b86cc19.

The current reproducibility receipt records checkout_sha=bc85547ebed648c03142edad661c57b1b42a2dc4, attested_source_sha=9bce154cd1cc25744c9bd9aae95b30222649faa7, signer_sha=9bce154cd1cc25744c9bd9aae95b30222649faa7, and certificate identity ending in @refs/pull/91/merge. Both clean builds remained byte-identical: cwl-pingora-gateway SHA-256 f0ae908d3cee73753aa2cc94e97276ef9ba71ff1d1043bba09fb8b545c694292; cwl-pingora-pg-erd-migration SHA-256 415693a89546d9491c75130dc315d680a0d3e3fea04653489e5f1a2ccbd120a9.

Fresh review-thread enumeration is empty. Exact-current technical COMMENT review 5182370343 re-read the complete three-path child range after terminal execution and found no additional writer-safe source/test/documentation/DDD or authority-boundary defect. It is technical evidence only, not independent APPROVED governance credit.

Promotion boundary

A GREEN pull-request attestation authenticates these candidate binary digests to the GitHub Actions certificate identity and the synthetic PR-merge execution digest while separately enforcing which PR-head was checked out and built. It does not turn the PR checkout claim into a certificate-backed Source Repository Digest. Promotion-grade provenance is re-established after protected integration, where the protected source checkout, attested source digest and signer digest converge on the same protected commit.

Foundation #56 remains prerequisite for repository-wide Rust 1.98.1 release-compiler integration; ordinary inherited CI/Supply Chain still run Rust 1.98.0 until that foundation is independently approved and protected-integrated. This child does not source-copy #56.

This PR is technically ready for independent governance review. No self-approval, bypass, protected merge, immutable version/tag/package release, independent-builder proof, OCI reproducibility, representative NUMA closure, shadow/canary, observed rollback, cutover, or Nginx/OpenResty removal credit is claimed.

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: b1e3d073-2257-4c67-8487-2f86d99d8185

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[seonghobae]

@coderabbitai review

Please review current exact 93b6b3fefc3981fd6dbf687316a9d7ebe6a19fcc only. Focus on attestation identity binding, least privilege, mutable-action avoidance, false-GREEN verification paths, and whether candidate provenance claims exceed what the workflow actually proves. This is technical review evidence only; do not treat it as the organization-required independent approving review.

[coderabbitai]

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

target='93b6b3fefc3981fd6dbf687316a9d7ebe6a19fcc'

echo '== Checked-out revision and target commit =='
git rev-parse HEAD
git show -s --format='commit=%H%nparents=%P%nsubject=%s' "$target"
echo

echo '== Changed files against first parent =='
git diff-tree --no-commit-id --name-status -r "$target"
echo

echo '== Exact diff =='
git diff --find-renames --find-copies "${target}^" "$target" -- .github README.md docs 2>/dev/null || true
echo

echo '== Workflow file inventory =='
fd -t f -e yml -e yaml . .github 2>/dev/null || true
echo

echo '== Workflow structural outline =='
for f in $(fd -t f -e yml -e yaml . .github 2>/dev/null); do
  echo "--- $f"
  nl -ba "$f" | sed -n '1,360p'
done

echo '== Attestation and release-related references, with context =='
rg -n -i -C 5 \
  '(actions/attest|attest-build-provenance|attestation|provenance|id-token|attestations|gh[[:space:]]+attestation[[:space:]]+verify|reproducib|permissions:|actions/checkout)' \
  .github README.md docs 2>/dev/null || true

echo '== All action uses and refs =='
rg -n -P '^\s*uses:\s*[^#\s]+@[^[:space:]#]+' .github 2>/dev/null || true

Length of output: 143467

---

🏁 Script executed:

#!/bin/bash
set -euo pipefail

target='93b6b3fefc3981fd6dbf687316a9d7ebe6a19fcc'

echo '== Exact test-only diff =='
git diff --find-renames "${target}^" "$target" -- tests/release_provenance_workflow_contract.rs
echo

echo '== Provenance contract test =='
nl -ba tests/release_provenance_workflow_contract.rs | sed -n '1,420p'
echo

echo '== Non-redacted verification stanza from Git =='
git show "${target}:.github/workflows/release-reproducibility.yml" | nl -ba | sed -n '200,230p'
echo

echo '== GitHub CLI verifier help =='
gh attestation verify --help
echo

echo '== Pull request `#91` checks at the requested head =='
gh pr checks 91 --repo ContextualWisdomLab/pingora-gateway --json name,state,link,workflow 2>&1 || true

Length of output: 18590

---

🤖 web_search executed:

GitHub CLI gh attestation verifydocumentation: explain the exact semantics and accepted values for--signer-workflow, --source-digest, --signer-digest, and --deny-self-hosted-runners; include whether signer workflow identity can be bound to an exact Git commit.

Length of output: 14642

---

@seonghobae

I found one identity-binding defect in .github/workflows/release-reproducibility.yml.

--signer-workflow on Lines 217 and 222 specifies only the workflow path. It does not specify a Git ref. The GitHub CLI accepts this value as a regular-expression match in affected implementations. The path also contains unescaped . characters. Therefore, this check does not require the certificate SAN to identify exactly the intended workflow reference.

--signer-digest "$EXPECTED_SHA" binds the signer workflow revision. It does not make the workflow-path match literal. A different workflow at the same commit can satisfy the repository, source-digest, signer-digest, and loose workflow-path checks.

Use an exact certificate SAN policy, or a fully ref-qualified and safely escaped signer-workflow policy that the installed gh version verifies literally. Extend tests/release_provenance_workflow_contract.rs to reject an unqualified signer identity and to require the selected exact-match policy.

The workflow otherwise has the required narrow attestation permissions, immutable actions/attest pin, post-byte-equality attestation order, two explicit binary subjects, default SLSA provenance predicate verification, source and signer digest checks, and hosted-runner denial.

The required workflow checks were still IN_PROGRESS. This review cannot confirm runtime provenance generation or verification until those checks are terminal.

This is technical review evidence only. It is not an independent approving review.

You are interacting with an AI system.

[seonghobae]

@coderabbitai review

Please re-review current exact b561214b75ace1ccc72ec242aadb8b0a9b647f38 only. The prior identity-binding finding was adopted: path-only --signer-workflow was removed in favor of exact --cert-identity SAN matching, while PR-head source SHA and synthetic merge signer SHA remain independently bound. Check this repair for false-GREEN paths or overclaims. Technical review evidence only; not governance approval.

[coderabbitai]

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

target='b561214b75ace1ccc72ec242aadb8b0a9b647f38'
repo='ContextualWisdomLab/pingora-gateway'

echo '== Target commit =='
git show -s --format='commit=%H%nparents=%P%nsubject=%s' "$target"
echo

echo '== Changed files from first parent =='
git diff-tree --no-commit-id --name-status -r "$target"
echo

echo '== Target diff: workflow, contract test, docs =='
git diff --find-renames "${target}^" "$target" -- \
  .github/workflows/release-reproducibility.yml \
  tests/release_provenance_workflow_contract.rs \
  README.md docs 2>/dev/null || true
echo

echo '== Workflow outline and relevant source =='
ast-grep outline .github/workflows/release-reproducibility.yml --items all 2>/dev/null || true
nl -ba .github/workflows/release-reproducibility.yml | sed -n '1,360p'
echo

echo '== Contract-test outline and source =='
ast-grep outline tests/release_provenance_workflow_contract.rs --items all 2>/dev/null || true
nl -ba tests/release_provenance_workflow_contract.rs | sed -n '1,460p'
echo

echo '== Provenance claims in repository text =='
rg -n -i -C 4 \
  '(attest|attestation|provenance|SLSA|Sigstore|reproducib|cert-identity|signer-digest|source-digest|candidate)' \
  README.md docs .github tests 2>/dev/null || true
echo

echo '== All workflow action references =='
rg -n -P '^\s*uses:\s*[^#\s]+@[^[:space:]#]+' .github/workflows 2>/dev/null || true
echo

echo '== Verification command help and installed version =='
gh --version
gh attestation verify --help
echo

echo '== PR metadata and checks =='
gh pr view 91 --repo "$repo" --json number,state,isDraft,headRefOid,mergeCommit,url,title 2>&1 || true
gh pr checks 91 --repo "$repo" --json name,state,link,workflow 2>&1 || true

Length of output: 224540

---

@seonghobae

I found one documentation overclaim in docs/doctoring/RELEASE_PROVENANCE_TRACEABILITY.md.

The exact SAN repair is correct. --cert-identity "$CERT_IDENTITY" requires an exact certificate SubjectAlternativeName match in the installed GitHub CLI. It removes the prior path-pattern ambiguity. The workflow also retains the separate source and signer digest checks, denies self-hosted runners, uses narrow declared permissions, and pins all referenced actions to full commit IDs.

However, Line 49 states that the successful --signer-digest and --source-digest checks “authenticate” the synthetic merge commit and PR-head source revision. GitHub CLI states that only the signing certificate and verified timestamps are not controlled by the originating workflow. The digest checks constrain fields in the verified provenance statement. They do not independently provide certificate-backed authentication of either commit identity.

Update the text to distinguish these properties:

• The certificate SAN authenticates the signer workflow identifier and GitHub ref.
• The source and signer digest options constrain the provenance statement to the expected source and signer values.
• A same-run verification in a PR workflow is candidate evidence. It is not an independent trusted-builder verification of workflow content or commit identity.

The documentation already states that this is candidate provenance. This change will make that boundary consistent with the GitHub CLI trust model.

All exact-head checks were still IN_PROGRESS, including release-binary-reproducibility. I cannot confirm runtime attestation generation or verification until those checks are terminal.

This is technical review evidence only. It is not an independent approving review.

You are interacting with an AI system.

[seonghobae]

Exact-current technical review on bc85547ebed648c03142edad661c57b1b42a2dc4 (evidence only; not governance approval).

Re-read the complete three-path child range against #90 069e11e170cd60b321e28ef4499c88d690a45956: .github/workflows/release-reproducibility.yml, tests/release_provenance_workflow_contract.rs, and docs/doctoring/RELEASE_PROVENANCE_TRACEABILITY.md.

The current lane keeps byte equality before attestation, pins actions/attest to exact commit 1e69f48acb82d1966a394da916b4c1698aa569d6, separates the explicitly checked-out PR-head EXPECTED_SHA from GitHub's PR-merge execution/signer SIGNER_SHA, verifies both binary subjects against the exact workflow certificate SAN plus source/signer digest, and denies self-hosted provenance. The traceability text correctly limits PR evidence to candidate provenance and requires protected-source execution for promotion-grade convergence of checkout/source/signer identity.

Exact-current hosted execution is terminal GREEN across CI 34634535392, Supply Chain 34634535334, PgErd bounded-origin capacity 34634535434, TLS H2 Performance 34634535401, and Release Reproducibility 34634535515. The provenance job completed both clean Rust 1.98.1 builds, byte comparison, GitHub OIDC/Sigstore attestation, in-run gh attestation verify for both binaries, and evidence upload successfully. Fresh inline review-thread enumeration is empty.

No additional writer-safe source/test/documentation/DDD or authority-boundary defect found in this exact range. This COMMENT is not an APPROVED review and must not be counted as the independent approval required by organization governance.