Skip to content

ci: fingerprint Cargo.toml's profile tables, not the whole manifest (#7282) - #7821

Merged
proggeramlug merged 1 commit into
mainfrom
ci/7282-narrow-baseline-fingerprint
Aug 11, 2026
Merged

ci: fingerprint Cargo.toml's profile tables, not the whole manifest (#7282)#7821
proggeramlug merged 1 commit into
mainfrom
ci/7282-narrow-baseline-fingerprint

Conversation

@proggeramlug

@proggeramlug proggeramlug commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

The public-baseline freshness gate no longer fires on Cargo.toml changes that cannot move a measured number (#7282, proposal 1 — the issue's "single biggest win").

Cargo.toml was fingerprinted whole-file. A new perry-ext-* workspace member, a dependency bump or a [workspace] restructure invalidated the published artifact without touching a benchmarked kernel — #6758/#6761's restructuring did exactly that, and the artifact then sat 40+ commits stale on a REQUIRED check, so every later lint step (file-size, GC store-site inventory, addr-class audit, #7253's gate-wiring check) never executed in CI at all, and every merge needed an --admin bypass.

Only the [profile.*] tables now participate — opt-level, lto, codegen-units, panic, i.e. the things that genuinely change comparability. Extraction is textual and conservative (no TOML parser is guaranteed in the CI Python, and generator and checker must agree byte-for-byte); a section header ends the capture unless it is itself [profile…, so [profile.release.package.x] subtables are kept. Measured: 19,466 bytes → 8,681, all 45 profile tables retained, [workspace.package] and every dependency line gone.

Sabotage-verified in both directions, by exit code rather than by message:

planted change ci_public_baseline_check.py
[profile.release] opt-level = 3 → 2 exit 2 — "benchmark inputs changed; regenerate it"
new crates/perry-ext-sabotage workspace member exit 0

So the gate keeps blocking absolutely on what matters and stops firing on what does not — the issue's explicit requirement that it stay required and hard-failing.

On the artifact's stored digests. Narrowing the fingerprint changes both keys, so they are recomputed in the same commit. That is sound rather than an attestation: the artifact matches under the broad fingerprint today (verified before the change), and the narrow fingerprint covers a strict subset of those inputs — an artifact valid under the broad one is necessarily valid under the narrower one. No measurement was re-run and none needed to be: everything in the file except the two freshness digests is byte-identical, asserted programmatically. Independently, the [profile.*] extract is unchanged from the artifact's own commit (38ff7eccc) through HEAD, and across HEAD~40 and HEAD~120.

Also worth recording: the issue's headline complaint — "Cargo.toml changes on every version bump" — was already fixed before this change. _fingerprint_bytes normalizes the version = "…" line to 0.0.0, which is why the gate was green today despite 0.5.1355 → 0.5.1461. This change addresses what was left.

Scope is proposal 1 only. HARNESS_PATHS still fingerprints scripts whose error handling cannot change a number (#7265's run.sh fix is the cited example); proposal 2's measurement-affecting/plumbing split is untouched. The deliberate circularity is preserved — ci_public_baseline_check.py stays out of public_baseline.py so the checker's own file is not in HARNESS_PATHS.

Verified: tests/test_public_baseline.py 7 passed, benchmarks/ci_public_baseline_check.py exit 0, file-size gate clean.


Also fixes #7290

Fixes #7290, which reports the same gate reddening main on dependency
churn specifically: SOURCE_PATHS included Cargo.toml whole-file, so a
dependabot/taze bump, a new dependency, or a new registry entry invalidated the
published artifact. #6528 had normalized only perry's own version line.

With the fingerprint narrowed to the [profile.*] tables, none of those three
participate any more — measured on the current manifest, extraction drops
19,466 bytes to 8,681 with every dependency line gone. That is #7290's first
suggested direction ("narrow SOURCE_PATHS so Cargo.toml contributes only
what actually affects benchmark output, e.g. the [profile.*] sections"),
which also subsumes its second (normalizing dependency pins) — a line that is
not hashed at all cannot need normalizing.

#7290's other observation — that a failing lint step silently skips the eight
gate steps after it, because lint is a step sequence — is a separate defect
in the job's structure and is #7278's scope, as #7290 itself notes. It is not
addressed here.

@coderabbitai

coderabbitai Bot commented Aug 11, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@proggeramlug, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 21 minutes

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: bc0ae11b-6c03-45a0-8bcd-6069ccfaef04

📥 Commits

Reviewing files that changed from the base of the PR and between 1804991 and 8c20ff3.

📒 Files selected for processing (3)
  • benchmarks/public_baseline.py
  • benchmarks/results/public-node-bun-v1.json
  • changelog.d/7821-narrow-baseline-fingerprint.md
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch ci/7282-narrow-baseline-fingerprint

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@proggeramlug
proggeramlug force-pushed the ci/7282-narrow-baseline-fingerprint branch from 2cd69d3 to 8c20ff3 Compare August 11, 2026 00:12
@proggeramlug
proggeramlug merged commit 4a49da4 into main Aug 11, 2026
14 of 18 checks passed
@proggeramlug
proggeramlug deleted the ci/7282-narrow-baseline-fingerprint branch August 11, 2026 05:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Public baseline freshness gate reddens main on unrelated dependency churn

1 participant