[flow-research-agents]

Change

WS-GHPROOF-001-03 - Compare Harnessy proof with GitHub facts

Goal

harnessy_proof fails a proof that does not match GitHub's facts, with one finding per mismatch.

Intent And Planning Context

• Intent: the comparison half of GitHub proof checking; the checker stays networkless and stdlib-only.
• Commitrail record or PR-only reason: WS-GHPROOF-001-03.md. Stacked on [WS-GHPROOF-001-01] Record GitHub proof facts initiative #681; independent of [WS-GHPROOF-001-02] Read-only GitHub App facts adapter #682.

What Changed

• checker.py: when the request input carries github_facts (the shape produced by chunk 02's model_dump(mode="json")) or configuration requires facts, it compares repository (configured and PR URL), PR number, merged, merge commit SHA against commit_sha, base branch against configuration, and every required check success. Missing facts when required is an error finding. Existing shape checks remain.
• publish_local.py: real per-phase configuration and input schemas; default version v2. The PRE configuration accepts only {}.
• docs/engineering/local-agent-loop.md: documents the mode.

Why It Changed

Today a proof with any well-formed values passes.

Design Chosen

Facts arrive in the hash-bound request input, not a second ART mount, so no Rust grant change is needed.

Alternatives Rejected

• A second mounted material role: needs grant changes and duplicates the input binding.

Scope Control

Allowed Files Changed

• The files listed above.

Files Outside Stated Scope

• None

Product Behavior

• Product behavior changed and is explained here: the local harnessy_proof checker gains a facts-comparison mode, active only when facts are supplied or required by configuration. Registration is local-only.

Evidence

Commands Run

pytest backend/tests/checkers/external/test_harnessy_proof_checker.py plus local pilot script tests
ruff; Agent Gates

Result Summary

56/56 checker tests pass; 86 passed with the local pilot scripts. Ruff and gates pass.
A scratch check (not committed) confirmed chunk 02's facts pass this checker with no findings; chunk 04 should commit that agreement test.

Acceptance Criteria Proof

• Matching proof passes.
• Each mismatch fails with its own finding: repository, PR number, merged, merge SHA, base branch, failed or missing required check.
• Facts required but missing fails.

Test Delta

Tests Added

• New cases in test_harnessy_proof_checker.py

Tests Modified

• test_harnessy_proof_checker.py

Tests Removed Or Skipped

• None

Impact-Routed Reviewer Results

Reviewer	Result	Blocking findings	Proof boundary and uncertainty
security	PROVISIONAL	none	Author-side review only.

Remaining Risks

The checker version bump requires re-registration in local loops.

Follow-Up Work

Chunk 04 supplies facts in production evaluation.

Human Review Focus

Please inspect:

• The comparison rules and the facts input shape.

Update: integrity rules and security-review fixes

• One fail-closed mode. Missing configuration or missing facts is an error; the shape-only pass path is removed. The checker is POST-only, because PRE never has facts.
• Integrity rules (configurable, safe defaults):
  • A required check open to any source fails (github.unbound_required_check).
  • A change to .github/**, CODEOWNERS or other CI configuration, rename sources included, can never be auto-accepted.
  • At least one approval on the head commit from a User who is not the author is required. Each reviewer's latest decisive review counts.
  • If only configured operators approved, a harnessy.operator_approved warning is raised: the work earns points but cannot be paid in money until an independent reviewer confirms.
• Exact identity.
  • pr_url must equal https://github.com/{repo}/pull/{n} and GitHub's html_url.
  • ci_run_url must equal https://github.com/{repo}/actions/runs/{id} for a judged required run.
  • Any check entry naming another commit fails (github.facts_binding).
• Tests: 108/108.

CI And Gate Integrity

• No workflow weakening.
• No lint/test/docstring gate weakening.
• Tests protect meaningful behavior; removals identify retained proof or retired requirements. Coverage percentages are diagnostic only.
• No package script weakening.
• No unpinned new GitHub Action.
• Checkout credential persistence disabled where checkout is used.

Human Merge Ownership

The human completes the acceptance/merge checks; agents must not tick them on
the human's behalf or infer approval from a passing CI status.

• I can explain what changed.
• I can explain why it changed.
• I know what could break.
• I accept the remaining risks.
• The user explicitly approved this specific PR for merge.

🤖 Generated with Claude Code

[coderabbitai]

Important

Review skipped

Bot user detected.

To trigger a single review, invoke the @coderabbitai review command.

Run configuration

• Source mode: Auto
• Repository: Unavailable
• Branch: Unavailable
• File: No configuration file from this repository applied
• Commit: Unavailable
• Layers: defaults
• Review profile: CHILL
• Plan: Advanced
• Run ID: 31823e0d-03b6-406e-a8fa-75625f3941c6

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}