fix(noema): bind central handoff to live PR base identity - #2111
fix(noema): bind central handoff to live PR base identity#2111seonghobae wants to merge 1 commit into
Conversation
Route approved OpenCode handoffs to the central receiver and preserve legacy base_branch compatibility. Validate live base identity at admission and before model setup without changing review, provider, or permission policy. Co-authored-by: Codex <noreply@openai.com> Signed-off-by: Seongho Bae <me@seonghobae.me>
|
Warning Review limit reachedNext included review available in 36 minutes. View limit detailsLimit details: You’ve used the included review currently available. You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Review configuration: ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Advanced Run ID: 📒 Files selected for processing (11)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
seonghobae
added
bug
ChatGPT Codex Connector
seonghobae
left a comment
seonghobae
left a comment
There was a problem hiding this comment.
Exact-head review (COMMENT, not approval): inspected the central sender, repository_dispatch admission, pre-model live identity check, and two-phase publisher. The handoff now carries validated base ref/SHA to the real central receiver; receiver admission rejects repository/number/base/head mismatches, while the existing prepared-verdict envelope independently snapshots base at preparation and re-fetches it immediately before publication. No token, provider, model, timeout, or permission broadening is present in this delta. Runtime Quality and Security are GREEN; remaining exact-head gates are still non-terminal, so this is not merge authorization.
![cwl-noema-review[bot]](https://avatars.githubusercontent.com/u/295022177?s=60&v=4){kind=small}
There was a problem hiding this comment.
Noema LLM review
The PR binds Noema handoff to the live PR base identity by routing dispatch to the central repository_dispatch endpoint (repos/ContextualWisdomLab/.github/dispatches), passing pr_base_ref/pr_base_sha alongside the existing target repo, PR number, and head SHA, and enforcing fail-closed live base identity validation at both initial admission (scripts/ci/noema_review_handoff.py dispatch_noema and workflow step 'Admit only the exact live Noema head') and immediately before model run (workflow step 'Validate current pull request head'). Legacy base_branch compatibility is preserved with explicit conflict rejection. The implementation is consistent across the producer script, both workflow shell blocks, docs, and a comprehensive parametrized test suite that executes the actual production shell blocks with a fake gh that never performs external writes. Both distinct functional probes confirm correct behavior: dispatch targets the central receiver with the exact payload, and the receiver binds live base/head identity across canonical/legacy/both, conflict, missing, malformed, base-change, and wrong-repository/PR/head negative cases. The only concern raised — that dispatch_noema's subprocess git check-ref-format may raise FileNotFoundError if git is absent — is not a production regression: the workflow runs on ubuntu-24.04 where git is always present, the identical check-ref-format is already used directly in the shell blocks, and the absence of git would fail closed (the uncaught exception aborts dispatch before any POST). No concrete regression hypothesis was confirmed in the deployed runtime.
Reviewed changed lines
scripts/ci/noema_review_handoff.py:200 (RIGHT): dispatch_noema now validates repo, number, head_sha, base_sha, and base_ref (via subprocess git check-ref-format) before building the payload, raising ValueError on any invalid identity. This fail-closed validation is exercised by test_malformed_dispatch_identity_never_posts which confirms no runner call occurs for invalid inputs. git is guaranteed present on ubuntu-24.04 runners and the same check is used in the shell blocks, so the FileNotFoundError hypothetical is not a deployable regression; absence of git would abort before any API call, i.e., fail closed.scripts/ci/noema_review_handoff.py:224 (RIGHT): Dispatch target changed from repos/{repo}/dispatches to repos/ContextualWisdomLab/.github/dispatches, and payload now includes pr_base_ref and pr_base_sha. test_dispatch_targets_central_noema_receiver verifies the exact central endpoint and complete payload. This corrects the prior producer/consumer endpoint mismatch documented in the runbook..github/workflows/noema-review.yml:65 (RIGHT): Expected base ref/SHA and legacy base_branch env are wired from pull_request_target event or client_payload, resolved with conflict rejection when both pr_base_ref and base_branch are present but differ. The admission shell additionally validates TARGET_REPOSITORY/PR_NUMBER/HEAD/BASE formats and git check-ref-format before any gh call, and the live PR base identity is matched via jq against .number, .base.repo.full_name, .base.ref, and .base.sha..github/workflows/noema-review.yml:83 (RIGHT): The admission step compares the live PR's base identity (number, base.repo.full_name, base.ref, base.sha) against expected inputs and fails closed on any mismatch before the stale-head notice path. This closes the gap where a queued run could proceed with an outdated base..github/workflows/noema-review.yml:612 (RIGHT): The model-time 'Validate current pull request head' step performs the same base identity conflict check, format validation, and live base binding before reading reviews/checking head state, preventing model execution against a stale or mismatched base..github/workflows/opencode-review-dispatch.yml:7721 (RIGHT): PR_BASE_REF and PR_BASE_SHA env are derived from needs.validate-pr-metadata.outputs.base_ref/base_sha and passed to noema_review_handoff.py via --base-ref/--base-sha, carrying OpenCode's verified metadata into the Noema dispatch. The contract assertions in test_required_workflow_queue_contract confirm this wiring.
Adversarial validation
scripts/ci/noema_review_handoff.py:224 (RIGHT)falsified: Dispatch_noema still posts to the consumer repository's own dispatches endpoint instead of the central receiver, preserving the original producer/consumer endpoint mismatch. — test_dispatch_targets_central_noema_receiver asserts calls[0][0] == ['api','-X','POST','repos/ContextualWisdomLab/.github/dispatches','--input','-'] and the payload includes target_repository, pr_number, pr_head_sha, pr_base_ref, pr_base_sha. The hypothesis is falsified..github/workflows/noema-review.yml:83 (RIGHT)falsified: The receiver accepts a dispatch whose base identity does not match the live PR (e.g., stale base ref or SHA), allowing model execution against an outdated base. — test_central_receiver_binds_live_base_and_head parametrizes base-ref mismatch, base-sha mismatch, wrong repository, wrong PR number, and head change, all asserting admitted=false; the canonical/legacy/both-positive cases assert admitted=true. Every mismatch case fails closed with exit 1 and no external API writes (fake gh only permits a specified PR GET). The hypothesis is falsified.scripts/ci/noema_review_handoff.py:200 (RIGHT)falsified: If git is absent from PATH, dispatch_noema raises an uncaught FileNotFoundError instead of failing closed cleanly, potentially masking validation or allowing an invalid dispatch. — This is the only remaining confirmed probe, but it is not a production regression: the workflow runs on ubuntu-24.04 where git is always present; the identical 'git check-ref-format' is already invoked directly in the shell blocks, so the Python path never executes in a git-less runtime. Furthermore, the uncaught exception aborts before any runner/API POST, so even in a hypothetical git-less environment the behavior fails closed rather than dispatching with an unvalidated base_ref. The probe is therefore not confirmable in the deployed environment and does not raise a blocking risk.- Residual risk: dispatch_noema's subprocess git check-ref-format is not mock-isolated in tests and relies on git being present on the runner; this is guaranteed on ubuntu-24.04 and fails closed if absent (uncaught exception aborts before API POST). The remaining operational verification — that an HTTP 204 followed by a trusted-publisher exact-head terminal review is actually produced in production — is explicitly deferred and not provable by local mocks; it is a documented post-deploy operator check rather than a code defect.
Findings
- [low] scripts/ci/noema_review_handoff.py:200 (RIGHT): dispatch_noema validates base_ref via subprocess.run(['git','check-ref-format',...]). git is guaranteed present on ubuntu-24.04 runners and the same check is used directly in the workflow shell blocks, and absence of git would fail closed before any API call, so this is not a blocker. For clarity and test isolation, the check could be wrapped in a small helper, but it is not required.
- Result: APPROVE
- Head SHA:
ec89c6f08f4cfe9756f7e113ca0fb5fc8fead372 - Reviewer credential:
noema-review-github-app-refresh - Actor:
cwl-noema-review[bot]
![opencode-agent[bot]](https://avatars.githubusercontent.com/in/1549082?s=60&v=4){kind=small}
There was a problem hiding this comment.
Pull request overview
OpenCode could not approve from deterministic current-head evidence because GitHub Checks have failed.
Findings
1. HIGH Current-head GitHub Checks - Fix failed required checks before approval
- Problem: Failed same-head checks remain for
ec89c6f08f4cfe9756f7e113ca0fb5fc8fead372. - Root cause: The model-unavailable evidence fallback is allowed only when peer GitHub Checks are complete and clean.
- Fix: Read and fix the failed check logs below, then rerun the current-head checks.
- Regression test: Keep the model-unavailable fallback gated on an empty failed-check rollup.
Failed checks:
- CodeQL PR/CodeQL compatibility analysis (actions): FAILURE (https://github.com/ContextualWisdomLab/.github/actions/runs/34690448456/job/103551299347)
- CodeQL PR/CodeQL compatibility analysis (python): FAILURE (https://github.com/ContextualWisdomLab/.github/actions/runs/34690448456/job/103551300145)
- CodeQL compatibility analysis (actions) check run: failure (https://github.com/ContextualWisdomLab/.github/actions/runs/34690448456/job/103551299347)
- CodeQL compatibility analysis (python) check run: failure (https://github.com/ContextualWisdomLab/.github/actions/runs/34690448456/job/103551300145)
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Workflow: noema-review.yml"]
S1 --> I1["GitHub Actions review job"]
I1 --> R1["Review risk: Workflow: noema-review.yml"]
R1 --> V1["actionlint plus required checks"]
Evidence --> S2["Workflow: opencode-review-dispatch.yml"]
S2 --> I2["GitHub Actions review job"]
I2 --> R2["Review risk: Workflow: opencode-review-dispatch.yml"]
R2 --> V2["actionlint plus required checks"]
Evidence --> S3["Repository file: AGENTS.md"]
S3 --> I3["repository behavior"]
I3 --> R3["Review risk: Repository file: AGENTS.md"]
R3 --> V3["required checks"]
Evidence --> S4["Repository file: CLAUDE.md"]
S4 --> I4["repository behavior"]
I4 --> R4["Review risk: Repository file: CLAUDE.md"]
R4 --> V4["required checks"]
Evidence --> S5["Docs: noema-central-handoff-base-binding.md (2 files)"]
S5 --> I5["operator or user guidance"]
I5 --> R5["Review risk: Docs: noema-central-handoff-base-binding.md (2 files)"]
R5 --> V5["docs review"]
Evidence --> S6["CI script: noema_review_handoff.py"]
S6 --> I6["review and security gate shell path"]
I6 --> R6["Review risk: CI script: noema_review_handoff.py"]
R6 --> V6["bash -n plus Strix self-test"]
Evidence --> S7["Test: test_noema_orchestrator_workflow_contract.py (4 files)"]
S7 --> I7["regression suite"]
I7 --> R7["Review risk: Test: test_noema_orchestrator_workflow_contract.py (4 files)"]
R7 --> V7["targeted test run"]
OpenCode Review Overview
|
|
Exact-head gate RCA for
No source-neutral commit, manual rerun, review override, or leaf workaround is warranted. Keep this PR Ready/Proposed for admission, but do not merge until the protected handler converges and the current-head requested change is cleared by ordinary review. |
|
Current head 다만 OpenCode의 |
Obsolete same-head check-state finding. The review correctly recorded CodeQL compatibility failures on ec89c6f08f4cfe9756f7e113ca0fb5fc8fead372; protected settlement subsequently recovered that unchanged required run to terminal SUCCESS, and Runtime Quality 34690448428, CodeQL 34690448456, SAST 34690448457, Python Security 34690448473, and Security Scan 34690448487 are now all terminal SUCCESS. No source finding or unresolved thread was attached to the review. Dismissal does not replace OpenCode's required status or authorize merge.
|
@opencode-agent Please revalidate the unchanged exact head |
|
Trusted stale-head OpenCode revalidation for exact The original central OpenCode run 34692201345 completed every job successfully, including model review and exact-head receipt enforcement, but published The prior comment-based revalidation did not create a replacement receipt. With no active replacement run observed, I requested a rerun of only the completed OpenCode review job |
|
Exact-head attempt-2 RCA: New exact-head consumer evidence (2026-09-12):
This is not a Run: https://github.com/ContextualWisdomLab/.github/actions/runs/34692201345 The PR remains Ready only for admission. |
1 participant
범위와 원인
G-02 후속. 기준
fb17ef556f94f673234aa557254ae52779e9a7b0, headec89c6f08f4cfe9756f7e113ca0fb5fc8fead372.OpenCode 승인 후 Noema handoff가 소비자 저장소로 repository_dispatch를 보내던 경로를 실제 중앙 receiver로 맞춥니다. 검증된 PR base ref/SHA를 전달하고 중앙 admission 및 기존 모델 직전 live PR 조회에서 repository/number/base/head를 함께 검증합니다. 기존 agent mention의
base_branch도 허용하되pr_base_ref와 충돌하면 거절합니다.권한·token 선택·provider·free-only/ZDR·concurrency·poll/sleep 정책은 바꾸지 않습니다. HTTP 204는 리뷰 완료가 아닙니다. 기존 trusted exact-head terminal review 확인을 유지합니다. #2051 remote terminal model-run binding, 리뷰 자체의 base/workflow/run provenance, webhook owner, Strix→OpenCode→Noema 순차 admission, #2052의 provider 502는 별도 gap입니다.
로컬 검증
실제 송신 함수+기존 추출기로 읽은 실제 수신 shell을 fake-gh로 검증했습니다. 외부 dispatch/실제 모델 호출은 하지 않았습니다.
/tmp/cwl-noema-handoff-red.log)./tmp/cwl-noema-handoff-model-boundary-red.log).-W error, exit 0. 전체 저장소 full suite 결과는 아닙니다.py_compile,git diff --checkexit 0. 기존 dispatch blob pin을 실제 git hash-object50224888f2d7e346afeeb282098a3505e21185e0으로 갱신했습니다.공통 실행:
/Users/seonghobae/cw-central-github/.venv/bin/python -m pytest <아래 그룹> -q -W error -p no:cacheprovider; CI는 앞에GITHUB_ACTIONS=true. cwd는/private/tmp/cwl-noema-central-handoff.jCV6AL입니다. 각 실행은 별도 log/JUnit에 직접 저장했습니다.tests/test_noema_review_handoff.py tests/test_noema_orchestrator_workflow_contract.py tests/test_noema_review_gate.py tests/test_noema_reviewer_token_lifetime.py tests/test_required_workflow_queue_contract.py tests/test_pr_review_autofix_nvidia_nim_contract.py tests/test_opencode_workflow_shell_syntax.py tests/test_agent_mention_downstream_idempotency.pyreceipt
/tmp/cwl-noema-handoff-{normal,ci}.{log,xml}.tests/test_required_review_runner_image_contract.py tests/test_contextual_orchestrator_review_sidecar_contract.py tests/test_docs_only_pr_runner_admission.py tests/test_central_required_workflow_exact_inventory.py tests/test_opencode_agent_contract.py tests/test_central_required_workflow_ruleset_audit.py tests/test_opencode_rust_coverage_toolchain_contract.pyreceipt
/tmp/cwl-noema-handoff-adjacent-{normal,ci}.{log,xml}.tests/test_pr_review_merge_scheduler.py tests/test_repository_branch_coverage_review_schedulers.py tests/test_repository_branch_coverage_reporting_edges.pyreceipt
/tmp/cwl-noema-handoff-scheduler-{normal,ci}.{log,xml}.Coverage:
COVERAGE_FILE=/tmp/cwl-noema-handoff.coverage /Users/seonghobae/cw-central-github/.venv/bin/python -m coverage run --source=scripts.ci.noema_review_handoff --branch -m pytest tests/test_noema_review_handoff.py -q -W error -p no:cacheprovider, followed bycoverage report --show-missingwith the same COVERAGE_FILE.actionlint 한계
Noema 단일 workflow는 원래 actionlint 1.7.12 설정으로 exit 0. 두 workflow의 native YAML/expression 검사(
-shellcheck= -pyflakes=)도 exit 0이며 변경한 3개 shell 블록은 ShellCheck exit 0입니다. 기존 shell-syntax/queue 회귀도 위 결과에 포함됩니다.그러나 OpenCode 단일 전체 actionlint는 제한 시간에서 exit 124였고 기준 fb17ef 원본도 동일 증상입니다. 최초/GOMAXPROCS=2 두파일 실행은 출력 없이 condition wait에 머물렀고, 확인 후 본인 PID만 종료했습니다(exit143). 원인 자체 미확정이며 OpenCode full-semantic 검사 완료를 주장하지 않습니다. 상세 실패 시도와 재발 판별 절차는
docs/doctoring/noema-central-handoff-base-binding.md에 남겼습니다. 다른 세션 PID는 변경하지 않았습니다.실제 GitHub Visual Inspection
검사 SHA
ec89c6f08f4cfe9756f7e113ca0fb5fc8fead372, viewport 1280×720, document langen, GitHub UI 영문/본문 한국어. 각 exact commit blob의 Preview를 실제 브라우저 screenshot으로 검사했습니다.docs/doctoring/noema-central-handoff-base-binding.md: 제목→원인→검증→남은 경계를 세로 스크롤로 확인. 기대한 문단/inline code 줄바꿈과 마지막 한계 문구가 표시됩니다. 표는 없습니다.AGENTS.md#verification-discipline,CLAUDE.md#conventions-and-gotchas-specific-to-this-repo: 추가 한국어 bullet과 runbook 링크 표시를 확인했습니다.docs/product-technical-gap-baseline.md#3-gap-register: 추가 문단과 runbook href가 동일 commit을 가리킵니다. 기존 4열 표는 좁은 본문에서 가로 스크롤이 필요하며 실제 오른쪽 스크롤로 마지막 열까지 확인했습니다. 무가로스크롤이라고 주장하지 않습니다.AGENTS·CLAUDE·doctoring·Gap baseline에 같은 원인과 후속 한계를 기록했습니다. 로컬 GREEN과 Preview는 protected merge/hosted review/provider runtime 성공 증거가 아닙니다.