Skip to content

test(ci): stage protected docs-only classifier foundation - #287

Open
seonghobae wants to merge 7 commits into
mainfrom
fix/docs-only-ci-classifier-foundation
Open

test(ci): stage protected docs-only classifier foundation#287
seonghobae wants to merge 7 commits into
mainfrom
fix/docs-only-ci-classifier-foundation

Conversation

@seonghobae

@seonghobae seonghobae commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Foundation repair for #279 / #282

This PR splits the trusted documentation-only CI classifier out of #282 and stages it on a workflow-free prerequisite branch before any workflow is allowed to use it as suppression authority.

Protected main is exact 87c4daa1830bac5a5228b6036752ad5633232085. Exact current head is 3975daf48e01a5e9d1cf9fb104a3be1aa03b0402, directly based on that protected generation and still Ready/mergeable.

Why this prerequisite is necessary

Current #282 correctly attempts to execute scripts/ci/classify_ci_change_scope.py from the protected base SHA and fails closed to full Rust verification when that file is absent from the base. However, #282 also carries the .github/workflows/ci.yml mutation that consumes the classifier. That is both a bootstrap problem and a writer-ownership problem: the decision code that can suppress heavy CI must become protected immutable authority before an authorized workflow owner wires it into CI, and this product/source lane does not own .github/**.

This prerequisite therefore preserves only #282's non-workflow classifier and focused security regressions plus truthful Unreleased CHANGELOG entries. There is no .github/**, Rust production, browser/policy, dependency, ruleset, secret, release, or publication delta.

Security contract retained from #282

The classifier consumes NUL-delimited non-combined git diff --raw -z --no-abbrev evidence and fails closed unless it can prove an ordinary prose-only change. The retained regressions cover mode/path authority, complete SHA-1/SHA-256 object identity, malformed or truncated records, impossible mode/OID combinations, symlink/gitlink/type-change and executable surfaces, non-canonical paths, contributor-instruction authority, prose boundaries, and rename/copy similarity-to-object-identity coupling.

The first split incorrectly retained a test that read/asserted .github/workflows/ci.yml, making the prerequisite depend on the future activation it was supposed to precede. ab6d5d1a04a020f37727e2f5a6950c58feae8559 removed only that workflow-specific test. af83c40dd2990a03064a92ca75430a9cc400f098 added the truthful CHANGELOG statement for the rename/copy similarity-to-blob-identity invariant.

A subsequent repository-root regression showed that the classifier tests imported scripts.ci... but default pytest collection failed with ModuleNotFoundError: No module named scripts; shell-only PYTHONPATH=. was not accepted as a durable fix. Stacked successor #300 exact 6bc57663b3bf36dd7e4a2750e2cd5d615f450cb2 moved that authority into checked-in pytest.ini with pythonpath = .. Its exact CI 34315238829 completed success: Python repository contracts, canonical formatting, workspace tests, strict Clippy, rustdoc, and exact production coverage all passed. #300 was then normally merged into this foundation as merge commit 3975daf48e01a5e9d1cf9fb104a3be1aa03b0402. No force update or history rewrite was used.

Exact-current verification

Exact current 3975daf48e01a5e9d1cf9fb104a3be1aa03b0402 now has terminal same-head evidence:

  • CI 34329801487: success;
  • Security Scan 34329801483: success;
  • SAST Semgrep 34329801598: success; and
  • CodeQL PR 34329801477: failure.

The CodeQL failure is not an observed classifier-source finding. Detect-languages job 102395468332 succeeded. Compatibility jobs 102397673743 (python), 102397673794 (actions), and 102397673913 (javascript-typescript) each successfully read the current-head dispatch verdict and failed only at Release runner or enforce current-head CodeQL verdict; later dispatch job 102400827942 succeeded. Exact specimen was handed to canonical ContextualWisdomLab/.github#712 in comment 5599230322. Keep this leaf fail closed: do not synthesize success, add a no-op commit, duplicate central CodeQL, or transfer another head's verdict.

The successful repository/security lanes establish exact-tree repository evidence only. Independent approval, the required central CodeQL verdict, protected-main integration, workflow activation, and release remain separate gates.

After this classifier foundation reaches protected main through normal required gates, the authorized #279/#212 workflow-owner path must reconstruct #282's workflow delta against that protected classifier generation and preserve current MV3 lifecycle/concurrency/security semantics. Only then can #283 prove that repository/documentation contracts execute while Rust/coverage heavy jobs do not materialize for a genuine prose-only delta.

Keep #282 open: its workflow/config delta is valid successor work for the authorized owner and must not be discarded. This PR is a prerequisite split, not a Close/supersession shortcut.

No self-approval, bypass, force-push, destructive rebase, workflow/ruleset/secret mutation, gate weakening, tag, release, or publication is authorized.

Summary by CodeRabbit

  • 새 기능

    • 변경 범위를 자동으로 판별해 일반 문서 변경에는 간소화된 검사를 적용하고, 코드·설정 변경에는 전체 검사를 요구합니다.
    • 파일 이동·복사, 경로 형식, 파일 유형 및 객체 식별자를 검증해 잘못된 변경이 검사를 우회하지 못하도록 했습니다.
  • 문서

    • 테스트 실행 시 저장소 표준 import 경로를 사용하도록 개발 가이드를 보완했습니다.
  • 테스트

    • 문서 전용 변경, 경계 사례, 잘못된 입력 및 파일 이동·복사 처리에 대한 회귀 테스트를 추가했습니다.

@coderabbitai

coderabbitai Bot commented Sep 4, 2026

Copy link
Copy Markdown

Review Change StackReview Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: 2e09bb7f-1b70-4508-bf4e-896453e611a8

📥 Commits

Reviewing files that changed from the base of the PR and between ab6d5d1 and 3975daf.

📒 Files selected for processing (4)
  • AGENTS.md
  • CHANGELOG.md
  • CLAUDE.md
  • pytest.ini

Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review.


📝 Walkthrough

Walkthrough

문서 전용 CI 변경을 판별하는 CLI를 추가했습니다. Raw diff와 name/status 입력을 검증합니다. 모드 정보가 없거나 입력이 잘못되면 Rust 검사를 요구하도록 fail-closed 처리합니다. 회귀 테스트와 테스트 import 설정도 추가했습니다.

Changes

CI 변경 범위 분류

Layer / File(s) Summary
Raw diff 계약과 검증
scripts/ci/classify_ci_change_scope.py
NUL 구분 raw diff와 legacy name/status 입력을 파싱합니다. 경로, 모드, 객체 ID, 상태, rename/copy 유사도 규칙을 검증합니다.
문서 범위 분류와 CLI 출력
scripts/ci/classify_ci_change_scope.py, CHANGELOG.md
일반 문서 경로와 파일 모드만 문서 전용으로 분류합니다. 빈 입력, mode-blind 입력, 비문서 변경은 Rust 필요로 처리합니다. CLI는 GitHub Actions 출력 형식을 생성하고 오류 시 종료 코드 2를 반환합니다.
기본 분류와 CLI 검증
tests/test_ci_change_scope.py, AGENTS.md, CLAUDE.md, pytest.ini
문서 및 코드 변경, 특수 Git 모드, 잘못된 경로와 스트림, 출력 형식, CLI 오류 처리를 검증합니다. 저장소 루트 scripts 모듈 import에 pytest.ini 설정을 사용하도록 문서화합니다.
Fail-closed 회귀 검증
tests/test_ci_change_scope_modes.py, tests/test_ci_change_scope_object_ids.py, tests/test_ci_change_scope_path_identity.py, tests/test_ci_change_scope_prose_boundary.py, tests/test_ci_change_scope_similarity_scores.py
Mode-blind 입력, 객체 ID 의미, rename/copy 경로 동일성, 문서 경계, 유사도 점수 형식을 검증합니다.

Priority: ⬇️ Low

Estimated code review effort: 4 (Complex) | ~45 minutes

Merge Risk: ⚪ Minimal · up to 3975d

This change adds a fail-closed documentation-only CI classifier and repository-root pytest import configuration. Invalid, incomplete, or non-documentation changes continue through Rust validation, with no active merge-readiness risk identified.

Sequence Diagram(s)

sequenceDiagram
  participant Git
  participant ClassifierCLI
  participant ScopeClassifier
  participant CIOutput
  Git->>ClassifierCLI: raw diff 또는 name/status 입력
  ClassifierCLI->>ScopeClassifier: 변경 범위 파싱 및 검증
  ScopeClassifier->>CIOutput: documentation_only, rust_required
  CIOutput-->>ClassifierCLI: GitHub Actions 출력
Loading
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 74 functions across 7 files. (4 skipped: 4…
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed 제목은 보호된 문서 전용 CI 분류기 기반을 단계적으로 추가한 주요 변경을 정확하고 간결하게 설명합니다.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/docs-only-ci-classifier-foundation

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@seonghobae seonghobae left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exact-head foundation review for bb052af8a9c27d88561bd5b2f98b6a7a74808af9: fresh compare against protected main@87c4daa... is 1 ahead / 0 behind with merge base exactly protected main. The effective delta is only the classifier plus six focused Python security-regression files; there is no .github/** delta. This is the correct prerequisite shape for #282's protected-base classifier execution. Native CI 33924362459 is intentionally skipped while Draft under #286 and is not GREEN; Security/Semgrep/CodeQL are queued. Keep Draft until the truthful classifier-only CHANGELOG debt is repaired and a Ready transition produces fresh exact-head verification.

@seonghobae
seonghobae marked this pull request as ready for review September 4, 2026 23:03

@seonghobae seonghobae left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Current-head review for ab6d5d1a04a020f37727e2f5a6950c58feae8559: the foundation is now workflow-independent. The first split still carried a test that read and asserted the future .github/workflows/ci.yml activation, which inverted the prerequisite relationship; that test has been removed while the classifier and fail-closed security regressions remain. The ready_for_review event on predecessor 7688798e... produced CI 33928039835 as skipped, so it is not GREEN. Fresh synchronize CI 33928174824 and security lanes are the only acceptance evidence for this generation. Do not merge unless exact-head required checks become terminal GREEN and counted approval/thread gates are satisfied.

Commit-Message-Assisted-by: Claude (via Claude Code)
Signed-off-by: Seongho Bae <me@seonghobae.me>
@seonghobae

Copy link
Copy Markdown
Contributor Author

Exact-head repair at af83c40dd2990a03064a92ca75430a9cc400f098: the previous head failed its own release-record contract because CHANGELOG omitted the tested rename/copy similarity-to-blob-identity invariant. The repair adds that single missing record without changing classifier behavior or workflows. Verification on this exact tree: focused classifier suite 53 passed plus 38 subtests; full Python repository suite 205 passed plus 581 subtests; cargo fmt --all --check, workspace all-target tests, strict Clippy, rustdoc, and nightly branch coverage all pass with 100% functions/lines/regions/branches. git diff --check is clean. Fresh hosted checks/reviews for this moved head remain required; predecessor results do not transfer.

@seonghobae seonghobae left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exact-head review on af83c40dd2990a03064a92ca75430a9cc400f098: the concurrent CHANGELOG follow-up is a valid explanation of the rename/copy similarity-to-blob-identity invariant and is adopted, not treated as a race. The prior ab6d5d1a... CI/Security/Semgrep/CodeQL runs are now terminal cancelled after branch movement and are historical only. Fresh current-head CI 33931806137, Security 33931806226, Semgrep 33931806165, and CodeQL 33931806139 are queued. No current-generation GREEN is claimed; keep the workflow-free classifier prerequisite separate from authorized #279/#282 workflow activation.

@seonghobae

Copy link
Copy Markdown
Contributor Author

Fresh exact-head review on af83c40dd2990a03064a92ca75430a9cc400f098 found no actionable defect in the mode-aware raw-diff classifier foundation. A real git diff --raw -z --no-abbrev main...HEAD feeds the executable boundary and correctly classifies this code-bearing branch as documentation_only=false / rust_required=true; 53 focused parser, object-identity, similarity, path, and prose-boundary regressions plus all 205 repository Python tests pass. The worktree is clean and there are no review threads. Hosted checks remain queued and the required independent review is unmet, so this is verification evidence only, not approval or merge authorization.

@seonghobae

Copy link
Copy Markdown
Contributor Author

Exact-head focused review on af83c40dd2990a03064a92ca75430a9cc400f098: rechecked the NUL-framed git diff --raw -z --no-abbrev parser, canonical repository-path boundary, status/path cardinality, file-mode semantics, SHA-1/SHA-256 object identity checks, rename/copy similarity binding, instruction-file exclusion, and fail-closed legacy path-only route. I found no new source-backed bypass in this review pass; current tests cover malformed/truncated records, traversal/non-canonical paths, executable/symlink/gitlink/type changes, object-id mismatches, and rename/copy preimages.

This is a COMMENTED review record, not approval or merge authorization. Hosted exact-head checks and an eligible independent approval remain required.

@seonghobae

seonghobae commented Sep 5, 2026

Copy link
Copy Markdown
Contributor Author

Exact-head handoff RCA at af83c40 against protected main 87c4daa (2026-09-05T08:34Z): repository-native CI 33931806137 is successful. Semgrep 33931806165 ran its scan job successfully. Security Scan 33931806226 ran Scorecard and Trivy successfully; gitleaks, OSV and dependency-review were skipped by scope, so those skips are not independent executed scan evidence.

CodeQL wrapper 33931806139 is failure, but current job logs identify an authenticated dispatch handoff rather than a terminal scan verdict: JavaScript/TypeScript 101244865461, Actions 101244865470 and Python 101244865526 each record DISPATCH_OUTCOME=success, VERDICT_STATE=pending, and intentional exit 1 so the dispatcher can replay the exact job after its terminal verdict. The gh run convenience route cannot resolve the cross-repository workflow (404); direct REST job logs provide the authoritative diagnosis.

The canonical .github repository has three matching target-head dispatches: 33954721186, 33955024697 and 33955164029, all queued. Their display titles identify OriginWeave#287af83c40..., while their immutable run.head_sha values belong to the central workflow repository (71dd84d... / 27d7331...), not the OriginWeave commit. This distinction is preserved; no PR association or title is substituted for source checkout validation. Do not rerun this wrapper or create a duplicate dispatch while these exact handles remain non-terminal. Revisit their authenticated terminal verdict and original-job replay after material state change.

Follow-up at 2026-09-05T09:07Z: Strix run 33931804438 / scan job 101243872506 executed Run Strix (quick) and uploaded reports successfully; skipped ancillary status publication is not the scan. Noema run 33931804409 / job 101245089068 failed at 09:00 UTC with gateway HTTP 502 after 2739.2 seconds and one caller attempt. The gateway owns provider repair/failover; no caller timeout, model, provider or workflow was changed. Current-head inventory showed no other Noema run or pending duplicate. With the exact PR head unchanged, the first safe corrective action was a failed-job rerun of that same run; REST now reports queued, not executed review, provider recovery, a verdict or eligible approval. Do not duplicate it while pending. The three central CodeQL dispatches remain queued and were not replayed manually. OpenCode remains queued. Live rules still require counted non-author approval and only the author is a listed collaborator.

seonghobae added a commit that referenced this pull request Sep 5, 2026
Bind #250 current-parent and #144 read-only review evidence, and distinguish #287's queued central CodeQL handoff from a terminal scan verdict. Preserve live target and central workflow revision boundaries.

Commit-Message-Assisted-by: Codex (via Codex)
Signed-off-by: Seongho Bae <me@seonghobae.me>

@seonghobae seonghobae left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exact-current verification update for af83c40dd2990a03064a92ca75430a9cc400f098: CI 33931806137, Security Scan 33931806226, and SAST Semgrep 33931806165 are terminal success. CodeQL PR 33931806139 is terminal failure, but all three language compatibility shards successfully reach the central dispatch step and fail only while waiting for the authenticated terminal verdict. Python job 101244865526 validates the unchanged live PR/head, obtains OIDC and the repository-scoped App token, successfully dispatches codeql-scan, then exits fail-closed with VERDICT_STATE=pending. Fresh exact-head commit statuses contain no codeql-dispatch/* verdict. This independently reproduces the same central stall as #272; I appended both canaries to ContextualWisdomLab/.github#712 comment 5551060102. No classifier/source workaround or gate weakening is warranted. Keep #282/#283 owner order unchanged until this exact CodeQL evidence becomes terminal.

@seonghobae

Copy link
Copy Markdown
Contributor Author

Exact head af83c40dd2990a03064a92ca75430a9cc400f098 read-only review found one reproducible local/CI-entry defect: the classifier tests import scripts.ci..., but the default pytest command fails during collection with ModuleNotFoundError: No module named scripts; adding shell-only PYTHONPATH=. makes the same 53 tests / 38 subtests pass. Successor #300 (c3538fcca729f4c21ae20541119250ade1b67cd4) adds only root pytest.ini import-path configuration plus CHANGELOG evidence. Its default focused command passes 53 / 38, and the full suite passes 205 / 581. No workflow or governance mutation is included; #287 stays the owner of classifier behavior.

@seonghobae

Copy link
Copy Markdown
Contributor Author

Current-head review found a reproducible repository-root test-collection defect on af83c40dd2990a03064a92ca75430a9cc400f098:

uv run pytest tests/test_ci_change_scope.py tests/test_ci_change_scope_modes.py tests/test_ci_change_scope_object_ids.py tests/test_ci_change_scope_path_identity.py tests/test_ci_change_scope_prose_boundary.py tests/test_ci_change_scope_similarity_scores.py -q
# ModuleNotFoundError: No module named scripts (six collection errors)

The stacked successor #300 (6bc57663b3bf36dd7e4a2750e2cd5d615f450cb2) contains the minimal durable repair: pytest.ini declares pythonpath = .. The identical command there is GREEN: 53 passed / 38 subtests; git diff --check is clean. Keep this foundation PR and merge it only after the successor is carried through required exact-head review and checks; the hosted green result on #287 does not replace the repository-root regression proof. This is a current-evidence comment, not an approval or a merge authorization.

Preserve the repository-owned pytest import-path repair on the #287 classifier foundation after exact-head CI success.

@seonghobae seonghobae left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exact-current review for 3975daf48e01a5e9d1cf9fb104a3be1aa03b0402: the workflow-free classifier foundation has terminal native CI 34329801487, Security Scan 34329801483, and SAST Semgrep 34329801598 success, and the current CodeRabbit pass reports no actionable source finding. CodeQL PR 34329801477 remains terminal failure only at the central Release runner or enforce current-head CodeQL verdict stage after successful verdict reads in python/actions/javascript-typescript and a successful later dispatch job. This is fail-closed owner-path evidence, not a classifier defect or approval. Exact specimen is now handed to ContextualWisdomLab/.github#712 as comment 5599230322. Keep #287 unmerged until an authenticated current-head CodeQL verdict and the independent review/ruleset requirements are satisfied; do not add a no-op commit, duplicate central scanning, or weaken the gate.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant