-
Notifications
You must be signed in to change notification settings - Fork 39
ci(triage): match test as a name segment, not just a whole component #277
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
beardthelion
wants to merge
3
commits into
main
Choose a base branch
from
fix/triage-detect-namespaced-test-attrs
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+288
−3
Open
Changes from all commits
Commits
Show all changes
3 commits
Select commit
Hold shift + click to select a range
b5db6db
ci(triage): match test as a name segment, not just a whole component
beardthelion 1105320
fix(ci): allow Rust token separators before test attribute delimiters
beardthelion 2259754
ci(triage): scan attribute closers token-aware, bound to adjoined add…
kevincodex1 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,172 @@ | ||
| #!/usr/bin/env node | ||
| // Case matrix for the inline-test detector embedded in | ||
| // .github/workflows/pr-triage.yml. The workflow runs on pull_request_target | ||
| // and deliberately never checks out PR code, so the detector cannot be tested | ||
| // where it runs; this script extracts the fenced TRIAGE_DETECTOR block from | ||
| // the committed workflow body and exercises it here, where pr-checks.yml DOES | ||
| // check out the proposed workflow. If the fence markers move or the block | ||
| // stops being self-contained, this script fails loudly rather than testing a | ||
| // stale copy. | ||
| // | ||
| // The matrix encodes the review contract for the detector | ||
| // (Gitlawb/node#277): legal Rust separators between the attribute path and | ||
| // its ]/( delimiter must be accepted (line comments, nested block comments, | ||
| // splits onto immediately following added lines), while unrelated patch | ||
| // records — delimiter-looking lines before the path, later in the hunk, or in | ||
| // another hunk — must never complete a path they do not adjoin. False | ||
| // negatives here SUPPRESS the needs-tests label silently, so every uncertain | ||
| // path in the detector is required to answer "no inline test". | ||
|
|
||
| import { readFileSync } from "node:fs"; | ||
| import { dirname, join } from "node:path"; | ||
| import { fileURLToPath } from "node:url"; | ||
|
|
||
| const repoRoot = join(dirname(fileURLToPath(import.meta.url)), ".."); | ||
| const workflow = readFileSync( | ||
| join(repoRoot, ".github/workflows/pr-triage.yml"), | ||
| "utf8" | ||
| ); | ||
|
|
||
| const BEGIN = "// TRIAGE_DETECTOR_BEGIN"; | ||
| const END = "// TRIAGE_DETECTOR_END"; | ||
| const begin = workflow.indexOf(BEGIN); | ||
| const end = workflow.indexOf(END); | ||
| if (begin === -1 || end === -1 || end <= begin) { | ||
| console.error("FAIL: TRIAGE_DETECTOR fence not found in pr-triage.yml"); | ||
| process.exit(1); | ||
| } | ||
| const block = workflow.slice(begin + BEGIN.length, end); | ||
|
|
||
| let patchAddsInlineTest; | ||
| try { | ||
| const factory = new Function(`${block}\nreturn patchAddsInlineTest;`); | ||
| patchAddsInlineTest = factory(); | ||
| } catch (err) { | ||
| console.error( | ||
| "FAIL: fenced detector block is not self-contained JavaScript:", | ||
| err.message | ||
| ); | ||
| process.exit(1); | ||
| } | ||
|
|
||
| // Each patch is the `patch` field GitHub's listFiles API returns: hunk | ||
| // headers plus +/-/space-prefixed lines, no ---/+++ file headers. | ||
| const cases = [ | ||
| // ── Accepted spellings ──────────────────────────────────────────────── | ||
| ["bare same-line", "@@ -1,0 +1,2 @@\n+#[test]\n+fn a() {}", true], | ||
| ["cfg(test)", "@@ -1,0 +1,1 @@\n+#[cfg(test)]", true], | ||
| ["indented with inner space", "@@ -1,0 +1,1 @@\n+ #[ test ]", true], | ||
| [ | ||
| "namespaced with args", | ||
| '@@ -1,0 +1,1 @@\n+#[tokio::test(flavor = "multi_thread")]', | ||
| true, | ||
| ], | ||
| ["test_case harness", "@@ -1,0 +1,1 @@\n+#[test_case(1)]", true], | ||
| ["wasm_bindgen_test harness", "@@ -1,0 +1,1 @@\n+#[wasm_bindgen_test]", true], | ||
| ["raw identifier", "@@ -1,0 +1,1 @@\n+#[r#test]", true], | ||
| [ | ||
| "line comment then closer on next added line", | ||
| "@@ -1,0 +1,2 @@\n+#[test // rationale\n+]", | ||
| true, | ||
| ], | ||
| [ | ||
| "nested block comment, same line", | ||
| "@@ -1,0 +1,1 @@\n+#[test /* outer /* inner */ outer */]", | ||
| true, | ||
| ], | ||
| [ | ||
| "block comment spanning added lines", | ||
| "@@ -1,0 +1,3 @@\n+#[test /* why\n+ still why */ ]\n+fn a() {}", | ||
| true, | ||
| ], | ||
| [ | ||
| "path-only line, ( on the immediately following added line", | ||
| "@@ -1,0 +1,2 @@\n+#[test_case\n+(1)]", | ||
| true, | ||
| ], | ||
| [ | ||
| "whitespace-only continuation before closer", | ||
| "@@ -1,0 +1,3 @@\n+#[test\n+\t\n+]", | ||
| true, | ||
| ], | ||
| // ── Rejected spellings and adversarial shapes ───────────────────────── | ||
| ["rstest stays excluded", "@@ -1,0 +1,1 @@\n+#[rstest]", false], | ||
| ["substring #[testable]", "@@ -1,0 +1,1 @@\n+#[testable]", false], | ||
| ["substring #[contest]", "@@ -1,0 +1,1 @@\n+#[contest]", false], | ||
| [ | ||
| "delimiter-looking line BEFORE the path", | ||
| "@@ -1,0 +1,2 @@\n+(\n+#[test_case", | ||
| false, | ||
| ], | ||
| [ | ||
| "raw-string fixture path + unrelated ( later in the same hunk", | ||
| '@@ -1,0 +1,5 @@\n+let s = r#"\n+#[test_case\n+not a separator token\n+"#;\n+let t = (1);', | ||
| false, | ||
| ], | ||
| [ | ||
| "path at end of one hunk, closer in another hunk", | ||
| "@@ -1,0 +1,1 @@\n+#[test_case\n@@ -10,0 +11,1 @@\n+(1)]", | ||
| false, | ||
| ], | ||
| [ | ||
| "closer only on a context line", | ||
| "@@ -1,1 +1,1 @@\n+#[test_case\n (1)]", | ||
| false, | ||
| ], | ||
| [ | ||
| "closer only on a removed line", | ||
| "@@ -1,1 +1,1 @@\n+#[test_case\n-(1)]", | ||
| false, | ||
| ], | ||
| [ | ||
| "unfinished block comment never closes", | ||
| "@@ -1,0 +1,2 @@\n+#[test /*\n+ still open", | ||
| false, | ||
| ], | ||
| [ | ||
| "continuation bound exceeded stays loud", | ||
| "@@ -1,0 +1,40 @@\n+#[test /*\n" + "+ filler\n".repeat(30) + "+ */ ]", | ||
| false, | ||
| ], | ||
| ]; | ||
|
|
||
| let failures = 0; | ||
| for (const [name, patch, expected] of cases) { | ||
| const got = patchAddsInlineTest(patch); | ||
| if (got !== expected) { | ||
| failures += 1; | ||
| console.error(`FAIL: ${name}: expected ${expected}, got ${got}`); | ||
| } | ||
| } | ||
|
|
||
| // Runtime probe: the detector walks fork-controlled input on | ||
| // pull_request_target, so a pathological head must not stall the job. The | ||
| // long `_a` run is the historical exponential-backtracking shape for the | ||
| // attribute-path regex; the comment run exercises the scanner loop. | ||
| const probes = [ | ||
| ["long _a run", "@@ -1,0 +1,1 @@\n+#[test" + "_a".repeat(30000), false], | ||
| [ | ||
| "long unclosed comment line", | ||
| "@@ -1,0 +1,1 @@\n+#[test /*" + " *".repeat(30000), | ||
| false, | ||
| ], | ||
| ]; | ||
| for (const [name, patch, expected] of probes) { | ||
| const t0 = process.hrtime.bigint(); | ||
| const got = patchAddsInlineTest(patch); | ||
| const ms = Number(process.hrtime.bigint() - t0) / 1e6; | ||
| if (got !== expected) { | ||
| failures += 1; | ||
| console.error(`FAIL: probe ${name}: expected ${expected}, got ${got}`); | ||
| } | ||
| if (ms > 1000) { | ||
| failures += 1; | ||
| console.error(`FAIL: probe ${name}: took ${ms.toFixed(0)}ms (>1000ms)`); | ||
| } | ||
| } | ||
|
|
||
| if (failures) { | ||
| console.error(`${failures} failure(s) across ${cases.length + probes.length} cases`); | ||
| process.exit(1); | ||
| } | ||
| console.log(`ok: ${cases.length + probes.length} detector cases passed`); | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Correct the false-positive rationale.
A false positive from
patchAddsInlineTestsetstouchedTeststo true and suppressesneeds-tests. A false negative applies the label instead. Update this comment so future matrix changes protect the correct silent-failure direction.🤖 Prompt for AI Agents