Skip to content

Fix CLI test suite hanging failures on platforms without O_NOFOLLOW/O_NONBLOCK - #31

Open
Kaushik2210 wants to merge 2 commits into
cloudflare:mainfrom
Kaushik2210:fix/windows-cli-test-skip-guard
Open

Kaushik2210 wants to merge 2 commits into
cloudflare:mainfrom
Kaushik2210:fix/windows-cli-test-skip-guard

Conversation

@Kaushik2210

Copy link
Copy Markdown

Summary

validate-findings.cjs correctly refuses to process any input when the OS doesn't provide O_NOFOLLOW/O_NONBLOCK (e.g. Windows, where these fs.constants are undefined), rather than opening the file unsafely — that's intentional, documented safe-open behavior.

validate-coverage-ledger.test.cjs already guards its CLI-invoking tests behind a HAS_SAFE_INPUT_OPEN check for exactly this reason. validate-findings.test.cjs defines the same HAS_SAFE_INPUT_OPEN constant and applies it to some of its CLI tests, but 7 CLI tests were missing the guard, so they failed outright (rather than skipping) on any platform lacking those flags:

# tests 34
# pass 22
# fail 7

Fix

Apply the existing { skip: !HAS_SAFE_INPUT_OPEN } guard — already used elsewhere in this same file, and in validate-coverage-ledger.test.cjs — to the 7 tests that were missing it. No behavior change to the validator itself; this only brings the test suite's skip conditions in line with the validator's own documented platform behavior.

Test plan

  • node skills/security-audit/validate-findings.test.cjs — 34 tests, 0 failures (was 7 failures before this change)
  • node skills/security-audit/validate-coverage-ledger.test.cjs — 31 tests, 0 failures (no regression)

🤖 Generated with Claude Code

validate-findings.cjs correctly refuses to process any input when the
OS doesn't provide O_NOFOLLOW/O_NONBLOCK (e.g. Windows, where these
fs.constants are undefined) rather than opening the file unsafely.
validate-coverage-ledger.test.cjs already guards its CLI tests behind
a HAS_SAFE_INPUT_OPEN check for this reason, but validate-findings.test.cjs
only applied the guard to some of its CLI tests, so the remaining ones
failed outright on platforms without those flags instead of skipping.

Apply the same HAS_SAFE_INPUT_OPEN skip guard, already defined in this
file, to the 7 CLI tests that were missing it.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@Dharundp6

Copy link
Copy Markdown

Reproduced this independently on Windows before finding your PR — same diagnosis, same counts (22 pass / 7 fail), and the validate-coverage-ledger.test.cjs comparison is the right framing. Agreed the validator behaviour is intentional and shouldn't change.

One gap this leaves, offered in case it's useful: once the guards are in place, the fail-closed path has no test anywhere. Right now it's only "covered" by those 7 tests going red on a platform without the flags, which this PR correctly silences. The inverted guard covers it on exactly the platforms that trigger it:

test("CLI fails closed when the OS cannot open input safely", { skip: HAS_SAFE_INPUT_OPEN }, () => {
  const result = runCli(JSON.stringify(producerShapedFindings()));
  assert.equal(result.status, 1, cliOutput(result));
  assert.match(result.stderr, /OS no-follow and nonblocking input protection is unavailable/);
  assertNoInjectedControlBytes(result.stderr);
});

Passing a valid document is the point — it makes the absent OS protection the only possible reason to reject, so the test can't pass for an unrelated reason. On Windows this gives 23 pass / 0 fail / 12 skip; on Linux and macOS it skips and nothing else changes.

Also worth a look while you're in this file: does not reflect controls from a failed CLI input path (line 429) is unguarded but passes on Windows for the wrong reason — the safe-open check fires before the missing path is ever opened, and its three assertions happen to hold anyway. It never exercises the missing-path branch there.

Happy to open a follow-up PR for either of these after this one lands, or you're welcome to fold them in here.

…wrong reason

Add an inverted-guard test that runs on exactly the platforms lacking
O_NOFOLLOW/O_NONBLOCK and asserts the CLI rejects a valid document with
the safe-open error, so the fail-closed behavior is covered rather than
only observed as failing tests.

Guard "does not reflect controls from a failed CLI input path" behind
HAS_SAFE_INPUT_OPEN: without the OS flags the safe-open check fires
before the missing path is opened, so its assertions held without ever
exercising the missing-path branch.

Suggested by Dharundp6 in review of cloudflare#31.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants