Skip to content

test: library-layer guard times out locally spawning eslint per assertion #24

Description

@amondnet

Symptom

scripts/lib/__tests__/library-layer-guard.test.ts fails locally on some machines:

(fail) Library Layer ESLint guard (T008, AC-6, NFR-1) > flags node:fs import in scripts/lib/*.ts
  ^ this test timed out after 5000ms.
expect(received).toMatch(expected)
Expected substring or pattern: /no-restricted-imports/
Received: ""

5 of the file's 6 tests fail this way on a clean main (verified with git stash); the whole file takes ~30s. CI is green — the failures do not reproduce on the GitHub-hosted runner, and Verify (typecheck / lint / test) passes on main and on PRs.

Cause

Each test spawns a real eslint process against a --stdin-filename:

const proc = Bun.spawn([ESLINT, '--stdin', '--stdin-filename', filename, '--no-warn-ignored'], )

ESLint flat-config startup plus @pleaseai/eslint-config resolution costs roughly 5s per spawn on a slower machine, against Bun's default 5000ms per-test timeout. The empty stdout in the assertion is the process being killed mid-startup, not a rule that failed to fire.

Why it is worth fixing

The guard tests a real invariant — the Library Layer ESLint rule — and that is the right way to test it. But a suite that is red on a developer's machine and green in CI trains people to ignore bun test output, which is how a real regression gets waved through.

Options

  • Raise the timeout for this file (test(name, fn, timeout) or a file-level setting) to something above a cold ESLint start. Simplest, keeps the coverage, costs nothing but wall-clock honesty.
  • Spawn ESLint once and lint all the fixture cases in a single invocation, amortising startup across the six assertions.
  • Reuse one ESLint instance through the Node API instead of the CLI, which removes process startup entirely but couples the test to ESLint's programmatic surface.

The second or third also cuts ~30s off every full bun test run.

Note

Observed while working on #21; unrelated to that change and deliberately left untouched there.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    type:testTesting improvements or additions

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions