Skip to content

Vitest verify: leftover review items (name-filter spellings, journey classification) #172

Open

Description

@inth3shadows

What happens

These are the low-severity findings left open after three review rounds of #171 (verify --runner vitest). In every case the result is a refusal or an incomplete result, never a false pass or a false failure. They are grouped here because most share one cause: how a journey that isn't a pass gets classified.

Checked on main at 10ea764, Vitest 4.1.9.

Checklist

  • Some spellings of the name filter reach Vitest instead of being refused up front.
    • The inputs are -t5, -t.x, -t^@tg:J1, -ut1, ---t x and --no-t (verified: the guard returns None for each).
    • Vitest's CLI parser gives every character of a single-dash cluster its own option, so t is set anyway. Vitest then crashes with no report, and the run is refused (exit 2) without saying why. A --no-t is inferred to cancel the journey filter instead.
    • Expected: the same up-front refusal that -t x gets.
  • Classify journeys once, in run, not in the renderer.
    • A journey whose tagged tests ran but never reached its entry symbols is reported under uncovered in --json (verified). uncovered means "nothing declared"; only the text output tells the two apart.
    • pytest has the same gap: a skipped covers(...) test, or one whose trace misses the entry, prints "nothing declared covers(...)" (inferred).
    • Fixing it in validate covers both runners. It also replaces the three passes that sort journeys into uncovered and incomplete.
  • "(listed below)" can name a journey that isn't listed. The unvalidated rows are capped at 8.
  • Tests:
    • The tag-pattern tests still put the file path in front of the name, but Vitest's -t never sees the path.
    • No test covers a tag at the very start of a title. Dropping the pattern's ^ branch leaves every test green.
    • The real-Vitest path-token test counts declarations, not which tests Vitest itself selected.
    • The mixed render (one journey unvalidated, another untagged) is not pinned.
  • A verdict with no full_name raises KeyError (exit 1, the "tests failed" code). That can't happen today, because the report parser always sets full_name. It should be a refusal (exit 2).
  • Cleanups:
    • On the Vitest path the plugin directory is looked up twice.
    • The runner's exit is kept under two keys (pytest_exit and runner_exit) instead of one key with a per-runner set of accepted codes.
    • Tag patterns are built again on every call.
  • Two titles that differ only in /, :: or . share one join key and refuse the run (exit 2). This predates feat: verify runs only the selected journeys' Vitest tests #171.

Related: #122 (Vitest and Playwright support), #171 (where these were found), #153 (pytest verify: a collection error also lands in "no journey-level test"; the same classification step, but a different cause)

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions