Skip to content

Fix SCOPE_ARGS quoting bug that silently runs 0 tests - #8

Open
lihsaa591 wants to merge 1 commit into
mainfrom
fix/scope-args-quoting-bug
Open

lihsaa591 wants to merge 1 commit into
mainfrom
fix/scope-args-quoting-bug

Conversation

@lihsaa591

Copy link
Copy Markdown

Background

Found on everest-forms PR themegrill/everest-forms#1669: a @claudegrill suite run failed with exit 2 (harness failure), ran_nothing: true, reason: "the runner executed 0 tests".

Root cause

In .github/workflows/suite.yml's "Run the suite (fresh tier)" step:

SCOPE_ARGS="--spec '${{ steps.scope.outputs.specs }}'"
...
node run-suite.mjs ... $SCOPE_ARGS ...

The single quotes are baked into the string as literal characters. When $SCOPE_ARGS is later expanded unquoted, bash word-splits it on whitespace but does not re-parse the embedded ' characters as shell quoting syntax — that only happens with eval or when the shell itself writes the quotes around an expansion. So run-suite.mjs receives a --spec value with literal quote characters glued onto the path, e.g. 'tests/e2e/specs/.../foo.spec.ts' as one malformed token (or split further depending on the value), matches no file, and Playwright runs 0 tests.

This hits any PR whose latest commit only adds/changes spec files, since that's exactly the scope: specs / --spec branch.

Fix

Switch SCOPE_ARGS to a bash array and expand it as "${SCOPE_ARGS[@]}", which is immune to this class of bug regardless of what the area/spec value contains (spaces, quotes, etc.).

Verification

  • bash -n on the extracted script block: clean.
  • Ran the fixed logic standalone with the exact failing case from PR #1669 (--spec set to tests/e2e/specs/submission/field-value-smart-tag-not-reresolved.spec.ts): produces exactly [--spec] [tests/e2e/specs/submission/field-value-smart-tag-not-reresolved.spec.ts] with no stray quote characters.

SCOPE_ARGS="--area '$X'" (or --spec) baked literal single-quote
characters into the string, then expanded it unquoted at the call
site. Bash word-splits an unquoted expansion but never re-parses
embedded quote characters as shell syntax, so run-suite.mjs received
an --area/--spec value with stray quotes glued onto it, matched
nothing, and silently ran 0 tests — reported as a harness failure
(exit 2), not a red suite.

Found on everest-forms PR #1669: a commit that only added a new spec
file put the workflow on the `--spec` branch, which exposed the bug.

Switches SCOPE_ARGS to a bash array and expands it as "${SCOPE_ARGS[@]}",
which is immune to this class of bug regardless of what the area/spec
value contains.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@lihsaa591
lihsaa591 force-pushed the fix/scope-args-quoting-bug branch from fd10a99 to 7f1afcb Compare September 30, 2026 04:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant