Skip to content

Skip the unit and E2E suites for screenshot-only changes - #785

Open
ryanwelcher wants to merge 1 commit into
trunkfrom
update/784-skip-suites-for-screenshot-changes
Open

ryanwelcher wants to merge 1 commit into
trunkfrom
update/784-skip-suites-for-screenshot-changes

Conversation

@ryanwelcher

Copy link
Copy Markdown
Collaborator

Why

A docs PR that updates screenshots runs the full unit and E2E suites on macOS and Windows, though nothing in them can be affected. #777 is an example: Markdown, PNGs under docs/public/screenshots/ and two files in scripts/screenshots/.

What changes

  • The changes job in unit-tests.yml and e2e.yml now treats files under docs/public/ and scripts/screenshots/ as documentation, alongside *.md. A PR that touches only those skips the suites, and the *-skipped jobs report the required checks as passed, as they already do for Markdown.
  • docs/.vitepress/ still counts as code. tests/unit/download-button-platform.test.cjs loads a file from it.
  • The skipped jobs now say "Only documentation changed" instead of "Only Markdown changed".

How to test this

Platforms: any. Nothing in the app changes. This PR edits the workflows themselves, so its own suites run in full.

Run the new filter against #777's file list from the repository root, in bash:

gh api --paginate repos/WordPress/contributor-toolkit/pulls/777/files \
  --jq '.[] | .filename, (.previous_filename // empty)' \
  | grep -v -E '\.md$|^docs/public/|^scripts/screenshots/' || echo "no code files: suites skipped"

Expected: it prints no code files: suites skipped.

What must not have happened: a change that a suite does depend on must still run it. Replace the gh api line with printf '%s\n' docs/public/a.png docs/.vitepress/config.mjs or printf '%s\n' docs/a.md src/main.js. Each should print the code path, which means the suites run.

Risks and limitations

  • node --test finds test files anywhere in the repository, not only in tests/unit/. A test file added under scripts/screenshots/ would be skipped on its PR and first run on trunk. No such file exists today, so I left it alone rather than add a guard for it (see the review outcome).
  • Review: 1 [fix here] finding, low severity, deferred.

Related

Fixes #784


Review outcome (required, see AGENTS.md)

1 [fix here] · 0 [follow-up]. Deferred.

  • Tests, 🔵 low, deferred. unit-tests.yml:62 / e2e.yml:69: bare node --test discovers *.test.*, *-test.*, *_test.*, test-*.* and test/** repo-wide, so a test file under scripts/screenshots/ would not run on its PR. No such file exists, and AGENTS.md asks not to add branches for hypothetical cases. If one is ever added there, the filter needs a carve-out for test-named files.
  • Checked clean: nothing in tests/unit, tests/e2e, src/ or eslint-rules/ reads docs/public/ or scripts/screenshots/ (only comments mention them). build.files packages only package.json, src/ and local-playground-web/. Renamed files count as code if either name is outside the skipped set. lint.yml has no path filter and does not ignore scripts/. No other workflow, .buildkite file or doc describes the Markdown-only rule.
  • Deterministic layer: npm run lint passed; npm test passed (2312 pass, 9 skipped, 0 fail).
  • Review: completed. Separate agent context (Claude Code /self-review), on the uncommitted working tree at head 78e9b3da / base 78e9b3da (trunk). Committed as 2b0de984 with no changes beyond the reviewed diff.
  • Since review: none.

🤖 Generated with Claude Code

The `changes` job in unit-tests.yml and e2e.yml now treats files under
docs/public/ and scripts/screenshots/ as documentation, alongside
Markdown. No test or packaged build loads them; docs/.vitepress/ still
counts as code.

Fixes #784
@coderabbitai

coderabbitai Bot commented Nov 17, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Skip the unit and E2E suites for screenshot-only changes

1 participant