ci: the heavy lanes decide their component scope; drop the scope jobs by cixzhang · Pull Request #8919 · meta/astryx

cixzhang · 2026-10-30T01:52:00Z

Who this helps

Everyone opening a pull request: the last two scope jobs (check-scope, check-components) disappear, so no job waits on a classifier job before starting.

What changes

The heavy lanes decide their own component scope.
- The accessibility, visual and RTL jobs run the test-scope step with component-audit: true. It also runs the base's trusted component-audit-scope.cjs and lists the changed files.
- Each audit still covers only the affected components on a PR (spec:AST-030 FR13), and skips its work, downloads included, when nothing in its scope changed.
- Release-check, which runs outside a PR, stays full scope.
- A release-branch PR applies no audit.
The RTL join accepts shards that decided their scope and did nothing.
The scope jobs are removed. An explicit release check keeps its authority in a small release-authority job, and release-bump runs on a release-branch PR by its base ref.

Required contexts are unchanged (FR7).

The heavy lanes' job-level conditions here are today's main conditions, minus check-components. When ci: visual, accessibility and RTL run on main, not on pull request updates #8877 lands, merging main keeps its ci:heavy-lanes opt-in unchanged.

Part 4 of 4 of the CI scope consolidation. Stacked on #8918.

Line count

+596 / −370, net +226.

The added lines are the per-job component-audit scope, its routing tests, and the action's audit cases.
Across the 4 parts: +1,719 / −1,582, net +137. The follow-up that drops off-PR checks removes more than 1,000 lines.

Verification

vitest run --project node .github/scripts internal/shadcn-registry internal/lab-readiness: all pass.
Routing tests cover:
- a labelled out-of-scope change, which leaves the RTL shards as no-ops and the join green;
- each heavy lane's analysis download, which is gated;
- a release bump, which gets no component audit.

The accessibility, visual and RTL jobs run the test-scope step with `component-audit: true`: it also runs the base's trusted component-audit-scope.cjs and lists the changed files, so each audit still covers only the affected components on a pull request (spec:AST-030 FR13) and skips its work, downloads included, when nothing in its scope changed. Release-check (outside a pull request) stays full scope, and a release-branch pull request applies no audit. With every consumer moved, check-scope and check-components go away. An explicit release check keeps its authority in a small release-authority job, release-bump runs on a release-branch pull request by its base ref, and the RTL join accepts shards that decided their scope and did nothing.

vercel · 2026-10-30T01:52:06Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
astryx	Ready	Preview	Oct 30, 2026 6:57am UTC

astracat-bot · 2026-10-30T01:52:33Z

AI review status for this pull request.

Review status	Updated
🟢 Reviewed (for maintainers only)	Oct 30, 2026, 7:12 AM UTC

#8888 is superseded by four atomic PRs; this change now sits on the last of them. The relevance-registry edit drops out: that registry is not on main.

#8888 is superseded by four atomic PRs; this change now sits on the last of them. The relevance-registry edits drop out: that registry is not on main.

…csite-test #8888 was split into #8916-#8919 on main. This carries the one-job docsite-test change onto #8919 as a 3-way merge against its v8: the docsite jobs are identical there, so the delta applies unchanged.

…t's base #8888 was split into four pull requests; this one now stacks on #8919. The tree is #8919 plus this pull request's own changes, re-applied to its job layout: the per-author heavy caps (FR16), the draft and outside-contributor gate (FR17), CONTRIBUTING and the tests.

astracat-bot

Changes requested: per-job component scope is a sound replacement for the two scope jobs, but a few steps still need gating and tests.

Gate pr-visual's Setup Node and pnpm and Install Playwright steps and each visual-scout leg's setup on the visual scope, so spec-, docs-, tooling-only and release-bump PRs stop running a full pnpm install and a Chromium download, and extend the out-of-scope routing test to those steps.
Add a test that fails when the scope action's failed-classifier fallback stops running every audit (AST-030 FR9).
Accept a skipped/success shard mix as an out-of-scope no-op in pr-rtl, pass the changed-file list to the visual scope step through a file instead of an environment variable, and keep the lab-readiness CI check failing when it cannot find the gate.

[Automated review]

astracat-bot · 2026-10-30T02:57:56Z

+          printf '%s\n' "$FILES" \
+            | node .github/scripts/visual-scope.mjs --github-output "$GITHUB_OUTPUT" "${VISUAL_ARGS[@]}"
+
      - name: Setup Node and pnpm


Setup Node and pnpm here and Install Playwright below (line 2205) have no scope condition, so every PR outside stable visual scope now runs pnpm install --frozen-lockfile and installs Chromium in this job, which the base skipped. Gating both on github.event_name != 'pull_request' || steps.visual-scope.outputs.has_stable_visual == 'true' keeps capture and release-check unchanged. Each visual-scout leg's setup (line 2057) needs the scout's own condition too.

astracat-bot · 2026-10-30T02:57:56Z

+      - name: Decide stable visual scope
+        id: visual-scope
+        env:
+          FILES: ${{ steps.scope.outputs.changed_files }}


Passing the file list as an environment variable caps it at Linux's 128 KiB per-string limit: around 2,300 paths (renames count twice) make this step fail to start with Argument list too long. Writing the list to a file under RUNNER_TEMP from the action avoids that. The same applies at line 2040.

astracat-bot · 2026-10-30T02:57:56Z

+                );
+              } catch (error) {
+                core.warning(`Component audit scope could not be decided (${error.message}); every audit runs in full.`);
+                componentScope = auditAll;


No test reaches this fallback: replacing auditAll with auditNone here leaves the scope-action, audit-scope and routing suites green, because the missing-classifier case fails earlier, in the fetch. A case whose base serves a classifier that throws would cover this fail-closed edge, as AST-030 FR9 asks.

astracat-bot · 2026-10-30T02:57:56Z

+    shardResult === 'success' &&
+    evidence.scopes.length === 0 &&
+    evidence.reports.length === 0;
+  return shardResult === 'skipped' || noOpShards


When build-storybook fails on a PR outside RTL scope, pr-rtl-shard is skipped while pr-rtl-merge (always()) succeeds as a no-op, so this receives skipped/success and reports that shards unexpectedly ran. Treating any mix of skipped and success with no evidence as the no-op would keep the base's pass.

astracat-bot · 2026-10-30T02:57:56Z

+  if (filtered) {
+    return filtered[1].split('|').map(name => `packages/${name}/src/`);
+  }
+  return registryRoots;


With no recognisable change detection in ci.yml, this now returns the registry roots, so the lab-coverage check can no longer fail when it cannot find the gate. Recognising the registry projection in .github/actions/test-scope/action.yml would keep an unrecognised gate failing.

- Drops the test join's check-scope gate again, now that check-scope goes. - The visual jobs install nothing outside their visual scope: visual-scout's setup and pr-visual's setup and Playwright install skip with the rest. - The changed-file list reaches the visual scope through a file, since a large pull request's list can outgrow an environment variable. - The RTL join accepts a skipped/success mix of the shard and merge jobs as an out-of-scope no-op. - A test pins that a failing audit classifier runs every audit in full (spec:AST-030 FR9). - The lab-readiness check recognizes the action's component-audit gate and still fails when ci.yml has no recognizable gate.

astracat-bot

Changes requested: the per-job component scope holds up and the earlier points are addressed, but one new fail-closed edge still lacks the mutation test AST-030 FR9 asks for.

In visual-scout and pr-visual, the new Decide stable visual scope fallback (a pull request without a readable file list, or a non-PR event, gets the full visual plan) can be switched to report no visual work, or lose its ! -s "$FILES_PATH" check, with every routing suite still green; a test that runs that step with an empty FILES_PATH and with a non-PR event would close it.
When the stack merges main, keep main's ci:heavy-lanes label in all eight heavy-lane job conditions, which all conflict, as the description says.
Nit: the ciComponentRoots doc comment in internal/lab-readiness/automated.mjs still describes the old registry-roots fallback.

[Automated review]

astracat-bot · 2026-10-30T05:44:37Z

+            printf 'has_stable_visual=false\nbroad_stable_visual=false\n' >> "$GITHUB_OUTPUT"
+            exit 0
+          fi
+          if [ "${{ github.event_name }}" != pull_request ] || [ ! -s "$FILES_PATH" ]; then


This branch is the AST-030 FR4 fallback (no readable file list, or not a pull request, gets the full visual plan), but no test reaches it: printing has_stable_visual=false on the next line, or deleting || [ ! -s "$FILES_PATH" ] here, leaves every routing suite green, and the same holds at line 2158. FR9 asks for a test that fails when it goes; running this step's script with an empty FILES_PATH on pull_request and with a non-PR event, and expecting has_stable_visual=true and broad_stable_visual=true, would cover both jobs.

astracat-bot · 2026-10-30T05:44:37Z

+ * Roots the ci.yml component audits cover. Without change detection in
+ * ci.yml (the audits run every registered package in full for release-check)
+ * or when it projects the registry, that is every registered package root.


Nit: the function returns no roots when it recognises no gate (line 132), so the lab check fails; this clause still describes the earlier fallback to every registered package root.

The visual jobs' scope step is run with an unreadable file list, outside a pull request and on a specialized lane, so changing either fallback fails the suite (spec:AST-030 FR9). The ciComponentRoots comment describes the gate it now recognizes.

astracat-bot

Changes requested: the visual-scope fallback test and the doc comment from the last round are in, but one more fail-closed edge in the accessibility jobs still lacks the mutation test AST-030 FR9 asks for.

pr-a11y-guards and pr-a11y-specs get has_components from their scope step only because they pass component-audit: 'true', and deleting that input leaves every suite green while both jobs would skip their audits, guards and specs and still report success, on release checks too; extending the RTL pin in .github/scripts/ci-test-routing.test.mjs (lines 417-420) to every job that reads the scope step's audit outputs would close it.
When the stack merges main, keep main's ci:heavy-lanes label in the heavy-lane job conditions, as the description says.

[Automated review]

astracat-bot · 2026-10-30T06:22:45Z

+        id: scope
+        uses: ./.test-scope/.github/actions/test-scope
+        with:
+          component-audit: 'true'


Nothing pins this input: deleting it here (or at line 1648 in pr-a11y-specs) leaves every suite green, yet the scope step then emits no has_components, so on a release check or a pull request whose file list cannot be read every step below is skipped, and this job, pr-a11y and the release-check join still pass (AST-030 FR9, FR12).

astracat-bot · 2026-10-30T06:22:45Z

-    expect(commands).toContain('has_rtl_harness=false');
-    expect(commands).toContain('force_full_component_audits=false');
-    expect(commands).toContain('has_stable_visual=false');
+    for (const job of [shard, join, workflow.jobs['pr-rtl-merge']]) {


This pin covers the three RTL jobs only. Extending it to pr-a11y-guards, pr-a11y-specs, visual-scout and pr-visual, or deriving the list from every job whose steps read steps.scope.outputs.has_components or changed_files_path, would make dropping the input from an accessibility job fail a test.

The a11y guards and specs, the RTL jobs and the visual jobs read the scope step's component-audit outputs; dropping `component-audit: 'true'` from any of them now fails the suite instead of silently skipping its audits (spec:AST-030 FR9).

astracat-bot

Approval recommended: the new routing test fails whenever a job that reads the scope step's audit outputs drops component-audit: 'true', which closes the last gap, and the per-job component scope otherwise follows the CI routing spec. When the stack merges main, keep main's ci:heavy-lanes label in the heavy-lane job conditions, as the description says.

[Automated review]

cixzhang requested review from imdreamrunner and josephfarina as code owners October 30, 2026 01:52

cixzhang mentioned this pull request Oct 30, 2026

ci: each test job decides its own scope; fold and drop unit-level jobs #8888

Closed

meta-cla Bot added the CLA Signed This label is managed by the Meta Open Source bot. label Oct 30, 2026

vercel Bot deployed to Preview October 30, 2026 01:59

View deployment

cixzhang added a commit that referenced this pull request Oct 30, 2026

Rebase onto the split consolidation (#8919)

b26fe17

#8888 is superseded by four atomic PRs; this change now sits on the last of them. The relevance-registry edit drops out: that registry is not on main.

cixzhang added a commit that referenced this pull request Oct 30, 2026

Rebase onto the split consolidation (#8919)

9d33e0d

#8888 is superseded by four atomic PRs; this change now sits on the last of them. The relevance-registry edits drop out: that registry is not on main.

This was referenced Oct 30, 2026

ci: drop off-PR and informational checks #8894

Open

ci: PR report links the branch preview at once and measures the base from main's build #8892

Open

cixzhang mentioned this pull request Oct 30, 2026

ci: one docsite-test job builds the docsite once and runs the data tests and browser contracts on it #8891

Open

cixzhang mentioned this pull request Oct 30, 2026

ci: drafts and outside contributors run CI on a maintainer's call; one author's PRs share heavy suites (AST-030 FR16–FR17) #8893

Merged

3 tasks done

astracat-bot Bot requested changes Oct 30, 2026

View reviewed changes

cixzhang added 2 commits October 29, 2026 21:51

Merge part 3's review follow-ups

ddfd951

cixzhang added a commit that referenced this pull request Oct 30, 2026

Merge #8919's review follow-ups

5b742af

cixzhang added a commit that referenced this pull request Oct 30, 2026

Merge #8919's review follow-ups

b87f1d3

vercel Bot deployed to Preview October 30, 2026 05:11

View deployment

astracat-bot Bot requested changes Oct 30, 2026

View reviewed changes

cixzhang added 2 commits October 29, 2026 22:50

Merge parts 2 and 3's test follow-ups

b13d7e7

cixzhang added a commit that referenced this pull request Oct 30, 2026

Merge #8919's test follow-ups

fe019ef

cixzhang added a commit that referenced this pull request Oct 30, 2026

Merge #8919's test follow-ups

9d26aef

vercel Bot deployed to Preview October 30, 2026 05:58

View deployment

astracat-bot Bot requested changes Oct 30, 2026

View reviewed changes

vercel Bot deployed to Preview October 30, 2026 06:57

View deployment

astracat-bot Bot approved these changes Oct 30, 2026

View reviewed changes

Conversation

cixzhang commented Oct 30, 2026

Who this helps

What changes

Line count

Verification

Uh oh!

vercel Bot commented Oct 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

astracat-bot Bot commented Oct 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

astracat-bot Bot left a comment

Choose a reason for hiding this comment

Uh oh!

astracat-bot Bot Oct 30, 2026

Choose a reason for hiding this comment

Uh oh!

astracat-bot Bot Oct 30, 2026

Choose a reason for hiding this comment

Uh oh!

astracat-bot Bot Oct 30, 2026

Choose a reason for hiding this comment

Uh oh!

astracat-bot Bot Oct 30, 2026

Choose a reason for hiding this comment

Uh oh!

astracat-bot Bot Oct 30, 2026

Choose a reason for hiding this comment

Uh oh!

astracat-bot Bot left a comment

Choose a reason for hiding this comment

Uh oh!

astracat-bot Bot Oct 30, 2026

Choose a reason for hiding this comment

Uh oh!

astracat-bot Bot Oct 30, 2026

Choose a reason for hiding this comment

Uh oh!

astracat-bot Bot left a comment

Choose a reason for hiding this comment

Uh oh!

astracat-bot Bot Oct 30, 2026

Choose a reason for hiding this comment

Uh oh!

astracat-bot Bot Oct 30, 2026

Choose a reason for hiding this comment

Uh oh!

astracat-bot Bot left a comment

Choose a reason for hiding this comment

Uh oh!

This branch was successfully deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

vercel Bot commented Oct 30, 2026 •

edited

Loading

astracat-bot Bot commented Oct 30, 2026 •

edited

Loading