Skip to content

ci: drafts and outside contributors run CI on a maintainer's call; one author's PRs share heavy suites (AST-030 FR16–FR17) - #8893

Merged
cixzhang merged 53 commits into
mainfrom
lane4/heavy-caps
Oct 30, 2026
Merged

cixzhang merged 53 commits into
mainfrom
lane4/heavy-caps

Conversation

@cixzhang

@cixzhang cixzhang commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

User impact

  • Before: every pull request started CI on every push, whether it was a draft, came from someone outside the organization, or belonged to an author with dozens open. One author's pull requests could fill the runner pool and delay everyone else's checks.
  • After:
    • A draft runs no CI at all, so a draft push starts no runner. Marking it ready starts its pull-request run, which runs what the change reaches.
    • A pull request from a fork runs CI once a maintainer labels it ci:run; the label takes effect on the next push or a close and reopen. Until then its required checks fail with the reason instead of passing as skipped.
    • Heavy browser suites run for each author's three oldest open pull requests, up to twelve heavy runs a day.
  • Unchanged: branches pushed to this repository run as before, apart from drafts and the heavy-suite caps.

Problem and solution fit

  • Why does one author crowd out the others? CI concurrency is per pull request, so nothing limits how many of one author's pull requests run heavy suites at once.
  • The fix: a heavy-caps step that each heavy job runs right after its checkout checks the per-author limits. No new job or workflow.
  • Gate safety: a skipped suite must never pass a required join. So a queued or capped heavy job fails with a clear message rather than reporting success. The pre-merge restart is a re-run, and a re-run is exempt, so the pull request about to merge always gets its full heavy suites.

Changes

  • .github/scripts/ci-pr-gate.mjs: when a pull request runs CI (spec:AST-030 FR17).
    • Events other than a pull request pass.
    • A draft never runs.
    • Otherwise the head branch must live in this repository (pushing one needs write access), or the pull request must carry ci:run.
    • Why not author_association: GitHub reports private org members as CONTRIBUTOR.
  • Six pull_request workflows (ci.yml, lint.yml, cli-smoke-test.yml, internal-registry.yml, codemod-verify.yml, crowdin-download.yml):
    • every job that can start on a pull request begins with that condition;
    • the jobs that report required contexts (build, test, docsite-test, lint) skip only a draft. For an unlabelled fork, their first step fails with "Waiting for a maintainer's ci:run label", since a skipped required check counts as passing;
    • pull_request also starts on ready_for_review;
    • there is no label trigger, because a run started by any label would skip the required checks and mask a real failure on the same commit;
    • the workflows stay on pull_request (never pull_request_target), so a fork's code runs without secrets.
  • The ci:run label exists, and CONTRIBUTING.md says when CI runs and who applies the label.
  • .github/actions/heavy-caps/action.yml (new): caps heavy suites per author (spec:AST-030 FR16).
    • Oldest three: one GraphQL search of the author's open non-draft pull requests, counting the ones older than this pull request by number. A fourth or later pull request fails with "Queued behind your 3 oldest PRs: runs on a push once it is among them, or at the pre-merge re-run."
    • Daily cap: each heavy job records that its suite ran in a Heavy suite runs step right after the caps step. The cap counts today's uncancelled runs on the author's oldest pull requests, matched by branch and the repository it lives in, in which that step succeeded. Past twelve, the job fails with "Daily heavy-run cap reached, runs tomorrow or at pre-merge."
    • Shared agent login: each branch prefix counts as an author, with a five-PR and forty-run backstop. Bots are exempt.
    • Exemptions: a re-run (the pre-merge restart) skips both caps, and a failed lookup runs the suite.
    • The limits are an ordered list in the step, so another limit joins as one more entry.
    • pr-interaction, test-build and fixture-contrast run it after their checkout, and gain actions: read and pull-requests: read for the lookups.
  • Tests:
    • ci-pr-gate.test.mjs checks every pull-request workflow's jobs and triggers against the condition. It also evaluates the conditions for a branch here, a fork with and without ci:run, another label, a draft, and non-PR events, and checks that each required job fails rather than skips.
    • heavy-caps-action.test.mjs runs the step's script against stubbed lookups for each cap rule, and pins the heavy jobs, their permissions and marker steps.
    • Workflow-contract tests that pin exact job conditions now expect the condition in front.
  • spec:AST-030: FR16 (per-author heavy caps) and FR17 (drafts and outside contributors), each with a Verification row.

Why a draft's skipped required checks are safe

A draft's required checks read as skipped, and GitHub counts skipped as passing. That window is closed in three ways:

  • A draft cannot merge.
  • Marking it ready starts a new run at once, whose checks queue under the same names, and the newest check run wins.
  • Merges are made on demand, on fresh checks finished at the exact head.

Failing them instead would start four runners on every draft push, against the rule that drafts run no CI at all.

LOC

+822/−41 against main, net +781. Tests are about +420 of it: the gate's contract and evaluation, and the cap cases. The workflow conditions mostly rewrite lines in place. The rest is the caps step (+115), the shared conditions (+45), CONTRIBUTING (+28) and the spec. Plan 1 as a whole stays net-negative, since the #8888 split removes the scope jobs, about 2,000 lines.

Evidence

  • .github/scripts Node suite: 1,487 pass. Mutation checks turn the tests red when a required job loses its fail-fast step, when build loses its start-for-a-pull-request clause, and when the daily cap moves off twelve.
  • pnpm check:knowledge and check:repo pass.
  • Decisions: the maintainer's decisions of Oct 28 (per-author caps) and Oct 29 (drafts run no CI; outside contributors run CI on a maintainer's ci:run label), within the approved CI tiers design. The label takes effect on the next push or reopen (no labeled trigger), and a draft's skipped required checks are accepted as safe, both confirmed by the maintainer.

Scope

  • One change: when pull-request CI runs (drafts, outside contributors, and per-author heavy suites).
  • No required context renamed or removed. Merge-queue entries, pushes to main, dispatches and release checks are unaffected.
  • Public text contains no internal Meta context.

Based on main, independent of the #8888 split.

Writing rubric results

  • R1 narration: none. FR16 and FR17 are present tense.
  • R2 research: none in the record. The replay numbers stay in this description.
  • R3 evidence: the FR16 and FR17 Verification rows name their test cases.
  • R4 rejected alternatives: none in the record. Per-author concurrency groups were rejected in the design, because GitHub cancels another pull request's pending run in the same group.
  • R5 change references: none.
  • R6 open questions: none.

…thout deepening history

Checking out the PR head (#8812) made pr-scope-touches.sh deepen a shallow
history back to the PR's recorded base. On long-lived branches that merge
main often, the interaction-scope decide step went from a few seconds to up
to three minutes, and one run hit the job's five-minute timeout, which fails
the required test join. Both scope jobs now check out the merge commit with
its parents and pass its first parent as the base, so the helper diffs the
PR's own change against the base it merged into with no deepening. The merge
queue keeps its base. A contract test pins the checkout and asserts no
deepening on a branch that merged main four times.
…dates

Pull request updates keep the fast checks and every required context. The
visual, accessibility and RTL owners run for release-check, which Main now
dispatches on each push to main (one at a time, skipping a commit main has
moved past, failing Main when it fails), and which the release gate already
runs before a cut. A pull request opts in with the ci:heavy-lanes label. A
main failure is fixed forward. AST-030 FR12 and the new DEC-7 record the
change.
One run per workflow and ref at a time, never cancelling a running check, so a
burst of pushes is checked once at its newest commit, and Main's fast checks
are no longer held behind the heavy wait.
lint.yml classifies a pull request's files with the trusted base classifier and
skips full ESLint for a spec-only, docsite-only or tooling-only change. Its
depth-1 checkout never let the merge commit reach its merge base, so the
classifier was never fed and every pull request ran full lint (about four
minutes of ESLint on a spec-only change). The checkout now takes the same
50-commit history ci.yml's check-scope does. A contract test runs the real
step and classifier with real git.
The job checked out the whole history on a 4-core runner for every pull
request into main, then learned from the branch name that only a release
merge-back has anything to validate: about six minutes of larger-runner time
per run for nothing. Its condition now names the merge-back branch pattern,
so other pull requests skip it outright; the exact branch contract is still
checked after the checkout, and required lint still validates merge-backs.
Internal Registry's dependency check runs only when a dependency
declaration or the lockfile changes, the CLI smoke test only when a
package, the workspace configuration or its own scripts change (its
paths-ignore becomes an allowlist), and the hardening-issue opener only
when Core component sources change. A registry records every workflow's
kind and reason, and a contract test holds each workflow to it: no
paths-ignore, no match-all glob, and no path filter on a workflow that
reports a required context.
…rds why (review)

Internal Registry's header no longer says to make its check required:
behind its path filter it never starts on a pull request that touches no
dependency file. Re-enabling gating means dropping the filter and
registering it as change-scope. A review-event workflow on
pull_request_target now records in the registry why every pull-request
update matters to review state, and the contract test requires it.
Removes the scope classifier jobs (check-scope, interaction-scope,
native-swift-scope, check-components). Each job that owns tests now runs
.github/actions/test-scope right after a sparse checkout: it reads the pull
request's file list and the base commit's trusted classifier, and skips the
job's remaining steps with success when nothing in its scope changed.
Anything it cannot decide puts everything in scope (spec:AST-030 FR3, FR4).
native-swift-runner is the one pre-job left, because only a job's runner has
to be chosen before it starts.

Also:
- fixture-contrast's steps run in test-build (same Playwright runner);
- registry-contract becomes test-registry-e2e and runs for release-check
  only; its network-free production gate runs with the unit tests;
- ci-duration and the weekly a11y and RTL workflows are removed (the full
  audits run on main);
- the Internal Registry job becomes test-deps, scoped by its trigger paths;
- the test join requires release-bump on a release-branch pull request.
The main-tier release-check dispatcher runs on every landed commit, like the
Main workflow, so it is a landed-commit workflow.
A heavy or end-to-end job now runs its suite only for a ready pull
request among its author's three oldest open ones, within twelve CI runs
that author started today. The shared agent login counts per branch
prefix, with a five-PR and forty-run backstop; bots are exempt. A draft
runs the fast checks and skips the heavy job; ready_for_review runs it. A
queued or capped heavy job fails with a clear message, so no pull request
passes the required joins without its heavy suites, and a re-run (the
pre-merge restart) is exempt. A failed lookup runs the suite.
@vercel

vercel Bot commented Oct 29, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
astryx Ready Ready Preview Oct 30, 2026 9:34am UTC

Request Review

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Meta Open Source bot. label Oct 29, 2026
@astracat-bot

astracat-bot Bot commented Oct 29, 2026 •

Copy link
Copy Markdown

AI review status for this pull request.

Review status Updated
🟢 Reviewed (for maintainers only) Oct 30, 2026, 9:52 AM UTC

…s and says how the queue clears

FR16 now says exactly what the daily cap counts: CI runs the author
started that day that were not cancelled, since a run a newer push
cancelled seldom reached the heavy suites. The queued message says how it
clears: on a push once the pull request is among the author's three
oldest, or at the pre-merge re-run.
Each heavy job records that its suite ran in a 'Heavy suite runs' step.
The cap counts the author's runs today, on their oldest open pull
requests, in which that step succeeded, so a draft's run, a queued pull
request's run or a cancelled run never counts toward the twelve.
@cixzhang

Copy link
Copy Markdown
Contributor Author

Posted by Cindy's agent on her behalf.

/approve-spec 4e3b0c0

@astracat-bot astracat-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes requested: the per-author caps let heavy suites through in cases FR16 says must wait. Please fix the two paging gaps in .github/actions/test-scope/action.yml and make FR16's job coverage match the jobs that pass heavy: 'true'.

  • A pull request missing from the 100-result search gets index -1 and skips the queue, which today covers 215 of one author's 315 open ready pull requests; place an unlisted pull request after every listed one (lines 205-210).
  • The daily cap reads one page of today's runs and GitHub's default 30 jobs per run; paginate both, or list runs per oldest branch (lines 216-224).
  • FR16 covers every heavy or end-to-end job, but docsite-browser has no heavy flag, and no contract test checks the flag or the Heavy suite runs step name.

[Automated review]

Comment thread .github/actions/test-scope/action.yml Outdated
'query($q: String!) { search(query: $q, type: ISSUE, first: 100) { nodes { ... on PullRequest { number headRefName } } } }',
{q: `repo:${context.repo.owner}/${context.repo.repo} is:pr is:open draft:false author:${login} sort:created-asc`},
);
const position = list => list.findIndex(p => p.number === pr.number);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

findIndex returns -1 for a pull request outside the 100-node answer, and -1 trips neither >= 3 nor >= 5, so that pull request skips the queue, and its runs never reach the daily-cap count. Today that is 215 of one author's 315 open ready pull requests. Placing an unlisted pull request after every listed one keeps FR16's "three oldest" rule:

const position = list => {
  const i = list.findIndex(p => p.number === pr.number);
  return i === -1 ? list.length : i;
};

Comment thread .github/actions/test-scope/action.yml Outdated
Comment on lines +216 to +219
const {data} = await github.rest.actions.listWorkflowRuns({
...context.repo, workflow_id: 'ci.yml', actor: login, event: 'pull_request',
created: `>=${new Date().toISOString().slice(0, 10)}`, per_page: 100,
});

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is one page of 100 runs, newest first. Once an author passes 100 ci.yml runs in a UTC day (draft, queued and cancelled runs fill the page too), earlier heavy runs drop out of the twelve and forty counts. github.paginate(github.rest.actions.listWorkflowRuns, {...}), or one call per oldest branch with branch:, counts the whole day.

Comment thread .github/actions/test-scope/action.yml Outdated
let loginHeavy = 0;
for (const run of data.workflow_runs) {
if (run.id === context.runId || run.conclusion === 'cancelled' || !oldest.has(run.head_branch)) continue;
const {data: {jobs}} = await github.rest.actions.listJobsForWorkflowRun({...context.repo, run_id: run.id});

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This reads GitHub's default page of 30 jobs, while a ci.yml pull-request run lists at least 39 (33 jobs, with test-ui and test-node as four shards each). Pass per_page: 100 or paginate, so the Heavy suite runs step is found wherever GitHub places test-build and pr-interaction.

Comment thread docs/specs/AST-030/spec.md Outdated
kind and reason, and a contract test holds every workflow to its entry.

- **FR16 — One author's pull requests share the heavy suites.** On a
pull request, a heavy or end-to-end job runs its suite only when the pull

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"a heavy or end-to-end job" also covers docsite-browser and the labelled visual, accessibility and RTL owners, but only pr-interaction and test-build pass heavy: 'true' at this head. Either name the jobs FR16 governs today or flag the others, and add a workflow contract test that each governed job passes the flag and runs a Heavy suite runs step.

…dest branch's runs

The oldest-three check counts the author's older pull requests by
number, so a pull request missing from the search's 100 oldest results
is still queued behind them. The daily cap lists today's runs per oldest
branch, so a busy author's runs are not cut off at one page (review).
@cixzhang

Copy link
Copy Markdown
Contributor Author

Posted by Cindy's agent on her behalf.

/approve-spec 23783bf

…per job

Review follow-ups:
- The registry contract, ci-duration, the weekly sweeps and the Internal
  Registry rename move to their own pull request, so this one only removes
  the scope classifier jobs and folds fixture-contrast into test-build.
  registry-contract stays a pull-request job here, deciding its own scope.
- A ci:heavy-lanes pull request keeps its visual and RTL audits on the
  affected components (spec:AST-030 FR13): the scope action can also run the
  base's component-audit-scope.cjs and list the changed files, and the RTL
  and visual jobs read those instead of a classification job.
- The interaction and native Swift scopes include the scope action, so a
  change to it runs the jobs it gates.

@astracat-bot astracat-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approval waits on an engineering owner's decision on AST-030 FR16 (one author's heavy suites run only on their three oldest open ready pull requests, at most twelve heavy runs a UTC day) and FR17 (a draft runs no CI; a fork's pull request runs CI only with the maintainer-applied ci:run label), which are new requirements. Approving FR16 as written also approves the per-branch-prefix lanes that .github/actions/heavy-caps/action.yml gives the cixzhang login, which the record calls only "the shared agent login". The branch now conflicts with main in .github/workflows/lint.yml; keep the ci:run step ahead of the checkout that #8880 gave fetch-depth: 50. Because that creates a new head, an engineering owner, including the author as one, then records the decision by commenting /approve-spec followed by that head's full commit SHA.

[Automated review]

…y-caps

# Conflicts:
#	.github/workflows/lint.yml

@astracat-bot astracat-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approval waits on an engineering owner's decision on AST-030 FR16 (one author's heavy suites run only on their three oldest open ready pull requests, at most twelve heavy runs a UTC day) and FR17 (a draft runs no CI; a fork's pull request runs CI only with the maintainer-applied ci:run label), which are new requirements. One change is also needed, and making it first saves a second approval: FR16 and FR17 write implementation detail into the requirements (the heavy-caps step's place after the checkout, the Heavy suite runs marker step, the ordered list of limits, and which step of which job carries the gate), which INV18 in docs/architecture/knowledge-contracts.md keeps out of new requirements, so please state only the observable rules and leave the mechanism to the action's comments or an architecture record. Once that head is pushed, an engineering owner, including the author as one, records the decision by commenting /approve-spec followed by its full commit SHA.

[Automated review]

Comment thread docs/specs/AST-030/spec.md Outdated
pre-merge restart, is exempt from both caps, and a lookup that fails runs the
suite. Merge-queue entries, `main`, release checks and the visual, accessibility and
RTL owners a maintainer opts a pull request into with `ci:heavy-lanes` are
unaffected. The limits are an ordered list in that step, so another limit

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

INV18 in docs/architecture/knowledge-contracts.md says a new requirement "MUST NOT prescribe … data structures … or CI job/workflow topology unless that exact mechanism is itself an intentional public contract". This sentence, the heavy job defined as "one that runs the heavy-caps step right after its checkout" (lines 254-255), and the Heavy suite runs step as the record of a heavy run (259-260) describe how the caps are built rather than what a contributor sees. Naming the capped checks, the limits, what counts as a heavy run, the messages and the exemptions is enough; the mechanism can stay in the action's header comment.

Comment thread docs/specs/AST-030/spec.md Outdated
its head branch lives in this repository, which pushing needs write access
for, or when it carries the maintainer-applied `ci:run` label; the label
takes effect on the next push or a close and reopen. Every job that can start
on a pull request begins with that condition, except the jobs that report a

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same INV18 point: "Every job that can start on a pull request begins with that condition" and "their first step fails" fix job and step topology. The observable rule is that no pull-request job runs for a draft or an unlabelled fork, except that build, test, docsite-test and lint read Skipped for a draft and fail with "Waiting for a maintainer's ci:run label" for an unlabelled fork.

…dge-contracts INV18) (review)

The requirements now name what a pull request's author and maintainers
observe: which suites are held and when, the failure messages, the
ci:run label and the required contexts. The step, job, workflow-trigger
and event-type details move out of the record; the tests that prove the
claims stay named in the Verification rows.

@astracat-bot astracat-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approval waits on an engineering owner's decision on AST-030 FR16 (one author's heavy browser suites run only on their three oldest open non-draft pull requests, at most twelve heavy runs a UTC day) and FR17 (a draft runs no CI; a fork's pull request runs CI only with the maintainer-applied ci:run label), which are new requirements; the rewording to observable outcomes resolves the earlier implementation-detail point. An engineering owner, including the author as one, records the decision by commenting /approve-spec followed by this head's full commit SHA. Two wording points in FR16 are worth settling first, since any edit needs its own approval:

  • "has had at most twelve heavy runs that UTC day" reads as allowing a thirteenth run, while the caps hold the thirteenth.
  • "no pull request passes the required checks without its heavy suites" does not cover fixture-contrast, which no required check depends on.

[Automated review]

request, the heavy browser suites (the interaction proofs and the Build
package's browser and fixture-contrast contracts) run only when the pull
request is among its author's three oldest open non-draft pull requests and
the author has had at most twelve heavy runs that UTC day. A heavy run is a

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This reads as letting a run start when the author has already had twelve heavy runs that UTC day, which would make it the thirteenth. The caps hold the thirteenth (laneHeavy >= 12 in .github/actions/heavy-caps/action.yml), as the FR16 Verification row's "thirteenth heavy run of the day" expects; "at most twelve heavy runs that UTC day, this one included" would say what the caps do.

Comment on lines +265 to +266
pre-merge.", so no pull request passes the required checks without its heavy
suites. A re-run, which is the pre-merge restart, is exempt from both limits,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This holds for the interaction proofs and the Build browser contracts, which feed the required test, but no required check depends on fixture-contrast: it is held only because test-build evaluates the same limits in the same run, and the two can disagree when the author's open pull requests or today's heavy runs change between their lookups. Narrowing the sentence to the suites test joins, or saying that a held suite's own check fails, would match what CI does.

if (login.endsWith('[bot]')) return;
// Agent lanes share one login: each branch prefix counts as an author.
const shared = ['cixzhang'].includes(login);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FR16 calls this "the shared agent login" without naming it, so the record alone does not say which account gets per-branch-prefix limits and the five-PR, forty-run backstop. With this login's 89 open non-draft pull requests, the backstop lets at most its five oldest run heavy suites and holds every newer one, this pull request included.

…y-caps

# Conflicts:
#	docs/specs/AST-030/spec.md
@cixzhang

Copy link
Copy Markdown
Contributor Author

Posted by Cindy's agent on her behalf.

Owner decisions applied in FR16/FR17 at this head:

  • Cindy's 18:54 rules: heavy suites run only on an author's three oldest ready pull requests, at most 12 heavy runs a day, and with the agent-lane backstops.
  • Cindy's 15:26 rule: draft pull requests run no CI at all. CodeBunny's draft-window point is accepted on that basis. A draft can't merge, marking it ready starts the run, and a merge needs fresh checks at the exact head.
  • Cindy's 15:29 rule: outside contributors' pull requests run CI only with the maintainer-applied ci:run label.

The outcome-only wording (knowledge-contracts INV18) doesn't change any of these rules.

@cixzhang

Copy link
Copy Markdown
Contributor Author

/approve-spec fd68c5e

@astracat-bot astracat-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes requested: FR16 and FR17 are approved for this head and the gate and caps behave as they describe, but FR16's daily cap is missing the test its Verification row promises. Every daily-cap case in heavy-caps-action.test.mjs puts the author's heavy runs on a single pull request's branch, so a regression that counted only that pull request's runs instead of the author's three oldest would still pass. Please add cases with heavy runs spread across the author's (and the shared login's) oldest pull requests, then have an engineering owner /approve-spec the new head. That push needs a fresh approval anyway, so it is a cheap moment to narrow FR16's "no pull request passes the required checks without its heavy suites", since no required check depends on fixture-contrast.

[Automated review]

Comment on lines +140 to +166
it("fails a heavy job past the author's 12 heavy runs today", async () => {
expect((await check({open: [me], runs: runsOn(12)})).failed).toEqual([
CAPPED,
]);
// Cancelled runs, runs whose heavy suites never ran (a queued PR) and runs
// on other branches are not heavy runs.
const light = await check({
open: [me],
runs: [
...runsOn(11),
...runsOn(5, 'fix/a', {conclusion: 'cancelled'}),
...runsOn(5, 'fix/a', {heavy: false}),
...runsOn(5, 'fix/old-merged'),
],
});
expect(light.failed).toEqual([]);
// A run is the pull request's by its branch and that branch's repository:
// another fork's same-named branch is not, whoever pushed.
const elsewhere = await check({
open: [me],
runs: [
...runsOn(11),
...runsOn(5, 'fix/a', {head_repository: {full_name: 'other/r'}}),
],
});
expect(elsewhere.failed).toEqual([]);
});

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Every case here puts the author's heavy runs on me's own branch (or on a branch outside the oldest three), so the sum over the author's three oldest pull requests that FR16 defines, built at .github/actions/heavy-caps/action.yml:64, is never exercised: counting only this pull request's runs, only two of the three oldest, or only the shared login's oldest pull request leaves all six tests green. Cases like these pass on this head and fail on each of those changes:

it("sums heavy runs over the author's oldest pull requests", async () => {
  const spread = await check({
    open: [...pulls('fix/b', 'fix/c'), me],
    runs: [...runsOn(6, 'fix/b'), ...runsOn(6, 'fix/c')],
  });
  expect(spread.failed).toEqual([CAPPED]);
  const third = await check({
    context: pullRequestContext({number: 2}),
    open: [
      ...pulls('fix/b'),
      {...me, number: 2},
      {number: 3, headRefName: 'fix/d', headRepository: fork},
    ],
    runs: runsOn(12, 'fix/d'),
  });
  expect(third.failed).toEqual([CAPPED]);
  const login = await check({
    context: pullRequestContext({
      user: {login: 'cixzhang'},
      head: {ref: 'lane4/x', repo: {full_name: 'o/r'}},
    }),
    open: [
      ...pulls('lane1/a', 'lane2/b'),
      {number: 7, headRefName: 'lane4/x', headRepository: here},
    ],
    runs: [...runsOn(38, 'lane2/b'), ...runsOn(2, 'lane4/x')],
  });
  expect(login.failed).toEqual([CAPPED]);
});

…ldest pull requests (review)

Every daily-cap case put its runs on the deciding pull request, so a cap
that counted only that pull request's runs passed. A case now splits
twelve heavy runs across three of the author's oldest pull requests;
counting only the deciding one turns it red.
@cixzhang

Copy link
Copy Markdown
Contributor Author

/approve-spec b1ad5e4

@astracat-bot astracat-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes requested: the new spread case now catches a daily count that reads only the deciding pull request, but FR16's Verification row still lacks tests for two other terms of the cap's sum. Counting only two of the author's three oldest pull requests, or only the shared login's oldest, in .github/actions/heavy-caps/action.yml still leaves heavy-caps-action.test.mjs green. Please add a case where twelve heavy runs on the author's third-oldest pull request hold the second-oldest, and one where 38 heavy runs on the shared login's second-oldest plus 2 on the deciding lane hold it (both pass on this head), then have an engineering owner /approve-spec the new head. That push is also a cheap moment to narrow FR16's "no pull request passes the required checks without its heavy suites", since no required check depends on fixture-contrast.

[Automated review]

open: [...pulls('fix/b', 'fix/c'), me],
runs: [...runsOn(6, 'fix/b'), ...runsOn(3, 'fix/c'), ...runsOn(3)],
});
expect(spread.failed).toEqual([CAPPED]);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The spread case now fails a count that reads only this pull request, but its deciding pull request is the third-oldest, and the shared login's forty-run case below keeps the other lanes' runs on the login's oldest pull request. So .slice(0, 3) → .slice(0, 2) or nodes.slice(0, 5) → nodes.slice(0, 1) at .github/actions/heavy-caps/action.yml:64 still leaves every test green, although FR16's Verification row expects a heavy suite that runs past the daily cap to fail one. These cases pass on this head, and each fails on one of those edits:

it('sums heavy runs over every one of the oldest pull requests', async () => {
  const third = {number: 8, headRefName: 'fix/c', headRepository: fork};
  expect(
    (await check({open: [...pulls('fix/b'), me, third], runs: runsOn(12, 'fix/c')}))
      .failed,
  ).toEqual([CAPPED]);
  const context = pullRequestContext({
    user: {login: 'cixzhang'},
    head: {ref: 'lane4/x', repo: {full_name: 'o/r'}},
  });
  const mine = {number: 7, headRefName: 'lane4/x', headRepository: here};
  expect(
    (
      await check({
        context,
        open: [...pulls('lane1/a', 'lane2/b'), mine],
        runs: [...runsOn(38, 'lane2/b'), ...runsOn(2, 'lane4/x')],
      })
    ).failed,
  ).toEqual([CAPPED]);
});

…red login's fifth-oldest pull request (review)

Two cases put the deciding pull request first and spread the heavy runs
over the rest: the author's three oldest reach twelve, and the shared
login's five oldest reach forty. Counting one fewer pull request in
either list turns its case red.
@cixzhang

Copy link
Copy Markdown
Contributor Author

/approve-spec ba51294

@astracat-bot astracat-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes requested: the two new daily-cap cases now pin the count's reach to the author's third-oldest and the shared login's fifth-oldest pull request, but FR17's gate test can miss a workflow. .github/scripts/ci-pr-gate.test.mjs examines only pull_request workflows that list types:, so a workflow added the way these six were written before this change (only branches:, or on: [pull_request]) can run ungated jobs for drafts and unlabelled forks with every case green; reading a missing types as GitHub's defaults, as pr-concurrency.test.mjs does, closes the gap. The pull request also conflicts with main, so the fix can ride that merge, and FR16's "no pull request passes the required checks without its heavy suites" can be narrowed at the same time, since no required check depends on fixture-contrast. The new head then needs an engineering owner's /approve-spec <full head SHA>, which the author can record as an engineering owner.

[Automated review]

Comment thread .github/scripts/ci-pr-gate.test.mjs Outdated
.filter(([, value]) => {
const pr = (value.on ?? value[true]).pull_request;
return (
pr !== undefined && !(pr?.types ?? []).every(type => type === 'closed')

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This drops any pull_request workflow without a types: list ([].every(...) is true), so a new workflow with only branches:, or on: [pull_request], is never checked and its ungated jobs would run for drafts and unlabelled forks with this suite green. pr-concurrency.test.mjs reads a missing types as GitHub's defaults (types == null || types.includes('synchronize')); doing the same here makes such a workflow fail the exact-list case until it is gated and listed, as FR17's Verification row expects.

…starts, with or without listed types (review)

A pull_request trigger with no types took GitHub's default ones but was
dropped by the selection, so an ungated job there left the test green.
The selection now keeps any pull_request trigger (bare, in a list, or a
map) except one that lists only closed, with cases for each shape.
…y-caps

# Conflicts:
#	.github/workflows/ci.yml
@cixzhang

Copy link
Copy Markdown
Contributor Author

/approve-spec 1be5505

@astracat-bot astracat-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks ready to merge once its required checks pass on this head: FR17's gate test now examines every pull_request workflow, including one that lists no types:, which closes the last gap in its verification, and FR16 and FR17 carry an engineering owner's approval for this exact head. Expect this pull request's own first run to hold its heavy suites, since 75 older open pull requests share its login, so test turns green at the pre-merge re-run FR16 exempts. One optional follow-up: FR16 says no pull request passes the required checks without its heavy suites, but no required check depends on fixture-contrast, so that sentence could name only the suites test joins.

[Automated review]

@cixzhang cixzhang closed this Oct 30, 2026
@cixzhang cixzhang reopened this Oct 30, 2026
@cixzhang
cixzhang merged commit 383dd29 into main Oct 30, 2026
92 of 97 checks passed
cixzhang added a commit that referenced this pull request Oct 30, 2026

This branch was successfully deployed

1 active deployment
Preview — 1be55059 Deployed Oct 30, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Meta Open Source bot.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant