Skip to content

chore(size): fail only on brotli-over-cap with minified growth past a 20 B allowance; lower-only ratchet - #3821

Merged
ryansolid merged 1 commit into
nextfrom
size-gate-minified-allowance
Oct 6, 2026
Merged

ryansolid merged 1 commit into
nextfrom
size-gate-minified-allowance

Conversation

@ryansolid

Copy link
Copy Markdown
Member

Problem

Scenario caps are brotli bytes at CI-measured + 10 B, and brotli's layout is not monotonic: a few minified bytes swing a scenario's brotli by ±50–90 B. PRs fail on noise, and each fix is either a cap raise (a permanent ratchet upward) or golfing the code until the layout comes out lucky. Examples from one day:

The rule

Caps stay in brotli (frozen floors and hello world included). Per scenario, in scripts/size/gate.mjs:

brotli vs cap minified vs base verdict
at or under anything pass
over grew by ≤ MINIFIED_ALLOWANCE (20 B) pass with a warning: over brotli cap by N B, minified +M B — layout noise; cap will be re-based at the next ratchet
over grew by more fail
over no base measurement fail (the cap is absolute, as today)
  • The allowance is one named constant, MINIFIED_ALLOWANCE. A scenario can override it with an optional minifiedAllowance field in scenarios.js; no scenario sets one.
  • Warnings show up in the PR size comment (new "minified vs base" column and a ⚠️ section), the job summary, and as Actions annotations.
  • Size-Exception: works exactly as before. It is still the override for raising a frozen floor cap (check-floor-caps.mjs is unchanged). Real growth still means lowering the bytes or raising the cap in the PR.
  • Ratchet: cd scripts/size && npm run ratchet -- --from size-head.json --note "RC.x, next @ sha" [--dry-run]. It re-bases every cap to measured + 10 B rounded up to 0.01 KB, and only ever lowers a cap.
    • It rewrites inline caps in scenarios.js and frozen floors in floor-caps.json, writing a dated Ratchet ledger line above each cap it lowers.
    • It prints lowered inline caps and lowered frozen floors as separate tables, then lists scenarios that are still over their cap, which it leaves alone.
    • Without --from, it measures the local checkout.

Workflow restructure (why)

check (the required status on next) measured only the head and decided on its own. The base numbers lived in compare, which ran after check (needs: check) and was continue-on-error. So the gate had no base numbers at decision time. Now:

  • head builds and measures the commit under test with size.mjs --no-gate, then uploads size-head. It also uploads on pushes, so the ratchet can use a push's numbers.
  • base builds and measures the base with the head's harness, exactly as compare did, then uploads size-base. It runs in parallel with head and is continue-on-error. If the base can't be measured, the caps stay absolute (fail-safe).
  • check (same name, so branch protection is untouched) needs both. It runs the gate tests, renders the comment and summary, posts the comment (same-repo PRs only, as before), runs gate.mjs, and runs the floor-freeze check.

Two consequences:

  • Pushes to next now compare against github.event.before instead of the absolute cap. Otherwise, a merge that passed with a noise warning would leave next red on the next push check.
  • Timing: the required check now waits for the base build. Today check takes ~2–2.5 min and compare adds ~2.5 min after it. Now head and base run in parallel and check is about +30 s on top. The required check is roughly +0.5–1 min slower, and the whole workflow is ~1.5 min faster.

base.sha is verified to be the first parent of the refs/pull/N/merge commit CI measures as the head, so the minified delta is the PR's own change.

Dry run

These are the actual CI numbers from each run's check log (head) and compare log (base, measured with the head's harness), fed through gate.mjs. Only rows that were over cap, or are otherwise relevant, are shown. The cap is the head harness's cap in that run.

PR / version scenario brotli / cap (over) brotli Δ minified Δ old new
#3807 original (9d1cf78) page: base server components 44,823 / 44,840 +36 +63 pass pass
#3807 original after next moved (5119e44) page: base server components 44,895 / 44,840 (+55) +97 +63 FAIL FAIL
#3807 setLookup (fd13b7d) page: base server components 44,801 / 44,840 +3 +61 pass pass
#3807 setLookup app: hydrating + every store primitive family 28,803 / 28,870 −54 +61 pass pass
#3814 (976fe1e) signals: + createStore 14,579 / 14,560 (+19) +50 +10 FAIL ⚠️ pass
#3814 app: hydrating + every store primitive family 28,879 / 28,870 (+9) +76 +10 FAIL ⚠️ pass
#3814 app: compiled CSR 25,170 / 25,080 (+90) +56 +10 FAIL ⚠️ pass
#3811 F6 before cap raise (1b31bce) signals: core floor 7,360 / 7,350 (+10) +11 +20 FAIL ⚠️ pass
#3811 F6 before cap raise signals: + isPending/latest 9,506 / 9,490 (+16) +20 +20 FAIL ⚠️ pass
#3811 F6 before cap raise app: CSR, observe tier + attribution 28,666 / 28,660 (+6) +24 +20 FAIL ⚠️ pass
#3811 F6 before cap raise page: live server components 48,551 / 48,510 (+41) +70 +20 FAIL ⚠️ pass
#3811 F6 after merging next (7837471) signals: + createStore 14,584 / 14,560 (+24) +55 +20 FAIL ⚠️ pass
#3811 F6 after merging next app: hydrating + every store primitive family 28,892 / 28,870 (+22) +89 +20 FAIL ⚠️ pass
#3811 F6 after merging next app: compiled CSR 25,164 / 25,080 (+84) +50 +20 FAIL ⚠️ pass
#3811 F6 after merging next app: compiled hydrating 30,963 / 30,910 (+53) +45 +20 FAIL ⚠️ pass
#3811 F6 after merging next page: base server components 44,854 / 44,840 (+14) +53 +20 FAIL ⚠️ pass
#3817 before reorder (92ea317) app: hydrating + every store primitive family 28,908 / 28,870 (+38) +105 +59 FAIL FAIL
#3817 before reorder app: compiled CSR 25,114 / 25,080 (+34) 0 0 FAIL ⚠️ pass
#3817 before reorder app: compiled hydrating 30,970 / 30,910 (+60) +52 +59 FAIL FAIL
#3817 before reorder page: base server components 44,881 / 44,840 (+41) +80 +59 FAIL FAIL
#3817 after reorder (76f8f65) app: hydrating + every store primitive family 28,818 / 28,870 +15 +39 pass pass
#3817 after reorder app: compiled hydrating 30,921 / 30,930 +3 +39 pass pass
#3812 (4e40669) signals: + isPending/latest 9,505 / 9,490 (+15) +19 +20 FAIL ⚠️ pass
#3800 create-time commit 91e4745 vs 93ae88d, local macOS, next's caps signals: core floor 7,415 / 7,350 (+65) +61 +170 FAIL FAIL
#3800 create-time app: CSR with Show/For/Loading/Errored/lazy 12,930 / 12,860 (+70) +78 +168 FAIL FAIL
#3800 create-time app: hydrating (no stores) 17,712 / 17,710 (+2) +34 +170 FAIL FAIL
#3800 create-time app: compiled CSR 25,127 / 25,130 −28 +179 pass pass

For #3800, 13 of 17 scenarios are over their cap at +164 to +179 B minified, and all 13 still fail. For #3811, all four Size-Exception cap raises in 896a85e would have been unnecessary.

Ratchet dry run on today's next (076a250, CI-measured):

  • Inline caps: + createStore 14.56 → 14.54 KB, hydrating + every store primitive family 28.87 → 28.82 KB, CSR 12.86 → 12.84 KB, CSR, observe tier 14.46 → 14.40 KB.
  • Frozen floors: hydrating (no stores) 17.71 → 17.69 KB, page: base 44.84 → 44.82 KB.

This PR changes no caps.

Open questions for the maintainer

  1. The ratchet is lower-only, but the warning says "cap will be re-based at the next ratchet." A scenario that merged over its cap on noise stays over it. While it is over, each later PR is held only to the 20 B per-PR allowance, so minified growth can accumulate unbounded until the layout drifts back under the cap. The ratchet lists such scenarios and does not raise them. The warning text is kept verbatim as specified. Options:
    • accept this behavior;
    • allow an explicit --rebase-over flag that raises a still-over cap to measured + 10 B, printed in its own table;
    • record the minified size at cap-set time, so growth is bounded across PRs, not per PR.
  2. The allowance does not rescue fix(signals): a retained projection draft row resolves to its store proxy (#3767) #3807's original version or fix(solid): a live node with a streaming server value takes over at hydration end (#3764) #3817 before its reorder. At +63 and +59 B minified, both are real growth above 20 B, so landing them still depends on brotli coming in under the cap.

Public API changes

None. This touches only scripts/size/ and .github/workflows/size.yml, with no changes under packages/*/src, so there is no changeset. The new internal tooling surface is gate.mjs, ratchet.mjs, size.mjs --no-gate, the optional minifiedAllowance scenario field, and the npm run gate / ratchet / test scripts in scripts/size.

🤖 Generated with Claude via Cursor

…r-only ratchet

A scenario now fails only when it is over its brotli cap AND grew more than
MINIFIED_ALLOWANCE (20 B) minified over the PR's base; over the cap within the
allowance passes with a layout-noise warning. The base measurement moves from
the informational compare job (which ran after check) into a parallel base job
that check waits on; pushes to next compare against the previous commit.
ratchet.mjs re-bases caps to measured + 10 B, lower only, floors listed apart.

Co-authored-by: Claude via Cursor <noreply@cursor.com>
@changeset-bot

changeset-bot Bot commented Oct 6, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: b651365

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@github-actions

github-actions Bot commented Oct 6, 2026

Copy link
Copy Markdown

Size (brotli, eager entry chunk)

scenario head vs base minified vs base cap lazy chunks (not counted)
signals: core floor (createSignal/Memo/Effect/Root/flush) 7.35 KB 0 B 0 B 7.35 KB ✅
signals: + createStore 14.53 KB 0 B 0 B 14.56 KB ✅
signals: + isPending/latest 9.49 KB 0 B 0 B 9.49 KB ✅
app: render + one signal (the simple-app floor) 9.85 KB 0 B 0 B 9.86 KB ✅
app: hydrating (no stores) with Show/For/Loading/Errored/lazy 17.67 KB 0 B 0 B 17.71 KB ✅ lazy-page.js 0.04 KB
app: hydrating + every store primitive family 28.80 KB 0 B 0 B 28.87 KB ✅ lazy-page.js 0.04 KB
app: CSR with Show/For/Loading/Errored/lazy 12.83 KB 0 B 0 B 12.86 KB ✅ lazy-page.js 0.04 KB
app: CSR, observe tier (same app on the observe artifacts) 14.39 KB 0 B 0 B 14.46 KB ✅ lazy-page.js 0.04 KB
app: CSR, observe tier + attribution engine enabled 28.64 KB 0 B 0 B 28.66 KB ✅ lazy-page.js 0.04 KB
app: compiled floor (one template, one text hole, one delegated click) 10.03 KB 0 B 0 B 10.05 KB ✅
app: compiled CSR (JSX todo app: spread/merge/omit, events, class/style, keyed For, Show, Loading + lazy, store) 25.11 KB 0 B 0 B 25.13 KB ✅ stats.js 0.18 KB
app: compiled hydrating (the same JSX todo app through hydrate(), compiled hydratable) 30.92 KB 0 B 0 B 30.93 KB ✅ stats.js 0.20 KB
frames: eager client consumer (frames client + transport, lazy codec) 13.77 KB 0 B 0 B 13.78 KB ✅
page: base server components (hydrating + dynamic + frames + sf reference) 44.80 KB 0 B 0 B 44.84 KB ✅ decode.js 6.07 KB, lazy-page.js 0.04 KB
page: live server components (base + live/GET + action + isPending/latest) 48.49 KB 0 B 0 B 48.51 KB ✅ decode.js 6.07 KB, lazy-page.js 0.04 KB
server: floor (getRequestEvent + isServer) 1.33 KB 0 B 0 B 1.34 KB ✅
server: renderToString (the server-render floor) 20.41 KB 0 B 0 B 20.42 KB ✅

Bundled with Rolldown (what Vite ships), brotli q11, decimal KB. A scenario fails only when it is over its brotli cap and grew more than 20 B minified over the base; over the cap within that allowance is brotli layout noise and passes with a warning. Caps in scripts/size/scenarios.js; the floor and page caps in floor-caps.json are frozen (lower only, or Size-Exception: in the PR body). Caps are re-based downward by npm run ratchet (scripts/size/README.md).

@coveralls

Copy link
Copy Markdown

Coverage Report for CI Build 37422687576

Warning

Build has drifted: This PR's base is out of sync with its target branch, so coverage data may include unrelated changes.
Quick fix: rebase this PR. Learn more →

Warning

No base build found for commit 79df376 on next.
Coverage changes can't be calculated without a base build.
If a base build is processing, this comment will update automatically when it completes.

Coverage: 75.991%

Details

  • Patch coverage: No coverable lines changed in this PR.

Uncovered Changes

No uncovered changes found.

Coverage Regressions

Requires a base build to compare against. How to fix this →


Coverage Stats

Coverage Status
Relevant Lines: 1195
Covered Lines: 962
Line Coverage: 80.5%
Relevant Branches: 925
Covered Branches: 649
Branch Coverage: 70.16%
Branches in Coverage %: Yes
Coverage Strength: 27.95 hits per line

💛 - Coveralls

@codspeed

codspeed Bot commented Oct 6, 2026

Copy link
Copy Markdown

Merging this PR will improve performance by 8.74%

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 1 improved benchmark
✅ 187 untouched benchmarks

Performance Changes

Benchmark BASE HEAD Efficiency
⚡ memo + sync render effect only (reference) 29.4 ms 27.1 ms +8.74%

Tip

Curious why performance improved? Comment @codspeedbot explain why performance improved on this PR, or directly use the CodSpeed MCP with your agent.


Comparing size-gate-minified-allowance (b651365) with next (79df376)

Open in CodSpeed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants