@ladamczyk/react-compiler-profiler

Empirically measure whether React Compiler is actually helping your app — instead of enabling it and hoping.

It builds your app twice (compiler on, compiler off), replays the same Playwright scenarios against both, and reports the difference in render counts and paint/load timings per scenario — plus which components your scenarios never touched.

✔ dashboard-tick: renders -20.0%, paint -4ms
· settings: renders +0.0%, paint +0ms

Coverage: 4/4 components measured.
1 scenario(s) flagged.

Install

npm i -D @ladamczyk/react-compiler-profiler

The tool resolves a few packages from your project rather than bundling them, so it always profiles against your app's own versions. Four are declared as ordinary peer dependencies, so npm installs them with this package and keeps whichever versions you already had:

@vitejs/plugin-react  ^5 || ^6      vite  ^7 || ^8
playwright            ^1           babel-plugin-react-compiler  ^1.0.0

One is not declared, and you install it yourself — but only if you are on @vitejs/plugin-react v6:

npm i -D @rolldown/plugin-babel@"^0.2.0"

It is left out because declaring it would pull its own vite: ^8 preference into every install and make this package uninstallable alongside Vite 7. Preflight fails fast with that exact command if you are on v6 without it.

Both @vitejs/plugin-react majors work, and the plugin picks its pipeline from whichever one you have installed — v6 (Vite 8) transforms with oxc, so the compiler runs as a Babel preset through @rolldown/plugin-babel; v5 (Vite 7 or 8) still has a Babel pipeline of its own, so the compiler goes straight into it and @rolldown/plugin-babel is not needed.

@babel/core isn't in either list on purpose: @rolldown/plugin-babel requires it as a peer of its own and @vitejs/plugin-react v5 depends on it outright, so it arrives with whichever of those you install — this tool neither owns that constraint nor re-checks it.

Preflight re-checks every one of them from your project root before either build runs, and never installs anything itself — so a package npm resolved to an unsupported version, or one that a workspace layout hid from this package, is reported as a command to run rather than as a stack trace mid-build.

Scenarios replay in headless Chromium, so Playwright needs its browser binary once. Only the headless shell is used — skip the other four downloads:

npx playwright install chromium-headless-shell

Quickstart

1. Add the toggle plugin to your Vite config, after react():

// vite.config.ts
import { defineConfig } from 'vite';
import react from '@vitejs/plugin-react';
import { reactCompilerToggle } from '@ladamczyk/react-compiler-profiler/vite';

export default defineConfig({
  plugins: [react(), reactCompilerToggle()],
});

reactCompilerToggle() reads the REACT_COMPILER_ENABLED env var the profiler sets per build pass, so this one unchanged config produces both halves of the comparison. On the compiler-off pass it is a named no-op that still shows up in Vite's plugin list — so "was the toggle actually wired in?" stays answerable.

2. Write a profiler.config.ts at your project root (.ts, .js, .mjs, or .cjs — auto-discovered):

export default {
  scenarios: [
    {
      name: 'dashboard-tick',
      route: '#/dashboard',
      steps: [{ action: 'click', selector: '[data-testid="tick-button"]' }],
    },
  ],
};

3. Run it:

npx react-compiler-profiler

Config reference

Field	Type	Default	Meaning
scenarios	IScenario[] (required)	—	The flows to profile. At least one.
thresholds	Partial<IThresholds>	{ renderCountDeltaPct: 5, paintTimeDeltaMs: 50 }	Deltas at or beyond these flag a scenario.
cooldown	Partial<ICooldown>	{ betweenScenariosMs: 1000, betweenComparisonRunsMs: 5000 }	Settle times so both builds face comparable conditions.
runs	number	3	Repetitions per build, collapsed by aggregate.
aggregate	'mean' | 'median' | 'trimmed'	'median'	How repetitions are collapsed into one number.
outDir	string	.react-compiler-profiler	Where report.json and report.html are written.
root	string	cwd	Vite root of the app under test, if not the cwd.

Each scenario has a name, a route, and ordered steps. There is no custom DSL — the five step actions each map to one literal Playwright call:

action	maps to
goto	page.goto(url)
click	page.click(selector)
fill	page.fill(selector, value)
select	page.selectOption(sel, value)
wait	page.waitForSelector(sel) (or waitForLoadState() with no selector)

CLI options

react-compiler-profiler [options]

  --runs <n>              Repetitions per build (default 3)
  --aggregate <strategy>  How repetitions are collapsed: mean | median | trimmed (default median)
  --out <dir>             Directory to write report.json and report.html to
  --json                  Output the machine-readable report on stdout
  -h, --help              Show help

CLI flags override the config file.

Exit codes

Code	Meaning
0	No scenario crossed a threshold and none failed.
1	At least one scenario crossed a threshold — in either direction — or failed.
2	Usage or config error (bad flag, invalid or missing profiler.config).

A large improvement flags too. A "flagged" scenario means the compiler made a meaningful difference, good or bad — so a big win exits 1, same as a regression. The point is to draw your eye to what moved, not to pass/fail a build. The human output colours improvements green (✔) and regressions red (✖); read the direction there, not the exit code alone.

Browsing the results

Every run writes two files into outDir (default .react-compiler-profiler/):

• report.html — a single self-contained file: inline CSS, hand-rolled inline-SVG charts, the raw report embedded as JSON. No server, no assets, no vendor JS, no network. Just open it:

  xdg-open .react-compiler-profiler/report.html   # or double-click it

  You can move or email that one file and it renders identically from file://.

• report.json — the source of truth, for tooling and LLMs. Per scenario: both builds' metrics, the delta, and whether it flagged. Plus a coverage block naming the components your scenarios exercised and the ones they never rendered.

Interpreting results — this is a local tool, not a CI gate

Timing metrics are only as stable as the machine producing them. Treat this as a local, on-demand tool run on a quiet machine — not a CI performance gate, where shared runners violate the determinism this depends on.

For trustworthy numbers:

• Run on an idle machine — close other heavy apps; disconnect from power-saving throttling if you can.
• Keep the default cooldowns (or raise them). They let the machine settle so the two builds face comparable conditions.
• Render counts are deterministic and exact — the core signal. Paint/load timings are inherently noisier; the aggregation (median by default) exists to blunt that.
• More --runs means a steadier verdict. The default of 3 keeps a run quick but makes the variance estimate itself coarse — enough to catch a wildly unstable scenario, not to certify a stable one. Bump --runs when a number looks marginal.

When a scenario's run-to-run spread is high enough to make its verdict unreliable, the tool prints an advisory ⚠ warning naming that scenario. It is advisory only: it never changes the pass/fail verdict or the exit code — a noisy machine is a fact about the run, not a finding about the compiler.

Programmatic API

import { profile, format } from '@ladamczyk/react-compiler-profiler';

const result = await profile({ runs: 5, aggregate: 'trimmed' });
// result.passed  · result.report  · result.outDir  · result.warnings
console.log(format(result)); // the same string the CLI prints

profile() returns a structured result and never prints or exits — rendering is format()'s job, exit codes are the CLI's.

Development

Note: this project was "vibe coded" — designed and implemented in collaboration with Claude Code, Anthropic's agentic CLI. The whole codebase, tests, and docs were produced that way.

See CLAUDE.md for architecture and conventions, and PRODUCT.md for what the tool does and why.

npm run build          # src/ → bin/ + lib/
npm run dev            # build then run the CLI
npm test               # vitest run
npm run qoq:check      # ESLint + Prettier + knip + jscpd