Skip to content

Compile classic CommonJS packages: late requires, JSON requires, nested builtin destructuring, settled require cycles - #1287

Merged
ctate merged 4 commits into
mainfrom
opt/cjs-interop
Oct 18, 2026
Merged

ctate merged 4 commits into
mainfrom
opt/cjs-interop

Conversation

@cramforce

Copy link
Copy Markdown
Contributor

Summary

Compiling a real Node service, and comparing against Bun's AOT compiler, showed that most refused npm packages were refused by four CommonJS-interop fences rather than by the code they run. This PR replaces those fences with Node's semantics where they can be modeled exactly, and keeps a named refusal everywhere else.

1. require() after code that could use its binding (SC1013). The reachability scan now follows Node's evaluation order instead of treating every earlier statement as running every function it names:

  • hoisted functions and classes run only when reachable code names them;
  • a class statement runs only its heritage, computed keys, decorators and static initializers;
  • timer, microtask and promise callbacks run after the module body;
  • publishing a value (module.exports = f, exports.n = f, Object.defineProperty(exports, "n", { get }), F.prototype.m = g) runs nothing until reachable code reads it back.

Nothing outside a module can read its exports before its body completes. Require cycles are fenced unless they meet the conditions in item 4, and a compiled graph has no reflective exports object.

This admits the export-first layouts of semver, glob@7, fs.realpath, on-finished, readable-stream, esbuild bundles and tsc-emitted CommonJS.

When code can still reach a const/let binding before its require, the require now compiles with an initialization flag (require-initialization.ts). The flag is set right after the require evaluates the module. A module.tdzCheck runs before the outermost expression of every reference that can run earlier. An early read therefore throws Node's ReferenceError: Cannot access 'x' before initialization before any argument evaluates. These cases keep SC1013:

  • a reference whose lowering leaves no expression to check (for example, exporting the binding before its require);
  • a var binding, where Node reads undefined;
  • a createRequire binding.

2. JSON require() (SC1012). Each JSON document becomes one object, parsed at build time (json-require.ts). Each requiring module creates it in its %init prelude unless another module already did. Every require, from any module and at any position (including lazy and inline require('./package.json').version), returns that same object, so mutations are visible to every requirer. A JSON require binding is ordinary storage, so destructuring copies at the require: nested patterns, defaults and missing keys behave as in Node. ES-module JSON imports keep their own record, as Node's ESM loader does.

3. Nested destructuring of a builtin require (SC1012). const { types: { isPromise } } = require('util') binds the members of the sub-module its key names (util.types is util/types). Defaults, rest elements and nested patterns over program modules keep SC1012, with a clearer message.

4. Require cycles (SC1016), narrowly. Node gives a module that requires a still-evaluating module that module's exports as they stand. The compiled bindings alias the final declarations. The two agree when all three conditions hold:

  • every module of the cycle assigns all of its exports before its first statement that can run another module's code;
  • every other use of its export object is a plain member read;
  • no module's top-level code can reach its binding of another member.

Each module then initializes exactly as it would outside the cycle. This admits semver's comparator/range cycle ("hoisted class for cyclic dependency"). Other cycles keep SC1016, with the offending statement named in the reason.

Package matrix (Linux sandbox, Node 24.15; the program imports the package and exercises its main API)

Package Before (e72ee03) After
semver 7.7.3 refused: SC1013 parseOptions compiles; runtime fence at exports = module.exports = {} in internal/re.js (assignment to the exports parameter)
glob 7.2.3 refused: SC1013 (glob, fs.realpath) CommonJS fences pass; SC1090 sloppy-mode parameterized arguments in sync.js
minimatch 3.1.2 no CommonJS fence; SC1090 constructing the ES5 Minimatch function unchanged (the service's SC1013 came from another 3.x release)
readable-stream 3.6.2 refused: SC1013 Stream CommonJS fences pass; SC1090 constructing ES5 constructor functions
fs.realpath 1.0.0 refused: SC1013 old compiles; runtime fence: fs.realpathSync stored as a value
on-finished 2.4.1 refused: SC1013 first compiles; runtime TypeError "a 'string' value is not representable in the target union"
@openfeature/server-sdk 1.19.0 (CJS) refused: SC1013 import_core6 compiles; runtime fence: class extends import_core2.OpenFeatureError (computed heritage)
@typescript-eslint/typescript-estree 8.46.1 refused: SC1013 in tsconfig-utils, project-service, typescript-estree, semver CommonJS fences pass; compiler crash (uncaught PoisonError in collectClassShapeInner via mapType)
undici 5.29.0 refused: SC1013 constants, then SC1012 nested destructuring SC1010 stream/web
dd-trace 5.73.0 refused: SC1012 JSON require JSON requires compile; SC1010 v8 (by design), native addon @datadog/native-appsec, destructuring defaults in crypto-randomuuid
tr46 0.0.3 (node-fetch 2.7.0) SC1010 punycode first; require("./lib/mappingTable.json") itself refused with SC1012 punycode still first; the JSON require is accepted but mappingTable.json ((string | number[])[][]) has no array representation (SC2009)

None of the ten runs end to end yet. Every one that was refused for CommonJS interop now gets past those fences, and the remaining blockers are listed above as findings, not fixed here.

Evidence

  • Corpus (Node vs native, byte-identical):
    • 6140-cjs-require-tdz-throws: uncaught TDZ, @exit: 1.
    • 6141-cjs-require-tdz-checked: call, call with an argument side effect, member, new, builtin, value and renamed reads before and after the require, plus stored callbacks run later.
    • 6142-cjs-require-published-early: semver, glob and esbuild layouts.
    • 6143-cjs-require-json: identity across modules, mutation visibility, lazy and inline requires, destructuring snapshots, array documents.
    • 6144-cjs-require-builtin-nested.
    • 6145-cjs-require-cycle: semver-shaped cycle, evaluation order.
  • Diagnostics:
    • cjs-require-tdz is now a var binding (still SC1013).
    • New cjs-require-tdz-unchecked: a const read with no checkable expression.
    • cjs-require-cycle names its reason.
  • program.test.ts:
    • The existing early-read cases now run through var bindings, so the analysis is still pinned.
    • New: code that cannot run early (class methods, stored class expressions, later-job callbacks, published getters, properties stored on a published function); the const case compiling; cycle admission plus four refusals.
  • npm-static harness: the published fs.realpath 1.0.0, on-finished 2.4.1 with ee-first 1.1.1, and statuses 2.0.1 (requires codes.json) are vendored and compile statically.
  • Preflight/order baselines recorded. A full SCRIPTC_UPDATE_BASELINES=1 re-record changed no fixture other than the new and intentionally changed ones.

Validation

All runs were in the Linux sandbox:

  • pnpm lint, pnpm format:check and pnpm test:ts7 --baselines-only pass.
  • Self-hosting build packages/compiler/src/native/cli.ts --emit=ir produces no diagnostics.
  • tests/harness/self-hosting-native-driver.test.ts passes (28 min). It ran on the candidate before the last commit's small reachability narrowing (export-member writes).
  • Focused suites:
    • diagnostics, npm-static, npm, library-mode and the frontend unit tests: 1194/1197. The three failures:
      • library-mode K10 (ASan, known baseline-only);
      • node24-url-authority (Node 24.15 vs the pinned 24.22);
      • the new npm-static test, since fixed in the last commit. It passes on the final candidate together with program.test.ts, the new corpus programs and the CommonJS diagnostics.
    • coverage.test.ts: all pass except "every corpus program is 100% static", which hits its 600 s timeout in the sandbox. scriptc coverage reports each new corpus program at 100% static.
  • Order parity: the default-lane order-parity.test.ts passes on the final candidate. A full SCRIPTC_UPDATE_BASELINES=1 re-record changed only the new and intentionally changed fixtures.
  • Differential corpus subset (cjs|require|cycle|tdz|commonjs|npm|json, ~380 programs): the only failures are 18 programs that fail identically on the e72ee03 baseline in this sandbox (Node 24.15 vs the pinned 24.22).
  • docs: NEXT_DIST_DIR=.next-check pnpm check passes (25/25 route tests).
  • pnpm node-compat:check reports the snapshot is current. No compiler decision tables changed, so pnpm manifest and pnpm node-compat were not needed.

🤖 Generated with Claude Code

cramforce and others added 2 commits October 17, 2026 19:58
…JSON requires

A top-level require binding that earlier code could reach was refused
(SC1013). The reachability scan now follows Node's evaluation order:
hoisted functions and classes run only when reachable code names them, a
class statement runs only its heritage, computed keys, decorators and
static initializers, timer and promise callbacks run after the module
body, and publishing a value through module.exports, exports.n,
Object.defineProperty(exports, ...) or F.prototype.m runs nothing until
reachable code reads it back. This admits the export-first layouts of
semver, glob@7, fs.realpath, on-finished, readable-stream and
tsc-emitted CommonJS.

A const or let binding that code can still reach before its require now
compiles with an initialization flag set right after the require and a
module.tdzCheck before the outermost expression of every reference that
can run earlier, so an early read throws Node's ReferenceError before
any argument evaluates. A reference whose lowering leaves no expression
to check, a var binding (Node reads undefined there) and createRequire
bindings keep SC1013.

require() of a JSON document now answers one parsed object per document,
created in the requiring module's %init prelude unless another module
already created it, so repeated requires return the same object and
mutations are visible to every requirer. Require bindings of JSON
documents are ordinary storage, so destructuring (nested patterns,
defaults, missing keys) copies at the require as in Node.

Nested destructuring of a builtin require binds the members of the
sub-module the key names (const { types: { isPromise } } =
require('util')).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A require cycle was always refused (SC1016). Node answers a require of a
module that is still evaluating with its exports as they stand, while the
compiled bindings alias the final declarations; the two agree when every
module of the cycle assigns all of its exports before its first statement
that can run another module's code, every other use of its export object
is a plain member read, and no module's top-level code can reach its
binding of another member. Each module then initializes exactly as it
would outside the cycle. This admits semver's comparator/range cycle
("hoisted class for cyclic dependency"); other cycles keep SC1016 with a
reason naming the offending statement.

The require reachability scan no longer treats a declaration's own name
as a read (esbuild's `var X = class {...}` before its requires), treats a
plain write of an export member as running only a setter published for it
(on-finished's `module.exports.isFinished = isFinished`), and keeps the
conservative scan for writes of __proto__, __esModule and members of a
replaced export object.

Evidence: corpus 6145 (the cycle), 6142 gains the esbuild layout; the
npm-static harness compiles the published fs.realpath@1.0.0,
on-finished@2.4.1 (with ee-first@1.1.1) and statuses@2.0.1 statically;
program.test.ts pins the TDZ analysis through var bindings plus the cycle
admission and refusals. Preflight/order baselines recorded for the new
fixtures; a full re-record changed no other fixture. Dependencies and
limitations docs describe the supported require forms.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vercel

vercel Bot commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
scriptc Ready Ready Preview, v0 Oct 18, 2026 1:57pm UTC

cramforce and others added 2 commits October 18, 2026 06:46
The test asserts that the TDZ analysis batches symbol queries over the
roots it scans, observed through an SC1013 refusal. Its const binding now
compiles with initialization checks instead of a refusal, so the test
uses a var binding, which keeps the fence and runs the same scan.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch was successfully deployed

1 active deployment
Preview — 04297f30 Deployed Oct 18, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants