Repository navigation
Conversation
The LLVM backend can emit calls to scr_worker_broadcast_channel (new BroadcastChannel), so llvm-runtime-abi.test.ts requires a prototype in a header scr_runtime.h includes; it now sits next to scr_worker_message_channel in scr_bytes.h.
The thread record keeps the Worker's name as an immortal string and frees it with the record. worker.threadName and the in-thread threadName returned that buffer as a string value, whose retains do not extend an immortal string's lifetime, so a name read before the Worker was released pointed at freed memory once the record was destroyed. Both getters now copy the name into a new string. Corpus 5992 keeps worker.threadName past the worker's exit and the Worker's release and then uses it; it matches Node, plain and with SCRIPTC_SAN=1, and fails under ASan without this change.
An async function called inside LifetimeRegion.run now keeps its region after every await: its fiber records the region that was current at spawn, and the event loop makes that region current while it resumes the fiber. A region is not freed (or attempted) while a fiber that carries it exists, so a region released while its async work is pending keeps taking that work's objects and is freed once the work ends. Before, objects created after an await went to the default region, which is never freed, and kept the request's region allocated for good, so per-request regions in async servers grew without bound. Freed region chunks now go to a per-thread cache (at most 8 MiB resident, the first 256 KiB of each kept cleared, fresh pages past it, no huge-page chunks) instead of being unmapped, so a region per request or per batch no longer maps and faults in new memory every time. Adds the requests.ts region benchmark (concurrent async handlers), corpus 5023 and its region-stats checks, and a chunk-reuse runtime unit check. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…storage
Frame storage for non-escaping records was limited to number, boolean and
inline-union fields. Records with string, array, record or class fields,
tuples, and same-shape spread updates (`{ ...state, count }`) were always
heap-allocated, even when every use only projected them.
Closed record and tuple shapes now qualify regardless of field types. The
frame owns the reference fields: they start empty in entry-addressed slots
and are registered before anything is stored, with the binding's lexical
scope for locals and with the statement frame for call-argument
temporaries, so an exception while the record is being built releases
exactly what was stored. Interned string literal fields need no owner.
A reassigned binding builds the new value apart, copies it over its slot
and then releases the replaced fields. Result-slot entries cover producers
returning such records, tuples and spreads of a binding, and store into
the zeroed caller storage without releasing an old value. A spread of a
binding is a projection for the lifetime analysis. Published shapes and
switch-case bindings of reference-owning shapes keep heap storage.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
scr_async.c reads scr_region_fiber_hooks, but several runtime test links (cipher, bytes, sha-kernels, runtime) take scr_async.c without scr_region.c and failed with an undefined reference. scr_object.c is in every runtime link, so the pointer now lives there; scr_region.c still installs the hooks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A for-of over a tuple producer's result only reads its elements, so the tuple now lives in frame storage and the producer runs through its result-slot entry. The test still checks that the receiver is computed once and that no array snapshot is allocated, whichever entry runs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… write When a write with a computed string key can replace any method, calls of every method name read the receiver's property bag without checking for the key, and the bag falls back to the class's prototype data. Generic class instantiations, mixin classes and their subclasses have no prototype data, so their methods read as undefined and calls threw "x.status is not a function". Reads on these classes now check for an own property and otherwise use the declared method. Fixes #963 Co-authored-by: Quim T. <26749475+espetro@users.noreply.github.com>
- URLSearchParams iterators compile outside a for-of head - URLSearchParams can be built from [string, string] tuple pairs - querystring.stringify writes bigint values - StringDecoder accepts an encoding known only at run time - os.setPriority is lowered
- util.format %d and %i format bigint, symbol, null and undefined arguments - util.format %j prints [Circular] for circular arguments - util.types answers the remaining brand checks - util.inspect prints WeakMap, WeakSet, Date and numeric typed array values as Node does - util.inspect lowers the breakLength, compact and sorted options
The worker series adds 16 corpus programs without measured costs in tests/harness/corpus-costs.json, which dropped the measured share below shard.test.ts's 0.9. Their plain and san costs are local timings of each program's differential and llvm-differential cases, scaled by the median ratio between the recorded and local timings of four programs timed in the same run (001-hello, 1000-json-stringify-basics, 5022-lifetime-region-workers, 5164-worker-field-callback-receiver). 5805 read its second message right after the first arrived; under load the parent's second postMessage could still be in flight, so the read answered undefined. The worker now waits on a shared flag that the parent sets after posting its first three messages. The output is unchanged.
The shard balance test reads measured costs for corpus programs. Add plain and sanitized costs for 5950, 5951, 5955 and 5956, timed locally and scaled to the CI runners' existing entries.
The --layouts padding object was linked directly before the program object. The runtime is linked before the program object, so every layout moved only the program's code and left each runtime function at the same address. A change to the runtime's code size then moved its functions in all layouts at once, and the verdicts could not tell that placement change from a real one. The padding object now goes before every other input on the link line, so each layout shifts the runtime and the program code.
Three analyses did work proportional to the whole function once per captured binding, so a factory with thousands of bindings shared by its methods spent most of its compile time in them (48 s of LLVM emission at 8,000 bindings): - LazyCaptures walked the body again for every capture it checked and rebuilt a locals map for every capture of every creation site. Each function's rebound bindings are now collected in one walk and the unchanged-parameter sets are memoized. - int32-slots walked the declaring function once per box to find its initializer and scanned capture lists per member. Both are now per-function sets built on first use. - CaptureFlow scanned all capture edges, capture lists and entry copies per binding and searched statement lists per hoisted or TDZ binding. Edges are grouped by source, captures, parameters and entry copies are indexed, and hoisted and TDZ initializers are found in one pass. The results are unchanged: the candidate compiler emits byte-identical LLVM for the self-hosted compiler's source. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every box pointer of a captured binding is a value live from its binding to its last use. A factory whose methods share thousands of bindings hands LLVM thousands of such ranges, each crossing every call in between, in the factory and in each method that receives them, and the inliner, instruction selection and register allocation grow quadratically: Bun's large-environment AOT program (8,701 captured bindings) spent about three minutes in native code generation. A function holding at least 512 capture boxes (the self-hosted compiler's largest holds under 100) is now emitted `noinline optnone`, and its box slots are loaded and stored volatile so no IR pass promotes them, which the native helper's pipeline would otherwise still do for optnone bodies. Each use reloads its box pointer from the frame. Boxes, environments and semantics are unchanged; only those bodies' machine code is unoptimized. large-environment.ts now compiles in 16 s instead of 227 s (peak RSS 543 MB instead of 777 MB) and runs in the same 0.18 s. Corpus 5973 pins such a factory's behavior (shared mutation, per-iteration closures, a nested closure, a forward-captured read in its TDZ), and a harness test bounds compile-time growth between 500 and 4,000 bindings. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…write observedPrototypeMethod skipped classPrototypeData for function-local classes, which also marks such a class as needing prototype data, so its instances got no prototype behind their property bag. A function-local class value's prototype holds only methods the program replaced there, so reads of a marked method name on its instances now check for the property, own or inherited, and otherwise use the declared method, as for generic class instances. The prototype data is built again, so a method replaced on the class's prototype stays visible.
The corpus programs 5810 to 5814 had no measured cost, which put the measured share of the corpus below shard.test.ts's 0.9 floor. Their plain and SCRIPTC_SAN=1 costs are cold local timings scaled to the CI runner by programs with recorded costs: 1440, 1441, 1442 and 1463, and 5133 for 5814's sanitized run, which links the AbortSignal runtime.
…JSON requires
A top-level require binding that earlier code could reach was refused
(SC1013). The reachability scan now follows Node's evaluation order:
hoisted functions and classes run only when reachable code names them, a
class statement runs only its heritage, computed keys, decorators and
static initializers, timer and promise callbacks run after the module
body, and publishing a value through module.exports, exports.n,
Object.defineProperty(exports, ...) or F.prototype.m runs nothing until
reachable code reads it back. This admits the export-first layouts of
semver, glob@7, fs.realpath, on-finished, readable-stream and
tsc-emitted CommonJS.
A const or let binding that code can still reach before its require now
compiles with an initialization flag set right after the require and a
module.tdzCheck before the outermost expression of every reference that
can run earlier, so an early read throws Node's ReferenceError before
any argument evaluates. A reference whose lowering leaves no expression
to check, a var binding (Node reads undefined there) and createRequire
bindings keep SC1013.
require() of a JSON document now answers one parsed object per document,
created in the requiring module's %init prelude unless another module
already created it, so repeated requires return the same object and
mutations are visible to every requirer. Require bindings of JSON
documents are ordinary storage, so destructuring (nested patterns,
defaults, missing keys) copies at the require as in Node.
Nested destructuring of a builtin require binds the members of the
sub-module the key names (const { types: { isPromise } } =
require('util')).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each instantiation of a generic class names the generic class as its base, and the generic class has no constructor function. The cycle analysis treated a base without a constructor function as one whose constructor let `this` escape, so every store in an instantiation's constructor counted as a write after construction. A generic class could not be declared Acyclic, and its constructor-only references were never treated as immutable. A program class without a constructor function runs no code at construction, so it now lets nothing escape. Runtime classes keep the conservative answer.
new DisposableStack() compiles to a native stack handle. use, adopt, defer, move, dispose, disposed and [Symbol.dispose] are members of its dynamic view, and the stack works with `using`. Disposal runs the resources in reverse order, calls use() resources' dispose methods with the resource as `this`, and raises a SuppressedError chain when more than one disposal throws; a single failure is rethrown as thrown. Calls on a disposed stack throw Node's ReferenceError, and invalid receivers, resources and callbacks throw Node's TypeErrors. use() of a class instance registers a function that calls the class's [Symbol.dispose] method, and returns the instance at its static type; use() of a union that includes class instances is refused. The stack is collected with its callbacks when a callback captures the stack. x instanceof DisposableStack compiles. Handles can expose symbol-keyed members through a new symbol_get hook.
`typeof zlib.Inflate` and `typeof zlib.createGzip` stopped at SC1090 or SC2020, because the operand lowered as a library function or class value. A typeof of a function or class exported by a Node builtin module, read through a plain binding, now folds to "function" as in Node: the read has no side effects to preserve.
Reads of defaultMaxListeners through an ES module namespace import or a named import answer the namespace's copy of the export, which keeps its value from program start until module.syncBuiltinESMExports() refreshes it, as in Node. The default import, the EventEmitter class and CommonJS module bindings keep reading the current default. A destructured CommonJS binding of defaultMaxListeners, which held a placeholder string in JavaScript files, is refused.
superCallStmt asserted the class base before checking for a callable base. A class that extends an ordinary function constructor has no base class info, so the assertion held null. Node ignores the unused value; the native compiler throws the TypeError of a non-null assertion that held null, and failed to compile programs with such classes (for example the Effect stream fixtures). The base is now read after that case returns.
pop and a discarded pop (the length - 1 store) branch on the array's length and hole-free prefix bound. When the bound equals the length, the last element is a present dense value: the emitted code clears its slot and state, lowers the bound and the length, and moves the slot out (or releases it for a discarded pop). An empty array answers undefined without a call. Holes, present undefined and sparse storage keep the runtime call.
Resolve the scr_resume_fiber conflict with stackless async coroutines (#1204): the region the fiber carries is made current before either resume path and restored after both. scr_async_spawn_stackless now captures the spawning region like the fiber spawns, so stackless bodies (every async body in 5013 and 5023 after the merge) keep allocating in their region after an await and keep it alive while they exist. 5024-lifetime-region-await-forms pins regions carried by stackless and fiber (for await) bodies that spawn and await each other. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Resolve the emitter.ts import conflict by keeping both RESULT_DST_ID (frame-stored result slots) and specializeCallbackCalls (#1206). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…er ASan The order-parity baseline for the new corpus program records its module order and empty preflight diagnostics. Uncached sanitized builds rebuild the runtime for every compile, which takes longer than either measured size, so the compile-time growth test runs in the plain lane only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A require cycle was always refused (SC1016). Node answers a require of a
module that is still evaluating with its exports as they stand, while the
compiled bindings alias the final declarations; the two agree when every
module of the cycle assigns all of its exports before its first statement
that can run another module's code, every other use of its export object
is a plain member read, and no module's top-level code can reach its
binding of another member. Each module then initializes exactly as it
would outside the cycle. This admits semver's comparator/range cycle
("hoisted class for cyclic dependency"); other cycles keep SC1016 with a
reason naming the offending statement.
The require reachability scan no longer treats a declaration's own name
as a read (esbuild's `var X = class {...}` before its requires), treats a
plain write of an export member as running only a setter published for it
(on-finished's `module.exports.isFinished = isFinished`), and keeps the
conservative scan for writes of __proto__, __esModule and members of a
replaced export object.
Evidence: corpus 6145 (the cycle), 6142 gains the esbuild layout; the
npm-static harness compiles the published fs.realpath@1.0.0,
on-finished@2.4.1 (with ee-first@1.1.1) and statuses@2.0.1 statically;
program.test.ts pins the TDZ analysis through var bindings plus the cycle
admission and refusals. Preflight/order baselines recorded for the new
fixtures; a full re-record changed no other fixture. Dependencies and
limitations docs describe the supported require forms.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The test asserts that the TDZ analysis batches symbol queries over the roots it scans, observed through an SC1013 refusal. Its const binding now compiles with initialization checks instead of a refusal, so the test uses a var binding, which keeps the fence and runs the same scan. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
exec(command, callback) and exec(command, { encoding, maxBuffer },
callback) lower like execFile's callback slice. The command line runs
with /bin/sh -c, as execSync runs it, the call returns the live
ChildProcess, and the error-first callback receives the captured utf8
stdout and stderr. A non-zero exit, including the shell's 127 for a
missing command, reports Node's "Command failed: <command>\n<stderr>":
scr_exec_file takes a shell flag so the message shows the command line.
Under @types/node, child_process.ExecException maps to Error, as
ExecFileException does. Its code member is the child's numeric exit
status, which the runtime error's string code slot does not hold, so
reading it, testing it with `in`, or testing its typeof is a named
refusal.
A constructor that let other code read a field before assigning it (a base constructor calling an override, or the constructor calling its own method) read an empty slot, which crashed the executable for string fields. Node reads undefined. Fields assigned after construction can expose the instance now admit undefined. Allocation writes undefined into them. String, number, boolean, bigint and union fields read as runtime-optional values, and class, array, record and collection fields use the deferred-init reads, whose member access throws a TypeError. Classes whose construction cannot expose the instance before their assignments keep their layouts.
…ed builtin destructuring, settled require cycles (#1287)
An object literal with a computed key that does not fold to a constant,
such as `{ [key]: 1 }`, entered the method replacement census as a write
with an unknown key and no receiver. That marked every class method name
in the program as replaceable, so method calls went through the checked
property lookup. A computed key in an object literal defines a property of
the new object, which is not a class instance, so the census now skips it.
Literals passed to Object.assign or Object.defineProperties are still
judged with their receiver.
Closes #962
Co-authored-by: Quim T. <26749475+espetro@users.noreply.github.com>
… semantics scriptc-pm overlays compile third-party TypeScript inside the consumer's program. Three properties of the package's own build were missing there, and each dropped the overlay back to published JS: - Ambient declarations. The manifest's new per-instance `ambientDeclarations` (additive, schema 1) lists the package's own ambient .d.ts files (typescript-estree's typings/typescript.d.ts). They join the checker's roots while the instance is active. - Untyped dependencies. A bare import from an overlay file that resolves only to a package's JavaScript (upstream's @types package is a devDependency) now selects that package, and the JavaScript packages it imports in turn, for compilation from published JS, typed by inference, as under the overlay package's own published JS. The scout ignores the checker's TS7016 at exactly those imports; a dependency that does not compile drops the importing overlay with that reason. - Class-field semantics. `semanticOptions.useDefineForClassFields: false` makes the overlay's classes assign parameter properties before running field initializers, the package's emitted order. TS2729 is suppressed only where an initializer reads a parameter property of its class. Initializer-less fields are admitted only where assign and define semantics agree (assigned before construction exposes the instance, in declaration order, after every initialized field); other shapes keep a named SC1090, so the overlay falls back. Without --sources none of this runs: programs compile byte-identically. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #1289 (
poc/spm-compiler, the--sourcescompiler side). Base branch for review:poc/spm-compiler.Summary
scriptc build --sourcescompiles an augmented package's upstream TypeScript inside the consumer's program. In the scriptc-pm proof of concept, 11 of 12 verified overlays fell back to published JS. Five of those fallbacks came from properties of the package's own build that the consumer's program does not reproduce:.d.tsfiles were not loaded (typescript-estree);noImplicitAny, because upstream's@typespackage is a devDependency that is never installed (fs.scandir, minimatch);useDefineForClassFields: false, so its field initializers read parameter properties that scriptc's define semantics had not assigned yet (fs.stat, fs.walk).This PR gives overlays those three properties. All of it is gated on overlay membership. Programs built without
--sourcescompile to byte-identical IR, LLVM and executables (zero-cost gate below).What changes
Manifest (schema 1, additive). Both fields are optional; when they are absent, behavior is unchanged.
instances[].ambientDeclarations: string[]lists declaration files relative tooverlayDir. They must be.d.tsfiles inside the overlay. If a listed file is missing, the instance is skipped with a note.instances[].semanticOptions.useDefineForClassFieldsis already written by scriptc-pm. The valuefalsenow selects assign semantics for that overlay's classes.Ambient declarations. While an instance is active, its ambient files join the checker's roots, so the overlay type-checks as it did in the package's own build.
Untyped dependencies. Some bare imports from an overlay file resolve only to a package's JavaScript. That package, and the JavaScript packages it imports in turn, are selected for compilation from published JS, typed by inference. This is how they compile under the overlay package's own published JS. The declaration-surface eligibility check of auto selection is bypassed for them; the compilation checks still apply.
The scout program ignores TS7016 only at exactly those imports. If such a dependency is refused, the importing overlay falls back with this reason:
its untyped dependency X does not compile from its published JS (…).Assign semantics (
useDefineForClassFields: false). These rules apply to classes in that overlay's files:Parameter-property assignments run before the field initializers, in tsc's ES2017 emit order. The construction-escape analysis already assumed that order.
TS2729 comes from the program's define-semantics checker. It is suppressed only where a field initializer reads
this.pandpis a parameter property of the enclosing class.Initializer-less fields are admitted only where the two semantics agree:
Other shapes keep a named SC1090, so the overlay falls back rather than diverging on property presence or order.
Evidence
Harness.
tests/harness/source-overlay-semantics.test.tshas one case per mechanism. Each case builds a small pnpm-shaped project with published JS and an overlay, and checks two things: without the mechanism, the overlay falls back with the expected reason; with it, the package compiles from source and stdout, stderr and exit status match Node, which runs the published JS. The case for assign semantics also checks the property order inconsole.logoutput and that the narrow refusal for a never-assigned optional field holds. All 3 cases fail on the parent commit and pass here. The unit test insource-overlays.test.tscovers manifest validation.spm demo (
examples/spm-demo/measure.sh, frompoc/spmmerged locally, Linux sandbox). scriptc-pm was patched locally to writeambientDeclarations(proposed forpoc/spmbelow). Coverage and build results with--sourcesare unchanged from the published-JS build for every probe, as in the POC.Per augmented package, before → after (first blocker in the overlay):
* In the typescript-estree probe, minimatch reports
its untyped dependency brace-expansion does not compile (SC1010 balanced-match). That probe still has type errors from other packages, so npm-static's last-resort probe drops every opted-in package, including balanced-match. The direct minimatch probe shows the package's own next layer.Overlays compiled from source: 1 of 12 before and 1 of 12 after, by the POC's measure (the overlay is kept). In both cases that is visitor-keys, which lowers no statements because its probe fails earlier. This PR removes every T13 blocker the demo hits (5 packages). None of the five reaches a full compile yet: two stop at generic lowering gaps, two at CommonJS interop for overlay sources (
import * as xof a CommonJS module, andexport =), and one at callback typing from an absent@typespackage. The demo's direct probes for the transitive packages are inresults/opt-overlay-t13/ext-setup.sh.Zero cost.
scripts/zero-cost-gate.mjs(fromopt/zero-cost-gate) compared the parent with this PR over the benchmark workloads, the self-hosted compiler and the TypeScript corpus. Result: PASS, 2347 of 2347 programs identical (IR, LLVM, release executables). Compile time changed by -0.61% (bench geomean, 95% CI -1.72% to +0.85%) and by -2.24% for the self-hosted compiler (1 pair), both within tolerance.Other checks.
tests/harness/source-overlays.test.ts,tests/harness/provenance.test.ts,packages/compiler/test/class-early-field-reads.test.ts, and the resolve and source-overlay unit tests.pnpm lintandpnpm format:checkare clean.pnpm test:ts7 --baselines-onlyis complete.--emit=irbuild ofpackages/compiler/src/native/cli.tshas no diagnostics.tests/harness/self-hosting-native-driver.test.tspasses on the final candidate.Considered and left out
strictNullChecks. RelaxingnoImplicitAnyfor overlay files would only turn fs.scandir's TS7006 into anany-typed value, which TypeScript lowering refuses.export =andimport x = require()for CommonJS-published overlays (DESIGN §5.2.6). fast-glob is the only demo package that needs it, and it is blocked again right after by runtimenamespacevalues and more CommonJS interop. That belongs with the "namespace import of a CommonJS module in an overlay" follow-up, which also unblocks fs.walk and typescript-estree.scriptc-pm side (proposed for
poc/spm)scriptc-pm writes
ambientDeclarationsfrom the.d.tsfiles that the package's tsconfigincludecovers (73-line patch:results/opt-overlay-t13/spm-ambient.patch). It is not part of this PR.🤖 Generated with Claude Code