Skip to content

Give constructor functions native stream state over the stream classes - #1533

Merged
ctate merged 6 commits into
mainfrom
opt/stream-ctor-bases
Oct 20, 2026
Merged

ctate merged 6 commits into
mainfrom
opt/stream-ctor-bases

Conversation

@cramforce

Copy link
Copy Markdown
Contributor

Follows #1512, which added the untyped EventEmitter and stream constructors.

Summary

Bundled npm code builds its streams as ES5 constructor functions:

function Source(opts) {
  if (!(this instanceof Source)) return new Source(opts);
  Readable.call(this, opts);
}
util.inherits(Source, Readable);
Source.prototype._read = function (size) { this.push(...); };

from2, bl, tar-stream, yazl and multistream all do this (Readable.call, Writable.call, Duplex.call, Transform.call, PassThrough.call). #1512 made the stream classes untyped constructor values with Node's prototype chain, but calling one with a constructor function's receiver threw SC2020.

Native stream state on the receiver. A new runtime unit, scr_dyn_builtin_stream.c, gives those constructors their stream half. Readable.call(this, opts) (and the other four) creates a runtime stream from scr_stream.c with the same option walk new Readable(untypedOptions) uses, and keeps it in a hidden slot of the receiver: an entry of the object's symbol table under a storage name no symbol produces, outside the key list, so reflection does not see it. As in Node, the constructor copies the option callbacks to _read, _write, _final, _destroy, _transform and _flush, sets allowHalfOpen on the duplex kinds, and adds _readableState / _writableState (read-only live views of the runtime state: highWaterMark, length, ended, endEmitted, flowing, destroyed, finished, needDrain and the other fields the runtime keeps). Listeners added before the constructor ran (bl adds its pipe listener first) are kept.

Hooks. The runtime stream calls the receiver's _read, _write, _transform, _flush, _final and _destroy through its property lookup with the receiver as this, and with Node's argument shapes and error handling: a throwing _read, _final, _flush or _destroy reaches the stream the way Node's try/catch hands it on; a throwing _write or _transform propagates out of write(). Completion callbacks keep the JavaScript error object, so 'error' listeners receive the same object with its code.

Prototype methods. The stream prototypes carry Node's methods and accessors over the hidden stream: push, unshift, read, setEncoding, pause, resume, isPaused, pipe, unpipe, destroy (with its callback), write, end, cork, uncork, the default _read/_write/_transform/_destroy, PassThrough's _transform, and readable, readableFlowing, readableEnded, writableNeedDrain, destroyed (with Node's setter), closed, errored and the other getters. EventEmitter's methods (from #1512) hand a receiver with stream state to this unit, which registers listeners with the runtime stream's own emitter, so the runtime's events and pipes reach JavaScript listeners with the receiver as this; events with other names carry every argument (tar-stream's emit('entry', header, stream, next)).

Typed code and runtime streams. In programs that link the unit, a checked cast of such an object to Readable, Writable, Duplex, Transform or PassThrough takes its runtime stream, a runtime stream that crosses into JavaScript shows as its constructor object, and the untyped views of other runtime streams pipe into constructor-function streams. stream.unboxDyn (#1518's untyped pipe destinations and pipeline stages) resolves to scr_dyn_builtin_stream_unbox in those programs, which takes constructor-function streams and otherwise behaves as scr_stream_unbox_dyn, so fs.createReadStream(file).pipe(ctorStream) and pipelines over them work. The backend picks these entry points only when the program links the unit; other programs keep their code.

Readable, Writable, Duplex, Transform and PassThrough values now lower to dyn.builtinStreamClass (EventEmitter and the legacy Stream keep dyn.builtinClass), which links the new dynBuiltinStream feature. Programs that only use EventEmitter or Stream as a base do not link the stream half. scr_stream.c gains one function, the destroyed setter, which no existing program references.

Not supported

These throw the not-supported Error where they run, and are listed on the limitations page:

  • the options the runtime streams refuse for untyped options (objectMode, decodeStrings: false, signal, construct, writev, a non-UTF-8 defaultEncoding);
  • _writev and _construct methods, symbol event names, listeners() / rawListeners(), wrap(), and strings written in encodings other than UTF-8;
  • pipes into a constructor-function stream that overrides write() or end() (bl's callback form): the runtime pipe writes the destination's stream directly, where Node calls the overriding methods.

Two existing divergences show up with these streams and are unchanged: a plain call made inside a listener or method keeps the caller's this (so Ctor() without new inside a listener of another Ctor instance runs the constructor on that instance), and the runtime pipe does not emit 'unpipe' on the destination when the source ends.

Interplay with #1524

#1524 (NodeJS.ReadableStream / WritableStream as checked-dynamic slots) dispatches member calls on non-native values through dynInvoke and pipes into them with stream.pipeDyn. Constructor-function streams are JavaScript objects there and work through their prototype methods. I verified this on a merge of an earlier state of this branch into an integration tree that contains #1518 and #1524: the new corpus programs, #1524's 6480-6482, and 6453/6454/5977 pass. The conflicts are mechanical: both changes add a runtime feature to the same tables.

Evidence

  • 6560-constructor-function-readable.cjs: the from2 patterns (call-without-new guard, destroyed set before the base constructor, _readableState.highWaterMark, callback-driven _read with asynchronous pushes, its own destroy), paused mode with 'readable' and read(), setEncoding, read(n) and unshift, an error from the source.

  • 6561-constructor-function-writable.cjs: a Writable with _write, _final, backpressure, 'drain', write and end callbacks and the writable getters; a bl-style Duplex with pre-constructor pipe/unpipe listeners, an end() override calling Duplex.prototype.end, and reading back; piping between constructor-function streams with backpressure and { end: false }.

  • 6562-constructor-function-transform.cjs: yazl's ByteCounter, a Transform with _flush, tar-stream's PassThrough entry stream with its own destroy, a PassThrough without _transform, pipe chains into zlib.createGzip() and fs.createWriteStream, and fs.createReadStream(file) piped into constructor-function streams.

  • 6563-constructor-function-stream-errors.cjs: write errors with identity and code, errored, destroy with an error and a callback, an asynchronous _destroy, ERR_METHOD_NOT_IMPLEMENTED, a throwing _read, write after end, custom events with three arguments, once/prependListener/removeListener/listenerCount, the unhandled 'error' rethrow, and setMaxListeners.

  • Node and native agree in both lanes, including the sanitized lane's RC audit and LeakSanitizer.

  • Lint, format, test:ts7 --baselines-only (baselines recorded for the four programs), the diagnostics suite, the npm stream and event cases, the runtime-pack and feature unit tests, and the focused stream, emitter, inheritance, zlib, class and util corpus (about 460 programs, plain lane; about 200 sanitized) pass apart from failures that also occur on the baseline in this environment. The self-hosting IR build passed before merging main; on current main it fails on both sides (wasm/binary.ts, SC2002), which this change does not touch.

  • The real from2 package does not compile yet for reasons outside this change (require("readable-stream") in an expression without the alias from Compile polyfill packages that are Node builtins as the builtins #1458, and new on a function value), so there is no npm case.

Zero cost

node scripts/zero-cost-gate.mjs --baseline=<main f4f2baf1a> --candidate=<this branch> --corpus=all: 3387 programs identical in IR, LLVM and release executables (benchmarks and corpus), 1 that fails on both sides (the self-hosted compiler, above), and 4 differences, all programs that use the untyped EventEmitter or stream constructor values from #1512. Bench geomean +1.12% with a 95% interval of [-0.77%, +1.84%], within tolerance (no bench program's code changed).

  • 5977-builtin-class-aliases.cjs (stores PassThrough in an array) and 6454-constructor-function-stream-bases.cjs (util.inherits onto the five stream classes) now read the stream classes through dyn.builtinStreamClass and link the stream half and the stream unit: +11.0% and +24.0% binary size. Their output is unchanged.
  • 6453-constructor-function-emitter.cjs and 5818-constructor-function-emitter-defaults.cjs link only the EventEmitter unit, whose methods now test for stream state: +40 bytes each.

After the first full run, the legacy Stream constructor moved back to dyn.builtinClass (it needs no stream half); a corpus rerun of the affected programs and the stream programs (97) confirmed the four differences above and nothing else.

🤖 Generated with Claude Code

cramforce and others added 5 commits October 20, 2026 09:46
Bundled npm code (from2, bl, tar-stream, yazl, multistream) builds its
streams as ES5 constructor functions: util.inherits(Ctor, Readable) and
Readable.call(this, opts), with _read/_write/_transform on the
prototype. Readable.call(this) threw SC2020.

A new runtime unit, scr_dyn_builtin_stream.c (feature
dynBuiltinStream), gives the untyped stream constructors their stream
half. Base.call(this, opts) creates a runtime stream over the receiver
and keeps it in a hidden slot; the runtime calls the receiver's
underscore methods through its property lookup; the stream prototypes
carry Node's methods and accessors over the hidden stream; and
EventEmitter's methods register listeners with the runtime stream's
emitter, so runtime events and pipes reach JavaScript listeners with the
receiver as this. _readableState/_writableState are live views.

Stream class values now come from dyn.builtinStreamClass, so only
programs that use them link the stream half. Typed code sees these
objects as their runtime streams: checked casts to the stream classes
take them, readable.pipe() into an untyped destination pipes into them,
and runtime streams crossing into JavaScript answer the constructor
object.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
pipe() into an untyped destination now comes from main's pipeIntoDyn
(stream.unboxDyn); programs with constructor-function streams unbox
through scr_dyn_builtin_stream.c, which also takes those streams, so the
separate dyn.builtinStreamPipe entry point is gone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…orm corpus program

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Programs whose only stream value is the legacy Stream do not need the
stream half.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vercel

vercel Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
scriptc Ready Ready Preview, v0 Oct 20, 2026 6:17pm UTC

Comment thread packages/runtime/src/scr_dyn_builtin_stream.c Outdated
Comment thread packages/runtime/src/scr_dyn_builtin_stream.c Outdated
A constructor-function stream now reads `_final` and `_flush` when its
writable side finishes, and `_transform` for each chunk, as Node does. A hook
assigned after `Writable.call(this)` or `Transform.call(this)` runs, and an
override of a PassThrough's `_transform` replaces the pass-through. Without
`_final` or `_flush` the finish continues as Node's does.

A runtime stream passed to a constructor-function stream's listeners, such
as the source of a 'pipe' event, is the view whose pipe and unpipe accept
constructor-function streams, so `src.unpipe(this)` in a 'pipe' listener
unpipes instead of throwing.
@ctate
ctate merged commit a0894cc into main Oct 20, 2026
6 checks passed
@ctate
ctate deleted the opt/stream-ctor-bases branch October 20, 2026 18:21

This branch was successfully deployed

1 active deployment
Preview — 53443672 Deployed Oct 20, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants