Repository navigation
Let NodeJS stream interface slots hold JavaScript stream objects - #1524
Merged
Merged
Conversation
Under the general stream representation, NodeJS.ReadableStream and NodeJS.WritableStream compile as the runtime Readable and Writable classes, so a JavaScript stream object (a constructor-function stream, a plain object with on/pipe, a stream from bundled npm code) passed through a TypeScript function typed with them took the checked unwrap and threw. A program that moves an untyped value into a runtime stream class slot, or passes an untyped pipe() destination or pipeline stage, now lowers once more with checked-dynamic stream slots: the two interfaces (and intersections over them) map to dyn. A native stream rides its typed capsule and keeps its identity; any other value is the JavaScript object itself. Member calls dispatch per value: stream.dynNative tests for a boxed native stream, which takes the ordinary class-method lowering; other values call their own member through dynInvoke. pipe() into a JavaScript destination follows Node's Readable.prototype.pipe listener protocol (stream.pipeDyn). for await and pipeline stages keep the not-supported error for values that are not native streams. The runtime code lives in scr_stream_dyn.c behind a new streamDyn feature, so programs without these libCalls link the same objects. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
6480 passes a plain object, a constructor-function stream and a native
Readable through a NodeJS.ReadableStream function and consumes them in
TypeScript (on("data"), pipe()) and JavaScript, including identity and a
native pipe into JavaScript sinks. 6481 covers NodeJS.WritableStream
slots and a copy helper across native and JavaScript sources and
destinations. 6482 covers the helper shapes real code writes: an async
producer, pipe with { end: false } and unpipe, pause/resume/setEncoding,
an optional destroy through an intersection type, for await and pipeline
over native values.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
With checked-dynamic stream slots, readable.unpipe() also unpipes every JavaScript destination stream.pipeDyn registered from the source, oldest pipe first. Each destination hears 'unpipe' and the source pauses, as in Node's unpipe(), which empties every pipe. Before, those pipes stayed attached and later chunks still reached them.
euphemism
pushed a commit
to euphemism/scriptc
that referenced
this pull request
Oct 20, 2026
…e headers The backend can emit scr_wasm_memory_construct, scr_wasm_table_construct, scr_wasm_global_construct, scr_wasm_construct_without_new, scr_wasm_module_imports, scr_wasm_module_exports, scr_wasm_module_custom_sections and scr_wasm_value_of (vercel-labs#1410), and scr_stream_dyn_native, scr_stream_pipe_dyn and scr_stream_unpipe_dyn (vercel-labs#1524). Their prototypes were in scr_wasm.c or nowhere, so the runtime ABI guard failed. They are now in scr_wasm.h and scr_stream.h, and scr_wasm.c no longer repeats the header's declarations. The speculative-hoisting test relied on `v instanceof Num` in a JavaScript function failing, which constructor-function instanceof now compiles (vercel-labs#1512). Its hoisted function reads the prototype through Reflect.getPrototypeOf instead, which still fences.
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Under the general stream representation (#1518),
NodeJS.ReadableStreamandNodeJS.WritableStreamcompile as the runtimeReadableandWritableclasses. JavaScript code also passes its own stream objects through TypeScript functions typed with these interfaces: constructor-function streams, plain objects implementingonandpipe, and the streams of bundled npm code. Such a value entering the slot took the checked unwrap, so it threw aTypeError(or the not-supported error atpipeandpipelinestages) instead of passing through.Decision
The choice stays per program, as a further step of the #1518 decision. A program whose reachable code moves an untyped value into a runtime stream class slot (the checked unwrap), or passes an untyped
pipe()destination orpipelinestage, is lowered once more with checked-dynamic stream slots:NodeJS.ReadableStreamandNodeJS.WritableStream, and intersections over them, map to the checked-dynamic representation. A native stream entering the slot is boxed as its typed capsule, so it unboxes to the same stream and keeps its identity in JavaScript. Any other value is the JavaScript object itself, and returning it to JavaScript gives back that object.stream.dynNativetests whether the value is a boxed native stream with the needed side. A native stream takes the ordinary class-method lowering on the unboxed stream, the same lowering a typedReadableorWritablegets. Any other value calls its own member throughdynInvoke, with typed callbacks boxed behind checked thunks.readable.pipe(destination)into an untyped destination pipes natively when the destination is a native stream. Otherwisestream.pipeDynfollows Node'sReadable.prototype.pipe: it registers the same listeners in the same order (endanddataon the source;unpipe, a prependederror,closeandfinishon the destination), emitspipe, writes each chunk, pauses onfalseuntildrain, ends the destination, and unpipes and cleans up onfinish,closeorerror.child.stdout,child.stderrandchild.stdinreads keep producing the bridged runtime streams, andprocess.stdoutandprocess.stderrbox as their fd-backedWritablewhere they flow into such a slot.for awaitover a checked-dynamicNodeJS.ReadableStreamiterates a native stream and throws the not-supported error for other values. Apipelinestage that is not a native stream keeps the not-supported error, and its message now names JavaScript stream objects.stream.Readableandstream.Writable(under@types/node),fs.ReadStream, the zlib classes and the other class spellings keep the runtime classes; only the two global interfaces become checked-dynamic.Changes
dyn-streams.ts(new): the interface test, per-value member dispatch, the native/other branch helper, andfor awaitover the slot.type-mapper.ts: thedynNodeStreamsmapping, including intersections and theChildProcessstdio members.lowerer.ts,general-streams.ts: the trigger (noteUntypedStreamFlow) and the extra lowering pass;receiverTypeOverridesfor the native arm.lower-calls.tscalls the dispatch first among member-call lowerings, andlowerObjectMethodCallhonors the receiver override.lower-stream.ts:pipe()into an untyped destination branches between the native pipe andstream.pipeDyn.stream.dynNativeandstream.pipeDynlibCalls (signatures, validator, LLVM emission, effects).scr_stream_dyn.c(new runtime unit) behind a newstreamDynruntime feature, so programs that do not use these libCalls link exactly the objects they linked before.Zero cost
Programs without such a flow take the same lowering passes as before, and the new runtime code sits in its own unit selected only by the new libCalls. Gating that unit on the existing stream and dyn-invoke features instead added 40 bytes to 34 existing executables, so it has its own feature.
node scripts/zero-cost-gate.mjs --baseline=<opt/node-stream-interfaces 1f58ff625> --candidate=<this branch> --corpus=all: 3316 programs identical in IR, LLVM and release executables, 1 intended difference, 1 program fails to build on both trees for an unrelated environment reason (node24-file-url-options.ts). Compile time is within tolerance (benchmarks geomean -0.62%, self-hosted compiler -0.42%).The intended difference is
6467-node-stream-untyped-sources: it returns an untyped JavaScript value asNodeJS.ReadableStreamand castsunknownto it, so it now takes checked-dynamic stream slots (and links the dyn-invoke unit). Its output is unchanged.Tests
tests/corpus/6480-node-stream-dyn-readable/: a plain object implementingon/pipe, a constructor-function stream and a nativeReadablepass through a function typedNodeJS.ReadableStream; TypeScript consumes them withon("data")andpipe()into a nativeWritable; JavaScript consumes the returned values and sees the same objects; a nativeReadabletypedNodeJS.ReadableStreampipes into JavaScript sinks, one of which records the listeners Node's pipe protocol registers and removes.tests/corpus/6481-node-stream-dyn-writable/:NodeJS.WritableStreamslots holding a nativeWritable, a plain JavaScript sink andprocess.stdout; a copy helper over both interfaces pipes native and JavaScript sources into native and JavaScript destinations.tests/corpus/6482-node-stream-dyn-helpers/: an async helper returning a file stream or a JavaScript part,pipe(out, { end: false })andunpipe,pause/resume/setEncoding/readable, an optionaldestroy?.()through an intersection type, andfor awaitandpipelineover native values in the slot.All pass under Node and native in the plain and sanitized lanes.
With #1512 merged locally (not part of this branch), constructor functions built with
util.inheritsoverEventEmitterand the legacyStreamalso pass through these slots:on("data"),pipe()into native and legacy destinations, and handing them back to JavaScript all match Node. Those cases are not in the corpus here because they need #1512.Not covered
pipeline()with a stage that is a JavaScript stream object throws the not-supported error. Node's pipeline over such objects depends on its end-of-stream detection for legacy streams, which this change does not model.for awaitover a JavaScript stream object throws the not-supported error.emitto JavaScript, so a JavaScript stream whose ownpipe()emits on its destination (the legacyStreampipe from Let constructor functions inherit from EventEmitter and the stream classes #1512) cannot pipe into a native stream. Addingemitto the native stream view would change the runtime unit existing programs link.readablethrows instead of readingundefined.stream.pipeDynpauses the source when its JavaScript destination unpipes, as Node does for a source's last pipe, even if the source also pipes into native destinations.🤖 Generated with Claude Code