fix(array): resolve a Buffer-backed Uint8Array receiver in the fused callback helpers (#8137) - #8173
Conversation
…callback helpers (#8137) Nine fused `js_array_*` callback/reduce entry points returned GARBAGE values — not empty, not a throw — for a Buffer-backed `Uint8Array` receiver codegen could not statically prove. Perry's `new Uint8Array([…])` is a `BufferHeader`, so it is absent from the typed-array registry and the `lookup_typed_array_kind` re-dispatch each helper performs never answers for it. The helper then reads the `BufferHeader` as an `ArrayHeader`. Both share the `{length, capacity}` prefix, so `length` reads CORRECTLY while the elements — decoded as NaN-boxed f64 slots at `base + 8 + i*8` over a payload of one byte per element — are raw bytes reinterpreted, `length * 7` bytes past the real payload. Ask the receiver-kind question ABOVE the array-only funnel and delegate to `dispatch_uint8_buffer_method`, the same dispatcher a statically typed receiver already reaches. It reads through `js_buffer_get` and passes the ORIGINAL Buffer as the callback's 3rd/4th argument, so a write through `arr` in `u.forEach((v, i, arr) => { arr[0] = 9 })` still lands on `u` — which delegating through `buffer_receiver_as_uint8_typed_array` (a COPY) would have silently lost. Also adds the missing `findLast` arm, which threw `TypeError: (Buffer).findLast is not a function` on BOTH dispatch paths.
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (9)
📝 WalkthroughWalkthroughChangesBuffer-backed Buffer-backed Uint8Array callbacks
Estimated code review effort: 3 (Moderate) | ~25 minutes Merge Risk: ⚪ Minimal · up to The PR corrects Buffer-backed Uint8Array callback behavior while preserving existing behavior for other receiver types, with the described tests and checks passing. No actionable merge-blocking risk remains beyond normal review. Sequence Diagram(s)sequenceDiagram
participant ArrayMethod
participant BufferReceiver
participant Uint8Dispatcher
participant Callback
ArrayMethod->>BufferReceiver: dispatch Buffer-backed Uint8Array method
BufferReceiver->>Uint8Dispatcher: pass callback and original Buffer
Uint8Dispatcher->>Callback: invoke with byte value and index
Callback-->>Uint8Dispatcher: return callback result
Uint8Dispatcher-->>ArrayMethod: return converted method result
Possibly related PRs
Suggested labels: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Closes #8137.
Nine fused
js_array_*callback/reduce entry points returned garbage values— not empty, not a throw — when the receiver was a Buffer-backed
Uint8Arrayand codegen could not statically prove the receiver type.
Root cause
Perry's
new Uint8Array([…])is aBufferHeader(buffer::js_uint8array_new),not a
TypedArrayHeader. It is therefore absent from the typed-array registry,and the
lookup_typed_array_kindre-dispatch each of these helpers performsnever answers for it. There is no Buffer arm, so the helper falls through and
reads the
BufferHeaderas anArrayHeader.BufferHeaderandArrayHeadershare the{length: u32, capacity: u32}prefix. So
lengthreads correctly while the elements — decoded asNaN-boxed f64 slots at
base + 8 + i*8over a payload that is one byte perelement — are raw bytes reinterpreted, and the read runs
length * 7bytes pastthe real payload. Correct length, garbage values. That is why the symptom is not
an empty result, and why any probe that only asks "did we return
[]?" is blindto it.
normalize_array_receiveris permissive here (it returns the raw address for aregistered Buffer rather than null), so this is a standing gap, not the
#8041 funnel-ordering regression that #8090 / #8109 / #8119 / #8130 / #8140 have
been closing. The receiver-kind question is nevertheless asked above the
funnel, because the sibling funnel
clean_arr_ptrdoes return NULL for the samereceiver and every caller reads that as "empty" — keeping the question above
means the ordering cannot rot back.
The design call
The issue laid out three options. This takes (c): delegate to
dispatch_uint8_buffer_method, the shared uint8%TypedArray%.prototypedispatcher a statically typed receiver already reaches via
dispatch_buffer_method's catch-all.It is the only option that keeps callback-argument identity. These nine pass the
receiver to the callback as the 3rd/4th argument, and the dispatcher passes the
original Buffer. Option (b)'s alternative —
buffer_receiver_as_uint8_typed_array— hands back a copy, sowould silently lose the write. That is not hypothetical: it is pinned by a test
that fails when the implementation is swapped to a copy (sabotage E below).
Delegating also closes the statically-typed holes for free, exactly as the issue
predicted:
reduceRightwas wrong even forconst u = new Uint8Array([3,1,2]),because codegen folds that call straight to
js_array_reduce_rightratherthan routing it through
dispatch_buffer_method. Measuredz|6.36e-314|5.09e-313|6.49e-319against node'sz|2|1|3.findLasthad no arm in the uint8 dispatcher at all, so it threwTypeError: (Buffer).findLast is not a functionon both paths. Itssibling
findLastIndexwas already served, which is why the hole survived —the two are always cited together. One arm added.
What is fixed
Measured with
{ u: new Uint8Array([3, 1, 2]) }against nodev26.5.1(
.node-version),--profile perry-dev, pinnedPERRY_RUNTIME_DIR,PERRY_NO_AUTO_OPTIMIZE=1:js_array_map[6,2,4][1.297723e-318,0,0][6,2,4]js_array_filter[3,2][][3,2]js_array_find1undefined1js_array_findIndex1-11js_array_sometruefalsetruejs_array_everytruefalsetruejs_array_reduce66.4886e-3196js_array_reduce_rightz|2|1|3z|0|0|6.4886e-319z|2|1|3js_array_forEach3;1;2;6.4886e-319;0;0;3;1;2;Two more of the same defect, found by the sweep and fixed here:
js_array_map_discard(.map()result unused)3;1;2;6.4886e-319;0;0;3;1;2;findLast(both dispatch paths)2TypeError2reduce/reduceRightare covered with and without a seed: an absentinitial value must stay absent through the delegation, because the dispatcher
keys on
args.len() >= 2and forwardinginitialunconditionally would turnevery seedless reduce into a seeded one — and the empty-receiver
TypeErrorinto a silent
undefined.What is deliberately NOT changed
flatMapis not a%TypedArray%.prototypemethod (node throws). The gateanswers
Nonefor it, so the caller keeps today's behaviour. Pinned by a test.ArrayBuffer/SharedArrayBuffer/DataVieware declined byis_typed_array_buffer— the same gatedispatch_buffer_method's catch-alluses, so the two receiver populations cannot drift apart. Perry answers
ab.map(cb) => [0,0,0,0]where node throwsTypeError; that divergence ispre-existing and byte-identical before and after this change, and is a
separate issue. Serving them here would have invented iteration node does
not have.
js_array_find_last/js_array_find_last_index(the generic helpers)are not gated. Measured: a Buffer receiver never reaches them —
findLastIndexalready answers correctly through the dynamic tower, and
findLastwas themissing dispatcher arm, not a missing helper gate. A guard there would be
untestable.
Cost on the ordinary path
buffer_receiver_dispatchopens withtypedarray::arena_payload_has_gc_type(addr, GC_TYPE_ARRAY): a receiver that isprovably an arena-backed
GC_TYPE_ARRAYcannot be a registered Buffer, so[1,2,3].map(…)reaches neither registry.That predicate rather than a bare header-byte read is deliberate, and it is the
correction #8142 wrote into
array_receiver_gc_tag's doc: a Buffer comes inboth backings, and an external one (
EXTERNAL_BUFFER_REGISTRY,shared_sab::alloc_shared_sab) has noGcHeaderat all — the eight bytes belowits payload are allocator bookkeeping that can read as any
obj_type,GC_TYPE_ARRAYincluded. A bare tag read would skip the probe for exactly thereceiver this function exists to catch, silently and only sometimes.
arena_payload_has_gc_typerange-checks, rejectsHeapSpace::Unknownfor theheader address, and validates through
gc_type_infofirst; it answersfalsefor an external buffer, which then falls through to the registry — the
authoritative answer.
The gate is asserted, not assumed:
a_plain_array_never_reaches_the_buffer_gatemeasures a
#[cfg(test)]probe counter onis_typed_array_bufferand fails ifthe fast path is deleted, even though every answer stays correct.
Testing — and the vacuity this family invites
18 tests in
array/typed_array_receiver_tests.rs. Every one asserts theobserved element values, never a predicate. The issue names the trap
explicitly, and it has already been shipped here once:
u.every(x => x > 0)answerstrueunder node and under the bug, because1.297723e-318 > 0. The callbacks record what they actually saw, so a garbageread fails the assertion whatever the predicate says.
Sabotage results — each mutation applied, tests re-run, then reverted:
buffer_receiver_dispatchalways returnsNone(pre-fix)arena_payload_has_gc_typegatea_plain_array_never_reaches_the_buffer_gatefindLastarmfind_last_is_served_for_a_buffer_receiveran_array_buffer_or_data_view_receiver_is_not_given_element_semanticsthe_callbacks_third_argument_is_the_receiver_itself_not_a_copySabotage A caught a vacuous test of my own: the 3rd-argument test passed
under it, because the pre-fix helper already passed the raw buffer as the 3rd
argument while reading garbage elements. It now carries an element assertion
too, so it fails under A as well as E; its comment says which mutation each
assertion is there for.
Controls that fail on over-reach: a plain array and a generic array-like object
(
Array.prototype.map.call({length: 3, 0: 3, …})) must still work through thesame helpers, and
Int32Array(a real registry typed array) is checkedend-to-end.
Verification
cargo test -p perry-runtime --lib— 2436 passed, 0 failed, 4 ignored(2418 on
main+ 18 new).v26.5.1for all 36 probe cases; the onlyremaining diff is the pre-existing ArrayBuffer/DataView one described above,
identical before and after.
cargo test -p perry-codegenisnot implicated.
lint-job gates generated from.github/workflows/test.yml(32 commands)plus
scripts/check_gc_env_knobs.py: all pass.Summary by CodeRabbit
Uint8Arraydata.findLaston Buffer-backedUint8Arraydata.ArrayBuffer,DataView, and unsupported cases.