You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Views behind null rows are unvalidated, so a dense kernel over VarBinViewArray can panic #9090
Drafted by Claude Code on Connor's behalf, and not yet edited by him.
VarBinViewArray::validate_views only validates the views of valid rows, which is intentional. So a legal array can hold a view behind a null row naming a buffer that does not exist.
ScalarFnVTable::is_strict consequence 2 reads as an unconditional licence to ignore that: "Values behind null slots are irrelevant, so kernels may compute g densely over all lanes (including garbage) and apply validity afterwards." That holds for a flat fixed-width payload, where a null row is unused bytes and reading it can't fault. It doesn't hold for a VarBinViewArray row, where reading the value means following a buffer index and offset into a data buffer.
A kernel that resolves every row densely panics on a legal array:
index out of bounds: the len is 1 but the index is 9
Repro:
let views = buffer[BinaryView::make_view(b"a longer string here",0,0),// Null row: buffer 9 does not exist and the offset is past the end of the data.BinaryView::new_ref(64,*b"junk",9,4096),];let array = VarBinViewArray::try_new(
views,Arc::from([ByteBuffer::copy_from(b"a longer string here")]),DType::Utf8(Nullability::Nullable),Validity::from_iter([true,false]),)?;
try_new succeeds, which is correct. Any kernel that then resolves row 1's bytes panics.
Note that no kernel on develop is broken by this, since byte_length reads view.len() out of the view rather than resolving the row. So this is a foot-gun plus a missing test rather than a live bug.
Two steps:
Narrow consequence 2. Dense evaluation is sound when reading a row can't fault, which excludes offset-following elements.
Add a regression test pinning byte_length over the array above to [Some(20), None], guarding against a later "optimization" into resolving rows.
Both exist on claude/strict-scalar-fn-abstraction-ah88x3 as a DENSE_SAFE const on the element type, but neither the doc fix nor the test needs any of that.
I think we need to be able to push down some notion of only compute these rows and this should be a row demand, if the function is non panicing you can push it down without any row demand
Activity
connortsui20 commentedon Jul 30, 2026
(llm generated, will touch up soon)
Verify DENSE_SAFE mechanically, and measure what the port costs
robert3005 commentedon Jul 30, 2026
I think we need to be able to push down some notion of only compute these rows and this should be a row demand, if the function is non panicing you can push it down without any row demand
myrrc commentedon Aug 11, 2026
Fixed by #9033 #9122
connortsui20 commentedon Aug 11, 2026
Those were docs changes to the idea of strictness, but we still dont have a good definition of "definedness" vs null values.
connortsui20 commentedon Aug 19, 2026
I believe this is closed by #9188