fix(streaming): copy deserialized numpy arrays so they are writable

[hdimer]

What does this PR do?

Fixes #818.

NumpySerializer.deserialize and NoHeaderNumpySerializer.deserialize return np.frombuffer(...) directly. np.frombuffer wraps its input as a read-only view, so every numpy array coming out of a StreamingDataset is non-writable and the default collate's torch.from_numpy warns on it:

UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior.

The warning is the visible symptom, but the view is the bigger problem. On the PyTreeLoader fast path, _slice_item_bytes returns a memoryview over a live mmap.ACCESS_READ mapping and passes it straight to the serializer, so the returned array aliases a mapped chunk file that _close_mapping / _evict_mapped_chunks can unmap under LRU pressure. That's the same hazard TokensLoader already guards against, with the same idiom, in item_loader.py:

# Copy out of the memmap. ``close()`` unmaps the previous chunk on the next sample, and
# DataLoader may still be pickling the last items — a view into a closed mmap is SIGSEGV.
data = np.frombuffer(buffer, dtype=self._dtype, count=self._block_size, offset=offset).copy()

So this just applies that existing pattern to the two numpy serializers.

Cost

One extra memcpy per numpy field per item: ~1 µs for a 3 KB embedding or CIFAR-sized image, ~240 µs for a 3 MB array. On the non-mmap path the slice of the source bytes already costs about the same, so for those callers it is roughly a doubling of an existing copy rather than a new one. setflags(write=True) isn't an option — numpy refuses it on a bytes-backed array — and frombuffer(bytearray(...)) costs the identical copy.

Scope

Only the two numpy deserializers change. I left the torch serializers and the warnings.filterwarnings("ignore", message=".*The given buffer is not writable.*") in reader.py alone: that one targets torch.frombuffer's different "given buffer is not writable" message and looks deliberate, and it never covered the numpy warning in #818. Happy to follow up if you'd like the tensor path to match.

While in here I noticed if tensor.shape == shape in NumpySerializer.deserialize compares a tuple against a list, so it is always False and the np.reshape path always runs. Harmless, and left alone to keep this diff to the reported bug. The .copy() sits before that branch, so it holds either way.

Tests

Added a flags.writeable assertion to the two existing numpy serializer tests rather than a new test: test_numpy_serializer already loops over every supported dtype and five shapes, so it pins writability across all of them, and keeping the two serializers in separate tests means a failure says which one broke. Both assertions fail on main and pass here.

test_serializer.py, test_item_loader.py and test_reader.py are green locally (72 passed, 4 skipped), as are ruff, ruff format, codespell and mypy. I also ran the reporter's reproducer from #818 end to end: the warning fires on main and is gone with this change.

I used an AI assistant while working on this. I reproduced the bug, verified the tests fail without the fix, and reviewed the change myself.

---

Used AI assistance on this; I reviewed and tested the change myself.

[codecov-commenter]

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 82%. Comparing base (a847d57) to head (86c2aee).
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@          Coverage Diff          @@
##            main    #887   +/-   ##
=====================================
- Coverage     82%     82%   -0%     
=====================================
  Files         65      65           
  Lines      13377   13377           
=====================================
- Hits       10956   10949    -7     
- Misses      2421    2428    +7     

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.