Skip to content

cuda.core: treat the NULL stream as a default-stream token #3243

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
fedonman wants to merge 2 commits into
base: main
Choose a base branch
from
Open

cuda.core: treat the NULL stream as a default-stream token #3243

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
2a33c03
cuda.core: treat the NULL stream as a default-stream token
fedonman Oct 27, 2026
2d0c7f7
Merge branch 'main' into core-null-stream-default-token
fedonman Nov 2, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions cuda_core/cuda/core/_stream.pxd
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,13 +37,14 @@ cdef inline bint Stream_handle_is_default_token(cydriver.CUstream s) noexcept no

These tokens carry no context of their own; they refer to whatever context
is current, so nothing resolved from one may be cached on the object.
The NULL stream (handle 0) is such a token too.
"""
cdef uintptr_t h = <uintptr_t>s
return h == <uintptr_t>cydriver.CU_STREAM_LEGACY or h == <uintptr_t>cydriver.CU_STREAM_PER_THREAD
return h == 0 or h == <uintptr_t>cydriver.CU_STREAM_LEGACY or h == <uintptr_t>cydriver.CU_STREAM_PER_THREAD

@coderabbitai coderabbitai Bot Nov 2, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Major · Stability & Availability · Heavy lift

Supported by static analysis

Script executed:

sed -n '890,940p' cuda_core/cuda/core/_memory/_virtual_memory_resource.pyx
git diff 57b27097c1263da6da0eccdcea8e85b793696365 HEAD -- cuda_core/cuda/core/_stream.pxd
git show 57b27097c1263da6da0eccdcea8e85b793696365:cuda_core/cuda/core/_memory/_virtual_memory_resource.pyx | rg -n -B6 -A10 'sync\(\)'

Repository: NVIDIA/cuda-python

Length of output: 4926

important: Keep the explicit sync for handle-0 streams in VirtualMemoryResource.deallocate.

On a host-located resource, deallocate(ptr, size, stream=Stream.from_handle(0)) now skips stream.sync(). The predicate change in _stream.pxd makes handle 0 a default-stream token. The guard at _virtual_memory_resource.pyx:915-916 then exempts it. Before this PR, the same call synced the stream.

cuMemUnmap takes no stream argument, so it does not wait for work queued on the NULL stream. Queued work that accesses the range can run after the unmap. The docstring promises stream.sync() when a stream is given. The exemption is written for unbound teardown tokens, not for a stream the caller passes explicitly.

Keep the sync when the caller supplies the NULL stream. Confirm how Buffer teardown passes its token before changing the guard. A check on the handle value alone may not separate the two cases.

@mangeshraut712 mangeshraut712 Nov 3, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With handle 0 now accepted into Stream_handle_is_default_token, the sibling predicate Stream_is_legacy_default_token (line 60, unchanged) is now inconsistent: a Stream.from_handle(0) stream is a "default token" to Stream_get_ctx/record()/sync(), but not a "legacy default token" to the copy paths.

Concretely, copy_batch() (_copy_ops.pyx:179) and copy(options=...) (_buffer.pyx:222) gate on Stream_is_legacy_default_token(s) to raise the friendly TypeError before calling cuMemcpyBatchAsync/cuMemcpyWithAttributesAsync — the same driver entry points whose own docs reject the NULL stream exactly like the legacy token. After this change, Stream.from_handle(0) slips past that gate and hits the driver with a raw 0 handle, where the user gets an opaque driver error (or undefined behavior for entry points that treat 0 as legacy) instead of the intended message.

If handle 0 is meant to behave like CU_STREAM_LEGACY everywhere, Stream_is_legacy_default_token should also match h == 0 — and the error messages in those two call sites may need to mention the NULL handle too. Worth adding a copy-path test with a from_handle(0) stream.



cdef inline bint Stream_is_default_token(Stream self) noexcept nogil:
"""Return True when this stream is CU_STREAM_LEGACY or CU_STREAM_PER_THREAD."""
"""Return True when this stream is the NULL stream, CU_STREAM_LEGACY or CU_STREAM_PER_THREAD."""
return Stream_handle_is_default_token(as_cu(self._h_stream))


Expand Down
7 changes: 7 additions & 0 deletions cuda_core/docs/source/release/1.3.0-notes.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -347,6 +347,13 @@ Fixes and enhancements
physical allocations.
(`#2694 <https://github.com/NVIDIA/cuda-python/issues/2694>`__)

- A :class:`Stream` over the NULL stream (handle 0), such as
``Stream.from_handle(0)``, no longer caches the first context and device it
observes. Like ``LEGACY_DEFAULT_STREAM`` and ``PER_THREAD_DEFAULT_STREAM``
since 1.2.0, it now resolves ``Stream.context``, ``Stream.device``,
``Stream.resources`` and ``Stream.record()`` against the current context.
(`#3242 <https://github.com/NVIDIA/cuda-python/issues/3242>`__)

Deprecations
------------

Expand Down
12 changes: 12 additions & 0 deletions cuda_core/tests/test_green_context.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@
ProgramOptions,
SMResource,
SMResourceOptions,
Stream,
WorkqueueResource,
WorkqueueResourceOptions,
launch,
Expand Down Expand Up @@ -560,6 +561,17 @@ def test_stream_and_event_track_green_context(self, green_ctx):
stream.sync()
event.sync()

@pytest.mark.agent_authored(model="claude-opus-5-5")
def test_null_stream_follows_current_context(self, init_cuda, green_ctx):
stream = Stream.from_handle(0)
assert stream.context == init_cuda.context

with use_context(init_cuda, green_ctx):
assert stream.context == green_ctx
assert stream.record().context == green_ctx

assert stream.context == init_cuda.context

@pytest.mark.agent_authored(model="gpt-5.6")
def test_device_receiver_targets_stored_green_context(self, init_cuda, green_ctx):
primary_ctx = init_cuda.context
Expand Down
Loading