You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 1f41281
Browse filesBrowse the repository at this point in the historyBrowse files
cuda.core: share VirtualMemoryResource buffers between processes (#3028)
* cuda.core: share VirtualMemoryResource buffers between processes
Buffers of a VirtualMemoryResource whose handle_type is "posix_fd" can now
be exported and imported like pool-backed buffers. Buffer.ipc_descriptor
exports every physical allocation that backs a VirtualMemoryBuffer as a
file descriptor, Buffer.from_ipc_descriptor imports the descriptor with a
VirtualMemoryResource of the receiving process, and a buffer or its
descriptor can be sent through multiprocessing directly. A grown buffer
imports as one contiguous range with the same byte layout.
The import reuses MemAllocationHandle: import_mem_allocation_handle wraps
cuMemImportFromShareableHandle in the same box and cuMemRelease deleter as
create_mem_allocation_handle, with the importer's access descriptors.
VirtualMemoryIPCBufferDescriptor carries one IPCAllocationHandle and one
size per allocation and owns the file descriptors; multiprocessing
duplicates them into the receiving process. VirtualMemoryResource pickles
as (device, options), which lets a buffer pickle as (resource, descriptor).
Fabric handles are not shared yet: no available test system reports fabric
support or exposes an IMEX channel, so that path could not be verified.
Part of #2980.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* cuda.core: compare the descriptor's chunk count with a signed length
The CI wheel build compiles with -Werror and -Wsign-compare, and the VMM
import path compared a size_t count with the Py_ssize_t that len() returns.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* cuda.core: mark two VMM IPC tests thread-unsafe
Under pytest-run-parallel the same test body runs in several threads. The
alias test checks mapping state by address, and a freed address can be
reused by another thread's allocation; the descriptor test checks a closed
file descriptor number, which another thread can reuse.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* cuda.core: validate and harden VirtualMemoryResource IPC imports
Address the review of the VirtualMemoryResource IPC path.
Import: check every imported allocation's location against the
resource's before mapping (the same device, or the same host location
type) and raise a ValueError that names both; reject a descriptor with
a size but no allocations; attach notes to driver failures that say
which chunk failed and why a peer-controlled size fails at the map.
The imported buffer no longer keeps the descriptor: the driver copies
the shared allocation into a handle of its own and keeps no reference
to the file descriptor, so the descriptors live only as long as the
caller holds the descriptor object.
Export: an imported buffer, and a buffer grown from one, cannot be
exported again because the driver exports only allocations created
with the requested handle type; say so instead of surfacing
INVALID_VALUE. modify_allocation() rejects a config whose handle_type
differs from the resource's, so every chunk of a buffer is exportable
the same way. __reduce__ preserves subclasses.
Tests: parametrize the memory_ipc suite over VirtualMemoryResource
(VirtualMR), with the pool-only scenarios guarded or skipped;
PatternGen works on the first `size` bytes of a larger buffer; the
fd-leak harness asserts on the success path; new VMM-only tests for
many-chunk ranges, a second device with and without peers, descriptor
validation and rollback, re-export, empty buffers, and a DLPack
consumer.
Docs: file descriptor and memory lifetime, multiprocessing-only
transport, trust boundary, device rule, re-export limit;
VirtualMemoryIPCBufferDescriptor in api_private; the
from_ipc_descriptor stub has its mr type again.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* cuda.core: no descriptor cache, no re-export, raw-fd transport for VMM IPC
Apply the decisions on the VirtualMemoryResource IPC design.
No exporter-side cache. VirtualMemoryBuffer.ipc_descriptor exports on
every access and returns a new descriptor that owns its file
descriptors; they close when the descriptor is released. Pickling a
buffer for multiprocessing builds a transient descriptor, so a sender
holds no file descriptors between sends. A spawned Process is the one
exception: it receives the file descriptors when it is created, after
pickling, so the transient descriptor stays on the spawning Popen until
the Process object is released. A descriptor pins the physical memory
while it exists, and nothing else does.
No re-export of imported memory. Every physical allocation records
whether cuMemImportFromShareableHandle produced it
(mem_allocation_is_imported), and modify_allocation reuses the
allocation handles of its input, so the mark reaches every alias,
in-place grow, and move. Exporting such a buffer, directly or through
pickling, raises a RuntimeError before any driver call that says the
buffer contains memory imported from another process, that it cannot be
exported again, and that the caller should forward the descriptor it
imported from or copy into a buffer it owns. A Queue feeder thread
reports the same error and drops the item.
Raw-fd transport. VirtualMemoryIPCBufferDescriptor exposes handle_type,
sizes, handles, and fds, and from_fds() rebuilds a descriptor in the
receiving process from file descriptors received another way (the
integers are duplicated; IPCAllocationHandle objects are shared). The
descriptor's own pickling stays multiprocessing-only.
No ipc_enabled option: handle_type is the switch and is_ipc_enabled the
query, and the docs say so.
Tests: file descriptor counts return to baseline on both sides, export
under a lowered RLIMIT_NOFILE, the re-export errors for imported and
derived buffers (direct, Pipe, Process arguments, Queue feeder thread),
and raw-fd round trips through os.dup and a Unix socket with
SCM_RIGHTS.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* docs(cuda.core): record the rejected retention design for VMM IPC forwarding
Write down in VMM_DESIGN.md the opt-in design that would let an imported
VirtualMemoryResource buffer be forwarded (keep the received handles with
the imported buffer and duplicate them on export), why the option would
belong on the resource, and its cost, so that it does not have to be
re-derived. Not implemented; nothing needs it yet.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* test(cuda.core): import resource inside the fd-limit test
The module-level import of the POSIX-only resource module made pytest
fail to collect tests/memory_ipc/test_vmm_ipc.py on Windows, so the
Windows rows ran no cuda.core tests at all. Every test in the module is
skipped on Windows by its fixture; only the import has to move.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* cuda.core: VMM IPC review fixes; no sharing of host-located virtual memory
Take ownership of each exported or duplicated file descriptor before
appending its handle, so a failed append cannot close the descriptor
twice. Validate imported chunk sizes against the minimum granularity
that cuMemMap requires rather than the resource's granularity option.
Name both causes in the note on a failed import: a handle that is not a
CUDA allocation of the expected type, or an owning device that this
process cannot use. Report is_ipc_enabled False for host-located
resources: host NUMA virtual memory has no verified allocation path, so
it is not shared yet; the docs and the design note say so.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* cuda.core: VMM IPC review fixes: posix_fd-only from_fds, scoped docs, no test skips
- from_fds() takes no handle_type; a file descriptor can only carry posix_fd
- The descriptor, resource, and release-note text scope the pin, pickle, and
same-device rules to POSIX file descriptors, and the fds docstrings state
the descriptor's lifetime rule
- The 1.3.0 notes record that a per-call modify_allocation config must keep
the resource's handle_type
- VMM_DESIGN.md records the fabric direction: a descriptor class of its own
- TestManyChunkBuffer and TestReexport no longer skip when the driver refuses
the decoy's address hint; the grow relocates in that case too
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* test(cuda.core): collect garbage before each file-descriptor baseline in the VMM IPC tests
init_cuda no longer runs gc.collect() at every teardown (#3053), so objects
left by earlier tests (child pipes, queues, descriptors) can be collected in
the middle of a test that counts process-wide file descriptors, and the
count then drops below its baseline. Collect before taking the baseline.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* test(cuda.core): mark the VMM re-export test thread-unsafe
TestReexport._derived asserts that a grow moved the buffer after a decoy
took the adjacent range. Under pytest-run-parallel another thread can
free that range between the decoy attempt and the grow, so the grow
extends in place and the assertion fails. Mark the test thread_unsafe
like the other address-probing tests in the file.
Also record in VMM_DESIGN.md the numbers behind the no-cache export rule
and the age of the multiprocessing machinery the Process path relies on.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
0 commit comments