Repository navigation
cuda.core: PinnedMemoryResource.is_device_accessible returns True without granting device access to the pool #3205
Description
PinnedMemoryResource.is_device_accessible returns True unconditionally, and its docstring says the resource "provides device-accessible buffers" (cuda_core/cuda/core/_memory/_pinned_memory_resource.pyx, the is_device_accessible property). The resource creates its pool with CUmemPoolProps only and never calls cuMemPoolSetAccess.
The driver documents the default differently. From cuMemPoolCreate: "In the case of pools created with CU_MEM_LOCATION_TYPE_HOST_NUMA or CU_MEM_LOCATION_TYPE_HOST, their default accessibility will be from the host CPU." Device access to such a pool is granted with cuMemPoolSetAccess.
Nothing in the test suite settles which is right in practice: every kernel launch on pinned memory uses LegacyPinnedMemoryResource, and the pool-backed tests only copy on the host.
Asks:
- A test that launches a kernel which reads and writes a
PinnedMemoryResourcebuffer, on the allocating device, with and withoutnuma_idset. - If the kernel faults: grant access for the device at pool creation (
cuMemPoolSetAccesswithCU_MEM_ACCESS_FLAGS_PROT_READWRITEfor the current device, or expose the choice), and keepis_device_accessibleasTrue. If it does not fault: keep the behavior, and make the docstring say what the access rule is, so it does not promise more than the pool provides.
Found while reviewing #2713, whose docstring text for allocate touched the same question.
Activity
saiyambharara commentedon Nov 1, 2026
@Andy-Jost I'd like to take this one, following up from #3262. Step 1 already faults here: on WSL2 with an RTX 5070 Ti (driver 13.05), a kernel on a
PinnedMemoryResourcebuffer hitsCUDA_ERROR_ILLEGAL_ADDRESSwith and withoutnuma_id, and works oncecuMemPoolSetAccessgrants the device read/write. Sincedevice_idis -1 and the no-options path uses the shared default host pool, should access go to every visible device or only the current one?Andy-Jost commentedon Nov 2, 2026
@saiyambharara Thanks for picking this up, and for running step 1. The fault on a plain
PinnedMemoryResourcebuffer settles the question: the pool needs an explicit grant.Only the current one. Please grant
CU_MEM_ACCESS_FLAGS_PROT_READWRITEto the current device when the resource is set up, on both paths: the created pool (options given) and the current/default host pool (no options). The grant only adds access, so applying it to the shared default pool does not take anything away from other users of that pool. Granting every visible device would touch devices the user never asked for, and can fail on devices that do not support host memory pools. If someone needs access from more devices, we can add an explicit option for that later.Please include the kernel test from step 1 (read and write, with and without
numa_id), keepis_device_accessibleasTrue, and update its docstring to say that access is granted to the device that was current when the resource was created.