[FEA] cuda.core: memory-pool location routing and new pool attributes (CUDA 13.0/13.2)
@Andy-Jost is already working on this.
Since Jul 23, 2026.
- Dominant language
- Cython
- Stars
- 3.4k
- Forks
- 329
- Avg merge
- 1d 23h
- Merged PRs (30d)
- 116
Description
Summary
CUDA 13.0 makes memory-pool routing location-keyed: pools are addressed by
(CUmemLocation, CUmemAllocationType) — covering device pools, host / host-NUMA pools, and
managed pools (CU_MEM_ALLOCATION_TYPE_MANAGED, 13.0) under one surface:
cuMemGetDefaultMemPool(CUmemoryPool* pool_out, CUmemLocation* location, CUmemAllocationType type)cuMemGetMemPool(CUmemoryPool* pool, CUmemLocation* location, CUmemAllocationType type)(current pool)cuMemSetMemPool(CUmemLocation* location, CUmemAllocationType type, CUmemoryPool pool)
CUDA 13.2 adds six CUmemPool_attribute members: CU_MEMPOOL_ATTR_ALLOCATION_TYPE,
EXPORT_HANDLE_TYPES, HW_DECOMPRESS_ENABLED, LOCATION_ID, LOCATION_TYPE, MAX_POOL_SIZE.
Current cuda.core state:
cuMemGetMemPoolis already used internally to obtain non-owning pool handles
(https://github.com/NVIDIA/cuda-python/blob/c000331de6c37aa4565af74b001271ffcf6d5c99/cuda_core/cuda/core/_memory/_memory_pool.pyx#L283);
cuMemGetDefaultMemPool/cuMemSetMemPoolare unused.- None of the 13.2 pool attributes are exposed on the memory-resource classes.
CU_MEM_LOCATION_TYPE_NONE(13.0) is used internally and, together with
CU_MEM_LOCATION_TYPE_INVISIBLE(13.2), is deliberately unmapped in the public StrEnum
(https://github.com/NVIDIA/cuda-python/blob/c000331de6c37aa4565af74b001271ffcf6d5c99/cuda_core/tests/test_enum_coverage.py#L59-L61)
— this issue should decide their public fate.
Relates to the memory-resource architecture work (#209, #528, #726) — none of which tracks
these APIs.
Underlying C APIs to cover
cuMemGetDefaultMemPool, cuMemGetMemPool, cuMemSetMemPool; CUmemPool_attribute members
ALLOCATION_TYPE, EXPORT_HANDLE_TYPES, HW_DECOMPRESS_ENABLED, LOCATION_ID,
LOCATION_TYPE, MAX_POOL_SIZE; CU_MEM_LOCATION_TYPE_{NONE,INVISIBLE} mapping decision.
Design sketch (draft — needs design-meeting review)
[!IMPORTANT]
Starting point only, not a settled design — review in the cuda.core design meeting.
- Read-only properties on
DeviceMemoryResource(and the host/managed resources) for the new
pool attributes;max_pool_sizemay warrant a setter — TBD. Device.default_memory_resource(and a host-side equivalent) backed by
cuMemGetDefaultMemPool, returning the corresponding memory-resource wrapper.cuMemSetMemPoolis a process-global state mutation — either expose it as an explicit,
loudly-documented function, or deliberately keep it unexposed initially.
Open questions for the meeting:
- Expose the global setter at all (footgun) vs. get-only in the first pass?
- How
(location, allocation type)keying maps onto cuda.core'sDevice/Hostlocation
objects (incl. host-NUMA IDs). - Does
HW_DECOMPRESS_ENABLEDbelong here or with a future decompress feature? - Public StrEnum mapping for
NONE/INVISIBLElocation types.
References
- Driver docs: https://docs.nvidia.com/cuda/cuda-driver-api/
- Found during the CUDA 12.8 → 13.3 bindings vs. cuda.core gap sweep (2026-07-14)
-- Leo's bot
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.