NVIDIA / NVIDIA/cccl

[EPIC] Roadmap for cuda/memory_resource

Open
#1,502 9 comments 1 reaction 2 assignees Claimed by @pciolkosz View on GitHub
cudax libcu++
Dominant language
C++
Stars
2.5k
Forks
486
Avg merge
2d 6h
Merged PRs (30d)
295

Description

`cuda::mr` is intended to be the future of heterogenous memory allocation in CUDA C++. It is inspired heavily by lessons learned in RMM and our experience with the `device_memory_resource*` and friends. `cuda::mr` does not seek to replace RMM, but instead distill and standardize the best parts of RMM into a more central location. Furthermore, RMM is already in the process of rebasing on top of using the `cuda::mr` interface.

### Scheduled for 3.1:
- `[synchronous_]resource` concepts
- Property system

### Scheduled for 3.2:
- [x] https://github.com/NVIDIA/cccl/issues/2128
- [x] https://github.com/NVIDIA/cccl/issues/2130
- [x] https://github.com/NVIDIA/cccl/issues/6097
- [x] https://github.com/NVIDIA/cccl/issues/2131
- [x] https://github.com/NVIDIA/cccl/issues/6009
- [x] https://github.com/NVIDIA/cccl/issues/6177
- [x] Type erased memory resource wrappers

### Future:
- [ ] https://github.com/NVIDIA/cccl/issues/2129
- [ ] https://github.com/NVIDIA/cccl/issues/2143
- [ ] https://github.com/NVIDIA/cccl/issues/2132
- [ ] https://github.com/NVIDIA/cccl/issues/2474
- [ ] https://github.com/NVIDIA/cccl/issues/2142

### Misc
- [ ] Add NVTX annotations to all memory resources
- [x] https://github.com/NVIDIA/cccl/issues/2313
- [x] https://github.com/NVIDIA/cccl/issues/2891

### Concrete types that satisfy the C++ allocator requirements
- [ ] A `cuda::mr::allocator` capable of preserving concrete type of the resource (no type-erasure)
- [ ] A `cuda::mr::polymorphic_allocator` constructible from a `resource_ref`

Questions we'll need to answer along the way:
- What lifetime semantics do we want to use for resources + allocators + data structures?
- RMM took a very relaxed approach of using non-owning references everywhere, but this is worth reconsidering (see https://github.com/rapidsai/rmm/issues/1492
- Do all data structures only take Allocators? Or just resources? Both?
- In RMM, we took an approach of only constructing from resource_refs directly, but this was mostly for expediency and convenience, so it is worth reconsidering.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.