Make fixed size host block allocations cuda stream aware
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 59
- Forks
- 35
- Avg merge
- 17h 38m
- Merged PRs (30d)
- 3
Description
Issue:
Currently, fixed sized blocks returned by the memory resource are not stream aware. When a block is released, it is added back to the pool immediately, while there are other operations that are queued on a stream that uses those blocks. The released blocks could then be provided to a different allocation on a different stream. This could lead to stream ordering issues.
Expected behavior:
When a set of blocks is allocated, the allocating cuda stream needs to be tracked. And released blocks should only be added back to the pool when the work on those blocks are completed. Maybe we could use a cuda event to indicate a last write.
Ex: rmm fixed sized mr follows this. I have a draft PR that enables block allocations there. https://github.com/rapidsai/rmm/pull/2280
Also, the MR doesnt necessarily have to be "host". Any stream-aware upstream resource would be able to use the fixed size mr.
Some of the work is certainly overlapping. Maybe we should converge? @bdice @felipeblazing
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the fixed-size memory resource and the referenced RMM PR 2280 for the stream-aware allocation approach. Done means tracking the allocating CUDA stream and returning released blocks to the pool only after queued work on them has completed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- backend
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100