NVIDIA / NVIDIA/cuCascade

Make fixed size host block allocations cuda stream aware

Open
#93 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

rapidsmpf request
Dominant language
C++
Stars
59
Forks
35
Avg merge
17h 38m
Merged PRs (30d)
3

Description

Issue:

Currently, fixed sized blocks returned by the memory resource are not stream aware. When a block is released, it is added back to the pool immediately, while there are other operations that are queued on a stream that uses those blocks. The released blocks could then be provided to a different allocation on a different stream. This could lead to stream ordering issues.

Expected behavior:

When a set of blocks is allocated, the allocating cuda stream needs to be tracked. And released blocks should only be added back to the pool when the work on those blocks are completed. Maybe we could use a cuda event to indicate a last write.

Ex: rmm fixed sized mr follows this. I have a draft PR that enables block allocations there. https://github.com/rapidsai/rmm/pull/2280

Also, the MR doesnt necessarily have to be "host". Any stream-aware upstream resource would be able to use the fixed size mr.

Some of the work is certainly overlapping. Maybe we should converge? @bdice @felipeblazing

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the fixed-size memory resource and the referenced RMM PR 2280 for the stream-aware allocation approach. Done means tracking the allocating CUDA stream and returning released blocks to the pool only after queued work on them has completed.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.