NVIDIA / NVIDIA/raft

[FEA] Changes to mdarray & mdspan to support future CPU paths

Open
#656 3 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

feature request inactive-30d
Dominant language
Cuda
Stars
1k
Forks
251
Avg merge
1d 8h
Merged PRs (30d)
31

Description

The following tasks should bring us closer to being able to support CPU code paths in cuML, though these changes should be useful to users in general:

  • Ability to build against mdarray and mdspan headers without CUDA. I'm assuming once we have some CPU-enabled primitives within RAFT then this could also apply to those headers. The way this is achieved in Triton currently is through a preprocessor variable.
  • We should remove the requirement that raft functions accept explicit accessor policies directly. Instead, they should just accept mdspan itself.
  • Add support for tree-style layout indexing patterns, in addition to other potential layouts.
  • Owning facility that can better unify the allocation and indexing of host and device memory.

The goal is to have the mdspan itself not necessarily care about the fact that the memory underneath is device, host, pinned host, or managed (or even registers or shared memory) and to have the underlying owning structure / accessor handle the requests accordingly.

One use-case we have for FAISS integration, for example, is being given a large chunk of CPU memory as input and moving only smaller chunks or slices to the GPU. This would require the accessor of mdspan and the container policy / allocator of the underlying mdarray to be smart enough to know that gpu memory is desired, and to create a new allocation on the GPU and copy from the CPU. I'm not sure if this is something a submdspan alone would be able to handle?

@trivialfis and @wphicks have both suggested wrapping the mdarray in a more unified owning object which could then produce an mdspan. It seems like this might ultimately be the way to go, but I think there's still some details to iron out, such as the example above.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the mdarray and mdspan headers, their accessor policies, and the existing allocation or ownership paths. The issue lists several possible directions; completion would require an agreed design covering non-CUDA builds, alternate layouts, and unified host/device ownership and transfers.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
backend, performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.