apache / apache/arrow-rs

Support the C Device Data Interface (`ArrowDeviceArray`, `ArrowDeviceArrayStream`) over FFI

Open
#10,752 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
3.6k
Forks
1.3k
Avg merge
2d 18h
Merged PRs (30d)
169

Description

### Is your feature request related to a problem or challenge?

arrow-rs cannot receive or hand out an Arrow array whose buffers are not in host memory.
`arrow-array::ffi` binds `ArrowArray`, `ArrowSchema` and `ArrowArrayStream`; there is no binding
for `ArrowDeviceArray` or `ArrowDeviceArrayStream` from the
[C Device Data Interface](https://arrow.apache.org/docs/format/CDeviceDataInterface.html).

So a Rust crate that does own device memory has no shared struct to pass through. It either
defines its own `#[repr(C)]` copy of `ArrowDeviceArray` — and two such copies are incompatible
for no reason — or falls back to the host interface and copies. Producers on the other side of
the boundary already exist: Arrow C++, nanoarrow and pyarrow all implement the device interface,
and cuDF exports through it.

This came up as a blocker moving a GPU-resident batch between two Rust libraries: neither could
name what it was holding in a way the other would accept.

### Describe the solution you'd like

Bind the two sync structs, and implement export and import for CPU-resident data only:

- `ArrowDeviceType` and `FFI_ArrowDeviceArray` in `arrow-data`, next to `FFI_ArrowArray`
- `FFI_ArrowDeviceArrayStream` plus a `RecordBatchReader` importer in `arrow-array`, mirroring
`FFI_ArrowArrayStream` / `ArrowArrayStreamReader`
- `to_device_ffi` / `from_device_ffi`, where import rejects any `device_type` other than
`ARROW_DEVICE_CPU` with an error rather than reading a device pointer as host memory
- public constructors so a crate that does own device memory can supply its own buffers and
callbacks

Not proposed: any CUDA or vendor dependency, device allocation, copying, or synchronisation.
`sync_event` would be carried as an opaque pointer and never dereferenced. arrow-rs would gain
the ability to speak the interface and to carry a device array's metadata faithfully; the device
half stays in the crates that already manage that memory.

This is testable without a GPU: field offsets against `abi.h`, a CPU round trip, and assertions
that a non-CPU `device_type` is refused.

### Describe alternatives you've considered

- A `#[repr(C)]` struct in each downstream crate, which is what happens today. Two crates that
both do it cannot pass an array to each other, which is the point of a C ABI.
- Waiting for device-aware kernels. That is the larger question in #7618; the struct binding is
separable, and is what lets a device array reach a kernel outside arrow-rs.
- The async device stream interface, which needs this first — see below.

### Additional context

Two prior threads point at this:

- #7618, on donating `arrow-gpu`, where @felipecrv notes that most GPU kernels could live outside
arrow-rs but "`arrow-rs` would at least be able to receive arrays stored in GPU memory and pass
them to kernels that can process them in the GPU".
- #7228, where @wjones127 notes the ecosystem need and points at the
[async device stream interface](https://arrow.apache.org/docs/format/CDeviceDataInterface.html#async-device-stream-interface),
saying "being able to accept these over FFI would be welcome".

The async interface is out of scope here and depends on this: `ArrowAsyncTask::extract_data`
writes into a `struct ArrowDeviceArray*`, so it cannot be bound in Rust until that struct exists.
It also raises the `futures` dependency question #7228 discusses, which the sync structs do not.

`device_id` for CPU data needs no decision here. apache/arrow#40801, resolved by
apache/arrow#41101, added a recommendation of -1 for device types with no intrinsic device
identifier. Arrow C++ and pyarrow 25.0.1 both export -1; nanoarrow uses 0. Exporting -1 and
accepting either on import matches what is deployed.

I have a branch implementing the above and will open a PR against this issue.

Assisted-by: Claude Opus 5 (claude-opus-5[1m]) via Claude Code 2.1.233

Contributor guide

Open the contributing guide

Research direction

Start in arrow-data beside FFI_ArrowArray and in arrow-array beside FFI_ArrowArrayStream and ArrowArrayStreamReader. Check field offsets against abi.h, then cover a CPU round trip and verify that non-CPU device_type values are rejected; keep sync_event opaque and leave the async interface out of scope.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, rust
Domain
data
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.