JuliaGPU / JuliaGPU/CUDA.jl

Explore using IPC for serialization?

Open
#747 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Julia
Stars
1.4k
Forks
281
Avg merge
1d 7h
Merged PRs (30d)
30

Description

This was discussed on Slack, so reposting here so it doesn't get lost. Using IPC to transfer GPU memory pointers between processes, which can also facilitate D2D copies between GPUs on the same host, seems exceedingly easy:

```julia
using Distributed
addprocs(1, exeflags="--project=@.")
@everywhere using CUDA
handle = @fetchfrom 2 begin
x = cu([1,2,3])
handle = Ref{CUDA.CUipcMemHandle}()
CUDA.cuIpcGetMemHandle(handle, pointer(x))
handle
end
buf = Ref{CUDA.CUdeviceptr}()
CUDA.cuIpcOpenMemHandle(buf, handle[], CUDA.CU_IPC_MEM_LAZY_ENABLE_PEER_ACCESS)
remote_x = unsafe_wrap(CuArray{Int,1}, convert(CuPtr{Int}, buf[]), (3,); own=false)
println(remote_x)
```

It might be interesting to explore an option where CUDA's serialization for Distributed distribution uses IPC. The idea is that serialization just gives the IPC handle, then the deserializer opens the handle and immediately makes a copy of the memory. This would make memory transfers D2D rather than the existing D2H2D. There's probably a small possibility that the original process frees the memory before the receiving end can make a copy, although this seems unlikely given typical usage (e.g. in a `pmap`, where the original process wouldn't be executing more code which might free the memory until the whole thing returns). In general, I'm imagining some option like `JULIA_CUDA_IPC_SERIALIZATION=1` which switches to this, and should work transparently with any existing code, assuming (based on user guarantee) that all the distributed workers are on the same host.

This may well be too niche and maybe its just best to wait for UCX.jl, but just documenting here for posterity.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.