Explore using IPC for serialization?
- Dominant language
- Julia
- Stars
- 1.4k
- Forks
- 281
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 30
Description
This was discussed on Slack, so reposting here so it doesn't get lost. Using IPC to transfer GPU memory pointers between processes, which can also facilitate D2D copies between GPUs on the same host, seems exceedingly easy:
```julia
using Distributed
addprocs(1, exeflags="--project=@.")
@everywhere using CUDA
handle = @fetchfrom 2 begin
x = cu([1,2,3])
handle = Ref{CUDA.CUipcMemHandle}()
CUDA.cuIpcGetMemHandle(handle, pointer(x))
handle
end
buf = Ref{CUDA.CUdeviceptr}()
CUDA.cuIpcOpenMemHandle(buf, handle[], CUDA.CU_IPC_MEM_LAZY_ENABLE_PEER_ACCESS)
remote_x = unsafe_wrap(CuArray{Int,1}, convert(CuPtr{Int}, buf[]), (3,); own=false)
println(remote_x)
```
It might be interesting to explore an option where CUDA's serialization for Distributed distribution uses IPC. The idea is that serialization just gives the IPC handle, then the deserializer opens the handle and immediately makes a copy of the memory. This would make memory transfers D2D rather than the existing D2H2D. There's probably a small possibility that the original process frees the memory before the receiving end can make a copy, although this seems unlikely given typical usage (e.g. in a `pmap`, where the original process wouldn't be executing more code which might free the memory until the whole thing returns). In general, I'm imagining some option like `JULIA_CUDA_IPC_SERIALIZATION=1` which switches to this, and should work transparently with any existing code, assuming (based on user guarantee) that all the distributed workers are on the same host.
This may well be too niche and maybe its just best to wait for UCX.jl, but just documenting here for posterity.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.