JuliaGPU / JuliaGPU/CUDA.jl

Simplify memory allocator with UVA

Open
#563 3 comments 0 reactions 0 assignees View on GitHub
cuda libraries enhancement
Dominant language
Julia
Stars
1.4k
Forks
281
Avg merge
1d 7h
Merged PRs (30d)
30

Description

We currently keep track of which device owns each GPU allocation, but that's not necessary. Since CUDA 4 we have unified virtual addressing for sm_20+ devices on 64 bit, so we should use that to get rid of the `PerDevice` accounting wherever possible.

- https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#unified-virtual-address-space
- https://developer.nvidia.com/blog/unified-memory-in-cuda-6/ (section 'Unified Memory or Unified Virtual Addressing')

PerDevice has other uses though, namely to kick allocations out of the pool when resetting the device. But since that's a rare operation, maybe we should do that in a slower way (e.g. by identifying devices using getPointerAttributes).

Some more details in this webinar: https://developer.download.nvidia.com/CUDA/training/cuda_webinars_GPUDirect_uva.pdf (@vchuravy suggests based on it that we should enable P2P transfers whenever possible)

Finally, looking at the code I noticed that we maybe have to switch contexts to free allocations, and this may be the cause for some `ptr not found in allocated` errors @jpsamaroo was running into. Regardless of whether we decide this is the user's responsibility for now, we could use the above getPointerAttributes to check if the context matches in some debug mode.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.