EnzymeAD / EnzymeAD/Reactant.jl
API to initialize directly on device
- Dominant language
- Julia
- Stars
- 370
- Forks
- 74
- Avg merge
- 18h 47m
- Merged PRs (30d)
- 30
Description
Love Reactant.jl — an extremely exciting piece of work! 🎉
I was wondering — is there a way to wrap an existing GPU array? Or even just create an empty Reactant array with particular dimensions?
Doing this causes an OOM on my L40S GPU:
```
Reactant.set_default_backend("gpu")
vectors_gpu = CUDA.zeros(Float32, 768, 10_000_000)
vectors_rx = Reactant.to_rarray(vectors_gpu)
```
`vectors_gpu` takes up around 75% of my GPU, which is intended, but `vectors_rx` allocates on both CPU and GPU. It just so happens that I have lots of RAM, so I can do:
```
Reactant.set_default_backend("gpu")
vectors_cpu = zeros(Float32, 768, 10_000_000)
vectors_rx = Reactant.to_rarray(vectors_cpu)
```
However, notably, `Reactant.to_rarray` copies the array. The problem is mitigated, of course, by not maintaining a reference to the source CPU array:
```
Reactant.set_default_backend("gpu")
vectors_rx = zeros(Float32, 768, 10_000_000) |> Reactant.to_rarray
```
However, in either case, it's wasteful to have a mirror of the GPU array on CPU when I really don't need it in my application.
How can I mitigate this problem?
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.