EnzymeAD / EnzymeAD/Reactant.jl

API to initialize directly on device

Open
#1,445 9 comments 0 reactions 0 assignees View on GitHub
Dominant language
Julia
Stars
370
Forks
74
Avg merge
18h 47m
Merged PRs (30d)
30

Description

Love Reactant.jl — an extremely exciting piece of work! 🎉

I was wondering — is there a way to wrap an existing GPU array? Or even just create an empty Reactant array with particular dimensions?

Doing this causes an OOM on my L40S GPU:
```
Reactant.set_default_backend("gpu")
vectors_gpu = CUDA.zeros(Float32, 768, 10_000_000)
vectors_rx = Reactant.to_rarray(vectors_gpu)
```

`vectors_gpu` takes up around 75% of my GPU, which is intended, but `vectors_rx` allocates on both CPU and GPU. It just so happens that I have lots of RAM, so I can do:
```
Reactant.set_default_backend("gpu")
vectors_cpu = zeros(Float32, 768, 10_000_000)
vectors_rx = Reactant.to_rarray(vectors_cpu)
```

However, notably, `Reactant.to_rarray` copies the array. The problem is mitigated, of course, by not maintaining a reference to the source CPU array:
```
Reactant.set_default_backend("gpu")
vectors_rx = zeros(Float32, 768, 10_000_000) |> Reactant.to_rarray
```

However, in either case, it's wasteful to have a mirror of the GPU array on CPU when I really don't need it in my application.

How can I mitigate this problem?

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.