huggingface / huggingface/candle
Transfering CUDA tensors between candle and tch-rs without synchronization
- Dominant language
- Rust
- Stars
- 21k
- Forks
- 1.8k
- Avg merge
- 16h 42m
- Merged PRs (30d)
- 25
Description
Is there a way to transfer CUDA tensors between candle and tch-rs without requiring GPU/CPU synchronisation?
I am doing reinforcement learning and currently I have a ResNet that I am training in tch-rs. I have found that the main bottleneck of my implementation is GPU/CPU synchronization when creating observations of the environment. To avoid this I want to try using candle to reimplement the environment to keep everything on the GPU. Before I can do that I need to figure out whether I can send CUDA tensors from candle to a network implemented in tch-rs.
---
Related questions that you may also be able to answer: How would I send tensors between candle and pytorch+jax? What about tch-rs and pytorch+jax?
Contributor guide
No contributing guide indexed for this repository
Research direction
The issue names no files, tests, or entry points. Start by investigating the tensor interoperability APIs for candle, tch-rs, CUDA, PyTorch, and JAX; done would require a clearly documented, supported way to transfer tensors without GPU/CPU synchronization, if one is feasible.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- pytorch, rust
- Domain
- machine-learning, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100