huggingface / huggingface/candle
Zero device memory allocation tensor ops support
- Dominant language
- Rust
- Stars
- 21k
- Forks
- 1.8k
- Avg merge
- 16h 42m
- Merged PRs (30d)
- 25
Description
Is there any way to do tensor ops(add/div/matmul...) without memory allocation?
eg:
```
let dst = Tensor::zeros((2, 2), DType::F32, &cuda_dev)?;
let lhs= Tensor::zeros((2, 2), DType::F32, &cuda_dev)?;
let rhs= Tensor::zeros((2, 2), DType::F32, &cuda_dev)?;
candle::add(&lhs,&rhs, &mut dst)?; //zero device memory allocation here
```
Contributor guide
No contributing guide indexed for this repository
Research direction
No source files or tests are identified. Start by tracing CUDA tensor creation and the candle::add operation, then determine how add, div, and matmul could write into a provided destination without device allocation; done means these operations demonstrate zero additional device-memory allocation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100