huggingface / huggingface/candle

Zero device memory allocation tensor ops support

Open
#1,634 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
21k
Forks
1.8k
Avg merge
16h 42m
Merged PRs (30d)
25

Description

Is there any way to do tensor ops(add/div/matmul...) without memory allocation?
eg:
```
let dst = Tensor::zeros((2, 2), DType::F32, &cuda_dev)?;
let lhs= Tensor::zeros((2, 2), DType::F32, &cuda_dev)?;
let rhs= Tensor::zeros((2, 2), DType::F32, &cuda_dev)?;
candle::add(&lhs,&rhs, &mut dst)?; //zero device memory allocation here
```

Contributor guide

No contributing guide indexed for this repository

Research direction

No source files or tests are identified. Start by tracing CUDA tensor creation and the candle::add operation, then determine how add, div, and matmul could write into a provided destination without device allocation; done means these operations demonstrate zero additional device-memory allocation.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.