huggingface / huggingface/candle
Tensor repeate is too slow (cpu & cuda)
- Dominant language
- Rust
- Stars
- 21.1k
- Forks
- 1.8k
- Avg merge
- 16h 42m
- Merged PRs (30d)
- 25
Description
```rust
#[test]
fn test_repeate() -> candle_core::Result<()> {
let a = Tensor::new(&[1.0], &Device::Cpu)?;
let a = a.reshape((1, 1))?;
let start = std::time::Instant::now();
let a = a.repeat((1, 32000))?;
println!(
"cpu tensor repeat to {:?} cost {:?}",
a.shape(),
start.elapsed()
);
let device = Device::new_cuda(0)?;
let b = Tensor::new(&[1.0], &device)?;
let b = b.reshape((1, 1))?;
let start = std::time::Instant::now();
let b = b.repeat((1, 32000))?;
println!(
"cuda tensor repeat to {:?} cost {:?}",
a.shape(),
start.elapsed()
);
Ok(())
}
```
rust result
```
running 1 test
cpu tensor repeat to [1, 32000] cost 51.383172ms
cuda tensor repeat to [1, 32000] cost 311.401749ms
test model_executor::ops::test::test_repeate ... ok
```
with pytorch
```python
#!/usr/bin/python3
import torch
import time
a = torch.rand(1, 1,dtype=torch.float32)
start_time = time.time()
a = a.repeat(1,32000)
print("cpu repeate cost ", time.time() - start_time, "s")
a=torch.rand(1, 1,dtype=torch.float32, device = 'cuda')
start_time = time.time()
a = a.repeat(1,32000)
print("cuda repeate cost ", time.time() - start_time, "s")
```
python result
```
cpu repeate cost 0.00014472007751464844 s
cuda repeate cost 0.004861593246459961 s
```
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reproducing the issue with the shown Rust test and inspect the Tensor::repeat implementation and its CPU and CUDA paths. Compare the timings and behavior with the provided PyTorch examples; done means repeat is substantially faster on both devices without changing the resulting shape or values.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- machine-learning, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100