huggingface / huggingface/candle

Tensor repeate is too slow (cpu & cuda)

Open
#1,623 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
21.1k
Forks
1.8k
Avg merge
16h 42m
Merged PRs (30d)
25

Description

```rust
#[test]
fn test_repeate() -> candle_core::Result<()> {
let a = Tensor::new(&[1.0], &Device::Cpu)?;
let a = a.reshape((1, 1))?;
let start = std::time::Instant::now();
let a = a.repeat((1, 32000))?;
println!(
"cpu tensor repeat to {:?} cost {:?}",
a.shape(),
start.elapsed()
);

let device = Device::new_cuda(0)?;

let b = Tensor::new(&[1.0], &device)?;
let b = b.reshape((1, 1))?;
let start = std::time::Instant::now();
let b = b.repeat((1, 32000))?;
println!(
"cuda tensor repeat to {:?} cost {:?}",
a.shape(),
start.elapsed()
);
Ok(())
}
```
rust result
```
running 1 test
cpu tensor repeat to [1, 32000] cost 51.383172ms
cuda tensor repeat to [1, 32000] cost 311.401749ms
test model_executor::ops::test::test_repeate ... ok
```

with pytorch
```python
#!/usr/bin/python3

import torch
import time

a = torch.rand(1, 1,dtype=torch.float32)
start_time = time.time()
a = a.repeat(1,32000)
print("cpu repeate cost ", time.time() - start_time, "s")

a=torch.rand(1, 1,dtype=torch.float32, device = 'cuda')
start_time = time.time()
a = a.repeat(1,32000)
print("cuda repeate cost ", time.time() - start_time, "s")
```
python result
```
cpu repeate cost 0.00014472007751464844 s
cuda repeate cost 0.004861593246459961 s
```

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reproducing the issue with the shown Rust test and inspect the Tensor::repeat implementation and its CPU and CUDA paths. Compare the timings and behavior with the provided PyTorch examples; done means repeat is substantially faster on both devices without changing the resulting shape or values.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
machine-learning, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.