using candle to make inference of yolov8 model is quite slow than PyTorch or onnxruntime!

### Env:
GPU: NVIDIA GeForce RTX 3060, 12036MiB)
CPU: 12th Gen Intel(R) Core(TM) i5-12400F
OS: Ubuntu 23.04
Model: yolov8s.pt, yolov8s.onnx, yolov8s.safetensors

#### speed test on 1000 images:
- candle: ~55ms (use --features cuda & cudnn)
- pytorch(python): ~5.5ms
- ort(rust): ~7ms

I also test dinov2, it is same slow compared with ort(rust) and pytorch.

### question
Why candle is so slow. Is candle fit for deployment(GPU server mainly)?

### candle code snippet
```
...
let image_t = {
let img = original_image.resize_exact(
width as u32,
height as u32,
image::imageops::FilterType::CatmullRom,
);
let data = img.to_rgb8().into_raw();
Tensor::from_vec(
data,
(img.height() as usize, img.width() as usize, 3),
// &Device::Cpu,
&Device::new_cuda(0)?,
)?
.permute((2, 0, 1))?
};
let image_t = (image_t.unsqueeze(0)?.to_dtype(DType::F32)? * (1. / 255.))?;
println!("preprocess: {:?}", std::time::Instant::now() - t0);

let t0 = std::time::Instant::now();
// let predictions = model.forward(&image_t)?.squeeze(0)?;
let predictions = model.forward(&image_t)?;
println!("inference: {:?}", std::time::Instant::now() - t0);

...
```

Start by reproducing the 1000-image benchmark and measuring the separate preprocessing and inference timings around Tensor::from_vec and model.forward. Compare the CUDA and cuDNN configuration with the reported PyTorch and ORT runs; done means the slowdown has a reproducible explanation and a documented deployment recommendation or clearly scoped follow-up.

Written by the indexing model from the issue text.