huggingface / huggingface/candle
Slow YOLOv8 using MKL
- Dominant language
- Rust
- Stars
- 21k
- Forks
- 1.8k
- Avg merge
- 16h 42m
- Merged PRs (30d)
- 25
Description
I had great success using MKL for some models, especially BERT-likes with huge improvements in speed (up to x25). Here the speedup is only of about 1 second.
However, I'm having troubles with the YOLOv8-X model which is very slow (same speed as PyTorch) and about 6 times slower than Onnx.
Are operations in YOLO not supported by MKL ?
Note that I'm using a fine tuned model, with bigger images so I expect it to be slow but I was expecting MKL to perform well. The same is happening with the provided example in the README.
Also, I don't see any improvements with the accelerate feature on my M2.
Thanks for you help
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the YOLOv8 example in the README and reproduce its timing with MKL and the accelerate feature on the M2, comparing the PyTorch and ONNX baselines described here. Done means determining whether specific YOLO operations lack MKL support and identifying a concrete next step.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- pytorch
- Domain
- machine-learning, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100