huggingface / huggingface/candle

Slow YOLOv8 using MKL

Open
#2,281 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
21k
Forks
1.8k
Avg merge
16h 42m
Merged PRs (30d)
25

Description

I had great success using MKL for some models, especially BERT-likes with huge improvements in speed (up to x25). Here the speedup is only of about 1 second.
However, I'm having troubles with the YOLOv8-X model which is very slow (same speed as PyTorch) and about 6 times slower than Onnx.
Are operations in YOLO not supported by MKL ?

Note that I'm using a fine tuned model, with bigger images so I expect it to be slow but I was expecting MKL to perform well. The same is happening with the provided example in the README.
Also, I don't see any improvements with the accelerate feature on my M2.

Thanks for you help

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the YOLOv8 example in the README and reproduce its timing with MKL and the accelerate feature on the M2, comparing the PyTorch and ONNX baselines described here. Done means determining whether specific YOLO operations lack MKL support and identifying a concrete next step.

Written by the indexing model from the issue text.

Assessment

Tech stack
pytorch
Domain
machine-learning, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.