microsoft / microsoft/onnxruntime
onnxruntime slower than xgboost & lightgbm in batch predictions
@xadupre is already working on this.
Since Mar 10, 2020.
- Dominant language
- C++
- Stars
- 21.9k
- Forks
- 4.2k
- Avg merge
- 4d 11h
- Merged PRs (30d)
- 184
Description
Have a xgboost and a lightgbm classifier model.
Converted both to ONNX Model format and then onnx model read using onnxruntime
onnxruntime (onnx model) for batch prediction is approx 5 to 10 times slower than the respective xgboost/lightgbm models in case of batch predictions.
Took batch size of 150 for predictions and with 21 numeric features.
Runtimes (in milliseconds)
|XGBoost(n_estimators=500)|ONNXRuntime|OS|
|:---------:|:-----:|:-----:|
|1.7 | 13.5 | macOS mojave|
|1.77| 25 | Linux Ubuntu 16.04|
|LightGBM(n_estimators=1000) |ONNXRuntime|OS|
|:---------:|:-----:|:-----:|
|4.94 | 22.5 | macOS mojave|
|1.77| 25 | Linux Ubuntu 16.04|
**System information**
- OS Platform and Distribution: Tested both on Mac and Ubuntu 16.04.5
- ONNX Runtime installed from: binary (using pip)
- ONNX Runtime version: 0.3.0
- Python version: 3.6
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.