microsoft / microsoft/onnxruntime
[Documentation] When is grpah optimisation applied?
- Dominant language
- C++
- Stars
- 21.9k
- Forks
- 4.2k
- Avg merge
- 4d 11h
- Merged PRs (30d)
- 184
Description
### Describe the documentation issue
Looking at the documentation, it is not clear to me when Graph Optimisations are applied depending on the backend. There are no restrictions for the basic level; for extended, it is limited to CUDA, ROCm, and CPU (simplifying here).
What is the behaviour when using DML or TensorRT EP?
- TensorRT: Comes with its graph optimiser and therefore would not need the ONNX graph optimiser. Is it, therefore, disabled automatically, or does this has to be handled by the end user? Especially when using TRT cache, it does load a compiled engine plan; Does it still run optimisations on the ONNX file if not disabled?
- DML is not listed in the supported EPs. Does applying Basic/Extended optimisations when running on DML makes a difference? If no, again, is it disabled by default?
- General: Do you, by any chance, have benchmarks outlying how much of a performance difference it makes to apply these optimisations for a given EP?
I know that this is more of a question, but it would be much appreciated if you could give me an answer on this, as going through the code to find out these things is not the easiest task 😅
### Page / URL
https://onnxruntime.ai/docs/performance/graph-optimizations.html
Contributor guide
Research direction
Start with the graph optimizations page linked in the issue. Determine and document the behavior for TensorRT and DML execution providers, including whether optimization is automatic, how cached TensorRT plans affect it, and whether benchmark evidence is available; done means these questions are answered clearly in the documentation.
Written by the indexing model from the issue text.
Assessment
- Domain
- documentation, machine-learning
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100