microsoft / microsoft/onnxruntime

[Documentation] When is grpah optimisation applied?

Open
#13,558 0 comments 0 reactions 0 assignees View on GitHub
documentation ep:CUDA ep:TensorRT
Dominant language
C++
Stars
21.9k
Forks
4.2k
Avg merge
4d 11h
Merged PRs (30d)
184

Description

### Describe the documentation issue

Looking at the documentation, it is not clear to me when Graph Optimisations are applied depending on the backend. There are no restrictions for the basic level; for extended, it is limited to CUDA, ROCm, and CPU (simplifying here).

What is the behaviour when using DML or TensorRT EP?
- TensorRT: Comes with its graph optimiser and therefore would not need the ONNX graph optimiser. Is it, therefore, disabled automatically, or does this has to be handled by the end user? Especially when using TRT cache, it does load a compiled engine plan; Does it still run optimisations on the ONNX file if not disabled?
- DML is not listed in the supported EPs. Does applying Basic/Extended optimisations when running on DML makes a difference? If no, again, is it disabled by default?
- General: Do you, by any chance, have benchmarks outlying how much of a performance difference it makes to apply these optimisations for a given EP?

I know that this is more of a question, but it would be much appreciated if you could give me an answer on this, as going through the code to find out these things is not the easiest task 😅

### Page / URL

https://onnxruntime.ai/docs/performance/graph-optimizations.html

Contributor guide

Open the contributing guide

Research direction

Start with the graph optimizations page linked in the issue. Determine and document the behavior for TensorRT and DML execution providers, including whether optimization is automatic, how cached TensorRT plans affect it, and whether benchmark evidence is available; done means these questions are answered clearly in the documentation.

Written by the indexing model from the issue text.

Assessment

Domain
documentation, machine-learning
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.