mlc-ai / mlc-ai/web-stable-diffusion
Huge performance gap between TVM and TRT on Stable Diffusion v1.5
Nobody has claimed this yet.
- Dominant language
- Jupyter Notebook
- Stars
- 3.7k
- Forks
- 238
- PR merge metrics
- No merged PRs in 30d
Description
GPU: Nvidia RTX 3090TI.
- Firstly, I use the log db in the repo, it gives me 3.7s to get the result.
- Then, I tried to tuning myself using meta-schedule(with trial count set to 50,000), it gives me 2.5s.
But, on TensorRT v8.6, for one iteration of unet, it gives me only 25ms, rather than 96ms with TVM(USE_CUBLAS =ON ; USE_CUDNN =ON; CUDA Version 12.1)
I wonder why the latency gap of stable diffusion model is so huge between TVM and TensorRT.
BTW, a few weeks ago, I got a different result between TVM and TRT, where my in-house model auto-tuned by TVM performs a wonderful infer latency (almost nearby TensorRT8.5).
Do you have any ideas about it? Thanks advance.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the repository's log database and reproduce the reported Stable Diffusion v1.5 timings on the RTX 3090 Ti. Compare the meta-schedule result with TensorRT's UNet iteration measurement under the stated CUDA, cuBLAS, and cuDNN settings. Done means identifying and documenting the cause of the latency gap with a repeatable comparison.
Written by the indexing model from the issue text.
Assessment
- Domain
- machine-learning, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100