❓ [Question] a10 performance drop significantly
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 3k
- Forks
- 410
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 78
Description
❓ Question
I converted the gfpgan model (https://github.com/TencentARC/GFPGAN) with torch_tensorrt, and I found torch_tensorrt is twice as fast as torch in 3070. But in one a10 server, torch_tensorrt and torch are closed; In other a10 server, torch_tensorrt is even twice as slow as torch. Statics shows below. （two type of a10 from two difference cloud server）.
| GPU | CPU | CPU core | CPU freq | memory | inference framework | CPU usage | memory usage | GPU usage | inference time |
|---|---|---|---|---|---|---|---|---|---|
| 3070 | AMD Ryzen 7 5800X 8-Core Processor | 16 | 2200-3800MHz | 32G | pytorch | 30-35% | 160-170% | 13.5g 987.7m | 33.889511s |
| 3070 | torch_tensorrt | 15-20% | 180-200% | 11.7g 1.1g | 16.259879s | ||||
| a10（v1） | Intel (R) Xeon (R) Platinum 8350C CPU @ 2.60GHz | 28 | 2593MHz | 112G | pytorch | 25-30% | 190-200% | 15.1g 1.2g | 33.933190s |
| a10（v1） | torch_tensorrt | 15-20% | 190-200% | 13.0g 1.2g | 31.899047s | ||||
| a10（v2） | Intel(R) Xeon(R) Platinum 8336C CPU @ 2.30GHz | 28 | 2300-4600MHz | 112G | pytorch | 20-30% | 180-200% | 15.1g 1.0g | 34.027398s |
| a10（v2） | torch_tensorrt | 10-15% | 160-170% | 13.1g 1.1g | 66.498723s |
I also tried torch2trt(https://github.com/NVIDIA-AI-IOT/torch2trt) and fixed some op error, finding it's twice as fast as torch_tensorrt in 3070. And performance didn't drop so strangely in a10 server.
Environment
Build information about Torch-TensorRT can be found by turning on debug messages
- PyTorch Version (e.g., 1.0): nvcr.io/nvidia/pytorch:23.08-py3
- CPU Architecture: as above
- OS (e.g., Linux): linux
- How you installed PyTorch (
conda,pip,libtorch, source): docker - Build command you used (if compiling from source):
- Are you using local sources or building from archives:
- Python version:
- CUDA version:
- GPU models and configuration: as above
- Any other relevant information:
Additional context
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The report names no repository files, tests, or entry points. Reproduce the GFPGAN conversion with torch_tensorrt on both A10 configurations, collect the missing CUDA, Python, build, and debug details, and compare against PyTorch and torch2trt. Done means a maintainer-confirmed cause or a minimal reproducible performance report.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, python, pytorch
- Domain
- machine-learning, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 15/100