why the yolov8 int8 quant using pytorch_quant is slower than trt --fp16 quant
Open
Nobody has claimed this yet.
triaged
- Dominant language
- C++
- Stars
- 13.4k
- Forks
- 2.4k
- Avg merge
- 5d 3h
- Merged PRs (30d)
- 2
Description
devicec : nvidia NX
1.using trt --fp16
/usr/src/tensorrt/bin/trtexec --onnx=best.onnx --workspace=4096 --saveEngine=best.engine --fp16
the result of infer speed is 36.8ms
2. using pytorch_quant int8
/usr/src/tensorrt/bin/trtexec --onnx=best.onnx --saveEngine=v8s_ptq.engine --int8 --workspace=4096
the result of infer speed is : 39.5ms
### Tasks
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the two /usr/src/tensorrt/bin/trtexec commands on the NVIDIA NX with best.onnx and the stated workspace settings. Compare the generated FP16 and INT8 engine behavior and document why the INT8 engine measures 39.5 ms versus 36.8 ms, including the conditions needed for a reliable comparison.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- pytorch
- Domain
- machine-learning, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100