No MHA (muti head attention) kernal is called in Tensorrt 10.3 in Orin with Jetpack 6.1
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 13.4k
- Forks
- 2.4k
- Avg merge
- 5d 3h
- Merged PRs (30d)
- 2
Description
Description
Use exact ONNX file attention_ln_opset13.onnx from https://github.com/NVIDIA/TensorRT/issues/3575#issuecomment-1874776406
Attention is like
When I use Tensorrt 10.3 with JetPack 6.1, command is trtexec --onnx=attention_ln_opset13.onnx --int8 --saveEngine=default_int8.engine, the nsys as follows doesn't use MHA kernel.
- default_int8.engine
- default_int8.nsys-rep
Jetpack info
Package: nvidia-jetpack
Source: nvidia-jetpack (6.1)
Version: 6.1+b123
Architecture: arm64
Maintainer: NVIDIA Corporation
Installed-Size: 194
Depends: nvidia-jetpack-runtime (= 6.1+b123), nvidia-jetpack-dev (= 6.1+b123)
Homepage: http://developer.nvidia.com/jetson
Priority: standard
Section: metapackages
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the report with attention_ln_opset13.onnx using the provided trtexec command on TensorRT 10.3 with JetPack 6.1. Inspect default_int8.nsys-rep and default_int8.engine from nsys_engine.zip, then compare the observed execution with the reported absence of an MHA kernel. Done means explaining whether this kernel selection is expected or identifying the cause of the discrepancy.
Written by the indexing model from the issue text.
Assessment
- Domain
- embedded-iot, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100