The CustomQKVToContextPluginDynamic plugin uses unsatisfactory results
@ttyio is already working on this.
Since May 18, 2023.
- Dominant language
- C++
- Stars
- 13.4k
- Forks
- 2.4k
- Avg merge
- 5d 3h
- Merged PRs (30d)
- 2
Description
Description
I was trying to use the CustomQKVToContextPluginDynamic plugin plugin and found incorrect results. I don't know what caused it.
The input I gave is as follows:
q_tensor [4 4 4 4] [S B E]
k_tensor [6 6 6 6] [S B E]
v_tensor [4 4 4 4] [S B E]
The result of the plugin is not the same as Torch.Is there anything wrong with my operation?
The correct result returned should be [4 4 4 4]
But what the plugin returns is
trt: [[[[[6.]]
[[4.]]
[[6.]]
[[6.]]]]]
Environment
TensorRT Version 8.4.11:
NVIDIA GPU A40:
NVIDIA Driver Version:
CUDA Version:
CUDNN Version:
Operating System:
Python Version (if applicable) 3.9:
Tensorflow Version (if applicable):
PyTorch Version (if applicable) 2.0:
The code is as follows:
https://drive.google.com/file/d/1zVklW5pBDfgEVMsBzsxS5kBfTMm1oLJg/view?usp=sharing
https://drive.google.com/file/d/1c5vSCeho2GSnxVel-fNHzh7T_3jL4twF/view?usp=sharing
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.