🐛 [Bug] Encountered bug when using Torch-TensorRT(torch.nn.LSTM)
@narendasan is already working on this.
Since Feb 16, 2024.
- Dominant language
- Python
- Stars
- 3k
- Forks
- 410
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 78
Description
Bug Description
Encountered error as follow when using Torch-TensorRT to convert torch.nn.LSTM in docker image nvcr.io/nvidia/pytorch:23.12-py3 :
NotImplementedError: aten::_cudnn_rnn_flatten_weight: attempted to run this operator with Meta tensors, but there was no abstract impl or Meta kernel registered. You may have run into this message while using an operator with PT2 compilation APIs (torch.compile/torch.export); in order to use this operator with those APIs you'll need to add an abstract impl
To Reproduce
example code:
import torch
import torch_tensorrt
import torch.nn as nn
class Model(nn.Module):
def __init__(self):
super().__init__()
self.lstm = nn.LSTM(1024, 1024, batch_first=True)
def forward(self, x):
x = self.lstm(x)[0]
return x
model = Model().half().eval().cuda()
inputs = [torch.randn(100, 200, 1024).half().cuda()]
trt_gm = torch_tensorrt.compile(model, ir="dynamo", inputs=inputs)
trt_gm(*inputs)
Expected behavior
Environment
Build information about Torch-TensorRT can be found by turning on debug messages
- Torch-TensorRT Version: 2.2.0a0
- PyTorch Version: 2.2.0a0+81ea7a4
- CPU Architecture:
- OS (e.g., Linux): Linux
- How you installed PyTorch (
conda,pip,libtorch, source): - Build command you used (if compiling from source):
- Are you using local sources or building from archives:
- Python version: 3.10.12
- CUDA version: 12.3
- GPU models and configuration: A100
- Any other relevant information:
Additional context
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.