pytorch / pytorch/TensorRT

🐛 [Bug] Flux perf scripts issue

Open
#3,601 0 comments 0 reactions 1 assignee View on GitHub

@cehongwang is already working on this.

Since Jun 17, 2025.

bug story: LLM & Generative AI
Dominant language
Python
Stars
3k
Forks
410
Avg merge
3d 18h
Merged PRs (30d)
78

Description

Bug Description

1) Add a README,md with clear instructions on how to run this. Also add `pip install gradio` or any other deps required to the README.  The instructions need to cover all cases (with and without DS)

2) Fix parse_args import error 

3)  Fix the following error 

Error : File "/root/.pyenv/versions/3.11.13/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1767, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.pyenv/versions/3.11.13/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1778, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/work/TensorRT/py/torch_tensorrt/dynamo/runtime/_PythonTorchTensorRTModule.py", line 714, in forward
    return run_standard_execution()
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/work/TensorRT/py/torch_tensorrt/dynamo/runtime/_PythonTorchTensorRTModule.py", line 460, in run_standard_execution
    self.setup_input_tensors(
  File "/work/TensorRT/py/torch_tensorrt/dynamo/runtime/_PythonTorchTensorRTModule.py", line 374, in setup_input_tensors
    contiguous_inputs[i].dtype == self.input_dtypes[i]
AssertionError: Dtype mismatch for 0th input(timestep). Expect dtype.f16, got torch.float32

To Reproduce

Steps to reproduce the behavior:

Expected behavior

Environment

Build information about Torch-TensorRT can be found by turning on debug messages

  • Torch-TensorRT Version (e.g. 1.0.0):
  • PyTorch Version (e.g. 1.0):
  • CPU Architecture:
  • OS (e.g., Linux):
  • How you installed PyTorch (conda, pip, libtorch, source):
  • Build command you used (if compiling from source):
  • Are you using local sources or building from archives:
  • Python version:
  • CUDA version:
  • GPU models and configuration:
  • Any other relevant information:

Additional context

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.