pytorch / pytorch/TensorRT

🐛 [Bug] Encountered bug when using Torch-TensorRT

Open
#4,137 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug story: Dynamo Frontend & Partitioning
Dominant language
Python
Stars
3k
Forks
410
Avg merge
3d 18h
Merged PRs (30d)
78

Description

Bug Description

When running ZoomASR model with Torch-TensorRT, I encountered this issue:

Traceback (most recent call last):
  File "/home/shahe/.local/lib/python3.12/site-packages/sympy/core/expr.py", line 4035, in _mag
    mag_first_dig = int(ceil(log10(xpos)))
                        ^^^^^^^^^^^^^^^^^
OverflowError: cannot convert float infinity to integer

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/shahe/scratch/torch-trt-models/ZoomASR/single_decode.py", line 650, in <module>
    main()
  File "/home/shahe/scratch/torch-trt-models/ZoomASR/single_decode.py", line 566, in main
    compile_decoder_with_trt(model, decoder_ep, args.export_dir)
  File "/home/shahe/scratch/torch-trt-models/ZoomASR/single_decode.py", line 378, in compile_decoder_with_trt
    model.decoder = torch_tensorrt.dynamo.compile(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/shahe/.local/lib/python3.12/site-packages/torch_tensorrt/dynamo/_compiler.py", line 788, in compile
    trt_gm = compile_module(
             ^^^^^^^^^^^^^^^
  File "/home/shahe/.local/lib/python3.12/site-packages/torch_tensorrt/dynamo/_compiler.py", line 1006, in compile_module
    submodule_inputs = partitioning.construct_submodule_inputs(submodule)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/shahe/.local/lib/python3.12/site-packages/torch_tensorrt/dynamo/partitioning/common.py", line 93, in construct_submodule_inputs
    get_input(input_shape, input_meta.dtype, name=input.name)
  File "/home/shahe/.local/lib/python3.12/site-packages/torch_tensorrt/dynamo/partitioning/common.py", line 63, in get_input
    return construct_dynamic_input(
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/shahe/.local/lib/python3.12/site-packages/torch_tensorrt/dynamo/partitioning/common.py", line 31, in construct_dynamic_input
    min_max_opt = extract_var_range_info(dim)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/shahe/.local/lib/python3.12/site-packages/torch_tensorrt/dynamo/utils.py", line 427, in extract_var_range_info
    min_val, max_val = int(var_range.lower), int(var_range.upper)
                                             ^^^^^^^^^^^^^^^^^^^^
  File "/home/shahe/.local/lib/python3.12/site-packages/sympy/core/expr.py", line 308, in __int__
    r = self.round(2)
        ^^^^^^^^^^^^^
  File "/home/shahe/.local/lib/python3.12/site-packages/sympy/core/expr.py", line 3856, in round
    digits_to_decimal = _mag(x)  # _mag(12) = 2, _mag(.012) = -1
                        ^^^^^^^
  File "/home/shahe/.local/lib/python3.12/site-packages/sympy/core/expr.py", line 4037, in _mag
    mag_first_dig = int(ceil(Float(mpf_log(xpos._mpf_, 53))/log(10)))
                                           ^^^^^^^^^^
AttributeError: 'Infinity' object has no attribute '_mpf_'

To Reproduce

Steps to reproduce the behavior:

  1. git clone -b aot-decoder ssh://git@gitlab-master.nvidia.com:12051/wenbingl/ZoomASR.git
  2. cd into the cloned directory
  3. pip3 install -r requirements.txt
  4. python3 export.py --config pseudo_model/config.yaml --output_path pseudo_model
  5. CUDA_VISIBLE_DEVICES="0" python3 single_decode.py --config pseudo_model/export/config_aot.yaml --export_dir pseudo_model --output_file decode.hyp --data_stem data/yt-medical-2024 --batch_size 4 --device cuda --dtype fp16 --use_trt --max_segment_length 10

The last command will fail with the above error message.

Expected behavior

The command should run without error.

Note that there is no issue when running with this Torch-TensorRT code:
https://github.com/wenbingl/Torch-TRT/tree/dev

Environment

Build information about Torch-TensorRT can be found by turning on debug messages

  • Torch-TensorRT Version (e.g. 1.0.0): torch-tensorrt-2.11.0.dev20260212+cu130
  • PyTorch Version (e.g. 1.0): torch-2.11.0.dev20260216+cu130
  • CPU Architecture: x86_64
  • OS (e.g., Linux): ubuntu24.04
  • How you installed PyTorch (conda, pip, libtorch, source): pip
  • Build command you used (if compiling from source):
  • Are you using local sources or building from archives: No
  • Python version: python3.12
  • CUDA version: cuda13.0
  • GPU models and configuration: rtx-pro-6000-blackwell-server-edition
  • Any other relevant information:

Additional context

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the final command from the issue, then inspect torch_tensorrt/dynamo/partitioning/common.py and torch_tensorrt/dynamo/utils.py around construct_dynamic_input and extract_var_range_info. Trace how the dynamic dimension range reaches SymPy and verify that the ZoomASR Torch-TensorRT command completes without the Infinity conversion error.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
compilers, devtools, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.