🐛 [Bug] Encountered bug when using Torch-TensorRT
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 3k
- Forks
- 410
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 78
Description
Bug Description
When running ZoomASR model with Torch-TensorRT, I encountered this issue:
Traceback (most recent call last):
File "/home/shahe/.local/lib/python3.12/site-packages/sympy/core/expr.py", line 4035, in _mag
mag_first_dig = int(ceil(log10(xpos)))
^^^^^^^^^^^^^^^^^
OverflowError: cannot convert float infinity to integer
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/shahe/scratch/torch-trt-models/ZoomASR/single_decode.py", line 650, in <module>
main()
File "/home/shahe/scratch/torch-trt-models/ZoomASR/single_decode.py", line 566, in main
compile_decoder_with_trt(model, decoder_ep, args.export_dir)
File "/home/shahe/scratch/torch-trt-models/ZoomASR/single_decode.py", line 378, in compile_decoder_with_trt
model.decoder = torch_tensorrt.dynamo.compile(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/shahe/.local/lib/python3.12/site-packages/torch_tensorrt/dynamo/_compiler.py", line 788, in compile
trt_gm = compile_module(
^^^^^^^^^^^^^^^
File "/home/shahe/.local/lib/python3.12/site-packages/torch_tensorrt/dynamo/_compiler.py", line 1006, in compile_module
submodule_inputs = partitioning.construct_submodule_inputs(submodule)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/shahe/.local/lib/python3.12/site-packages/torch_tensorrt/dynamo/partitioning/common.py", line 93, in construct_submodule_inputs
get_input(input_shape, input_meta.dtype, name=input.name)
File "/home/shahe/.local/lib/python3.12/site-packages/torch_tensorrt/dynamo/partitioning/common.py", line 63, in get_input
return construct_dynamic_input(
^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/shahe/.local/lib/python3.12/site-packages/torch_tensorrt/dynamo/partitioning/common.py", line 31, in construct_dynamic_input
min_max_opt = extract_var_range_info(dim)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/shahe/.local/lib/python3.12/site-packages/torch_tensorrt/dynamo/utils.py", line 427, in extract_var_range_info
min_val, max_val = int(var_range.lower), int(var_range.upper)
^^^^^^^^^^^^^^^^^^^^
File "/home/shahe/.local/lib/python3.12/site-packages/sympy/core/expr.py", line 308, in __int__
r = self.round(2)
^^^^^^^^^^^^^
File "/home/shahe/.local/lib/python3.12/site-packages/sympy/core/expr.py", line 3856, in round
digits_to_decimal = _mag(x) # _mag(12) = 2, _mag(.012) = -1
^^^^^^^
File "/home/shahe/.local/lib/python3.12/site-packages/sympy/core/expr.py", line 4037, in _mag
mag_first_dig = int(ceil(Float(mpf_log(xpos._mpf_, 53))/log(10)))
^^^^^^^^^^
AttributeError: 'Infinity' object has no attribute '_mpf_'
To Reproduce
Steps to reproduce the behavior:
- git clone -b aot-decoder ssh://git@gitlab-master.nvidia.com:12051/wenbingl/ZoomASR.git
- cd into the cloned directory
- pip3 install -r requirements.txt
- python3 export.py --config pseudo_model/config.yaml --output_path pseudo_model
- CUDA_VISIBLE_DEVICES="0" python3 single_decode.py --config pseudo_model/export/config_aot.yaml --export_dir pseudo_model --output_file decode.hyp --data_stem data/yt-medical-2024 --batch_size 4 --device cuda --dtype fp16 --use_trt --max_segment_length 10
The last command will fail with the above error message.
Expected behavior
The command should run without error.
Note that there is no issue when running with this Torch-TensorRT code:
https://github.com/wenbingl/Torch-TRT/tree/dev
Environment
Build information about Torch-TensorRT can be found by turning on debug messages
- Torch-TensorRT Version (e.g. 1.0.0): torch-tensorrt-2.11.0.dev20260212+cu130
- PyTorch Version (e.g. 1.0): torch-2.11.0.dev20260216+cu130
- CPU Architecture: x86_64
- OS (e.g., Linux): ubuntu24.04
- How you installed PyTorch (
conda,pip,libtorch, source): pip - Build command you used (if compiling from source):
- Are you using local sources or building from archives: No
- Python version: python3.12
- CUDA version: cuda13.0
- GPU models and configuration: rtx-pro-6000-blackwell-server-edition
- Any other relevant information:
Additional context
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the final command from the issue, then inspect torch_tensorrt/dynamo/partitioning/common.py and torch_tensorrt/dynamo/utils.py around construct_dynamic_input and extract_var_range_info. Trace how the dynamic dimension range reaches SymPy and verify that the ZoomASR Torch-TensorRT command completes without the Infinity conversion error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- compilers, devtools, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100