failure of TensorRT 10.15 when running utils/process_engine.py on GPU A30
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 13.4k
- Forks
- 2.4k
- Avg merge
- 5d 3h
- Merged PRs (30d)
- 2
Description
Description
When I attempted to run this script tools/experimental/trt-engine-explorer/utils/process_engine.py for model performance analysis, an error occurred.
python utils/process_engine.py model.onnx ./output_dir fp16
error msg:
Profiling metadata: generated output file ./output_dir_opset13/model.onnx.engine.profile.metadata.json
Generating graph diagram: ./output_dir_opset13/model.onnx.engine.graph.json
Error: model.onnx.engine.graph.json.gv: syntax error in line 432 scanning a quoted string (missing endquote? longer than 16384?)
String starting:"LayerType:kgen\nMetadata:[ONNX Layer: ReduceSumSquare__2367]+[ONNX Layer: Div__2
Traceback (most recent call last):
File "/data/handle_models/TensorRT/tools/experimental/trt-engine-explorer/env_trex/lib/python3.12/site-packages/graphviz/backend/execute.py", line 88, in run_check
proc.check_returncode()
File "/usr/lib/python3.12/subprocess.py", line 502, in check_returncode
raise CalledProcessError(self.returncode, self.args, self.stdout,
subprocess.CalledProcessError: Command '[PosixPath('dot'), '-Kdot', '-Tsvg', '-o', 'model.onnx.engine.graph.json.svg', 'model.onnx.engine.graph.json.gv']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/data/handle_models/TensorRT/tools/experimental/trt-engine-explorer/utils/process_engine.py", line 351, in <module>
do_work(args)
File "/data/handle_models/TensorRT/tools/experimental/trt-engine-explorer/utils/process_engine.py", line 345, in do_work
process_engine(args, build, profile, draw)
File "/data/handle_models/TensorRT/tools/experimental/trt-engine-explorer/utils/process_engine.py", line 279, in process_engine
success = generate_engine_svg(args, add_suffix=build)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/handle_models/TensorRT/tools/experimental/trt-engine-explorer/utils/process_engine.py", line 250, in generate_engine_svg
draw_engine(graph_json_fname, profiling_json_fname)
File "/data/handle_models/TensorRT/tools/experimental/trt-engine-explorer/utils/draw_engine.py", line 179, in draw_engine
trex.graphing.render_dot(graph, engine_json_fname, output_format)
File "/data/handle_models/TensorRT/tools/experimental/trt-engine-explorer/trex/graphing.py", line 409, in render_dot
dot_graph.render(outfile=output_fname, view=False, overwrite_source=False)
File "/data/handle_models/TensorRT/tools/experimental/trt-engine-explorer/env_trex/lib/python3.12/site-packages/graphviz/_tools.py", line 185, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/data/handle_models/TensorRT/tools/experimental/trt-engine-explorer/env_trex/lib/python3.12/site-packages/graphviz/rendering.py", line 122, in render
rendered = self._render(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/handle_models/TensorRT/tools/experimental/trt-engine-explorer/env_trex/lib/python3.12/site-packages/graphviz/_tools.py", line 185, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/data/handle_models/TensorRT/tools/experimental/trt-engine-explorer/env_trex/lib/python3.12/site-packages/graphviz/backend/rendering.py", line 326, in render
execute.run_check(cmd,
File "/data/handle_models/TensorRT/tools/experimental/trt-engine-explorer/env_trex/lib/python3.12/site-packages/graphviz/backend/execute.py", line 90, in run_check
raise CalledProcessError(*e.args)
graphviz.backend.execute.CalledProcessError: Command '[PosixPath('dot'), '-Kdot', '-Tsvg', '-o', 'model.onnx.engine.graph.json.svg', 'model.onnx.engine.graph.json.gv']' returned non-zero exit status 1. [stderr: b'Error: model.onnx.engine.graph.json.gv: syntax error in line 432 scanning a quoted string (missing endquote? longer than 16384?)\nString starting:"LayerType:kgen\\nMetadata:[ONNX Layer: ReduceSumSquare__2367]+[ONNX Layer: Div__2\n']
Environment
TensorRT Version :10.15:
NVIDIA GPU : A30:
NVIDIA Driver Version : 550.144.03:
CUDA Version : 13.1:
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the failure with tools/experimental/trt-engine-explorer/utils/process_engine.py using the reported command and inspect the generated .gv file around line 432. Read utils/draw_engine.py and trex/graphing.py, especially render_dot, to trace how the graph label is emitted. Done means the reported model produces its SVG without a Graphviz quoted-string error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100