🐛 [Bug] Draw svg for flux model hangs
Open
Nobody has claimed this yet.
bug
story: LLM & Generative AI
- Dominant language
- Python
- Stars
- 3k
- Forks
- 410
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 78
Description
Bug Description
Whenever I try to draw the flux model, it hangs, I have to kill it.
^^^^^^^^^^^^^^^^^^^
File "/workspace/TensorRT/tools/perf/Flux/../../../examples/apps/flux_demo.py", line 179, in compile_model
trt_gm = torch_tensorrt.dynamo.compile(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/TensorRT/py/torch_tensorrt/dynamo/_compiler.py", line 707, in compile
gm = post_lowering(gm, settings)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/TensorRT/py/torch_tensorrt/dynamo/lowering/passes/_aten_lowering_pass.py", line 104, in post_lowering
gm = ATEN_POST_LOWERING_PASSES(gm, settings)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/TensorRT/py/torch_tensorrt/dynamo/lowering/passes/pass_manager.py", line 135, in __call__
out = _pass(out, settings)
^^^^^^^^^^^^^^^^^^^^
File "/workspace/TensorRT/py/torch_tensorrt/dynamo/lowering/passes/pass_manager.py", line 22, in draw_fx_graph_pass
f.write(g.get_dot_graph().create_svg())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/envs/py311/lib/python3.11/site-packages/pydot/core.py", line 181, in __create_method
return self.create(format=f, prog=prog, encoding=encoding)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/envs/py311/lib/python3.11/site-packages/pydot/core.py", line 1824, in create
stdout_data, stderr_data, process = call_graphviz(
^^^^^^^^^^^^^^
File "/opt/conda/envs/py311/lib/python3.11/site-packages/pydot/core.py", line 258, in call_graphviz
stdout_data, stderr_data = process.communicate()
^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/envs/py311/lib/python3.11/subprocess.py", line 1209, in communicate
stdout, stderr = self._communicate(input, endtime, timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/envs/py311/lib/python3.11/subprocess.py", line 2115, in _communicate
ready = selector.select(timeout)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/envs/py311/lib/python3.11/selectors.py", line 415, in select
fd_event_list = self._selector.poll(timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
KeyboardInterrupt
To Reproduce
Steps to reproduce the behavior:
with torch_tensorrt.dynamo.Debugger(
"graphs",
logging_dir=DEBUG_LOGGING_DIR,
capture_fx_graph_after=["remove_num_users_is_0_nodes"],
save_engine_profile=True,
profile_format="trex",
engine_builder_monitor=True,
):
trt_gm = torch_tensorrt.dynamo.compile(
ep, inputs=dummy_inputs, **settings
)
Expected behavior
Environment
Build information about Torch-TensorRT can be found by turning on debug messages
- Torch-TensorRT Version (e.g. 1.0.0):
- PyTorch Version (e.g. 1.0):
- CPU Architecture:
- OS (e.g., Linux):
- How you installed PyTorch (
conda,pip,libtorch, source): - Build command you used (if compiling from source):
- Are you using local sources or building from archives:
- Python version:
- CUDA version:
- GPU models and configuration:
- Any other relevant information:
Additional context
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with examples/apps/flux_demo.py:179 and py/torch_tensorrt/dynamo/lowering/passes/pass_manager.py:22, then trace the draw_fx_graph_pass call into pydot's create_svg shown in the traceback. Reproduce the Flux model compilation with the provided Debugger settings; done means SVG graph generation no longer hangs during compilation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- compilers, machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100