🐛 [Bug] Serialization of Graph Break Fails
Open
Nobody has claimed this yet.
bug
Story: Runtime & Memory & Serialization
- Dominant language
- Python
- Stars
- 3k
- Forks
- 410
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 78
Description
Bug Description
Serialization of Graph Break Fails
To Reproduce
Steps to reproduce the behavior:
import torch
import torch.nn as nn
import torch_tensorrt as torchtrt
import torchvision
torch.manual_seed(0)
torch.cuda.manual_seed_all(0)
model = torchvision.models.resnet50().eval().to("cuda")
input = [torch.randn((1, 3, 224, 224)).to("cuda")]
BATCH = torch.export.Dim("BATCH", min=1, max=16)
exp_program = torch.export.export(model, tuple(input))#, dynamic_shapes=({0: BATCH}, ))
trt_mod1 = trt_gm = torchtrt.dynamo.compile(
exp_program,
tuple(input),
# use_python_runtime=use_python_runtime,
enabled_precisions={torch.float},
min_block_size=1,
immutable_weights=False,
torch_executed_ops={torch.ops.aten.add.Tensor},
reuse_cached_engines=False,
)
serialized = torchtrt.save(trt_mod1, "/home/other/trt_mod1.pt2", inputs=input)
trt_mod_saved = torch.export.load("/home/other/trt_mod1.pt2").module()
Expected behavior
Environment
Build information about Torch-TensorRT can be found by turning on debug messages
- Torch-TensorRT Version (e.g. 1.0.0):
- PyTorch Version (e.g. 1.0):
- CPU Architecture:
- OS (e.g., Linux):
- How you installed PyTorch (
conda,pip,libtorch, source): - Build command you used (if compiling from source):
- Are you using local sources or building from archives:
- Python version:
- CUDA version:
- GPU models and configuration:
- Any other relevant information:
Additional context
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the issue with the provided ResNet50 example, focusing on torchtrt.dynamo.compile, torchtrt.save, and torch.export.load. Inspect the serialization path used by these entry points and document the failure with the relevant environment details. Done means the compiled module can be saved and loaded successfully for this reproduction.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- compilers, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100