flagos-ai / flagos-ai/FlagPerf
stable_diffusion_v1_4使用torchtrt推理时报错
- Dominant language
- Python
- Stars
- 379
- Forks
- 124
- PR merge metrics
- No merged PRs in 30d
Description
ERROR: [Torch-TensorRT] - Unsupported operator: aten::to.dtype_layout(Tensor(a) self, *, ScalarType? dtype=None, Layout? layout=None, Device? device=None, bool? pin_memory=None, bool non_blocking=False, bool copy=False, MemoryFormat? memory_format=None) -> Tensor(a)
/workspace/luor/FlagPerf/inference/benchmarks/stable_diffusion_v1_4/pytorch/model_utils/unet2d.py(854): forward
/root/miniconda3/envs/fperf/lib/python3.10/site-packages/torch/nn/modules/module.py(1501): _slow_forward
/root/miniconda3/envs/fperf/lib/python3.10/site-packages/torch/nn/modules/module.py(1520): _call_impl
/root/miniconda3/envs/fperf/lib/python3.10/site-packages/torch/nn/modules/module.py(1511): _wrapped_call_impl
/root/miniconda3/envs/fperf/lib/python3.10/site-packages/torch/jit/_trace.py(1074): trace_module
/root/miniconda3/envs/fperf/lib/python3.10/site-packages/torch/jit/_trace.py(806): trace
/workspace/luor/FlagPerf/inference/inference_engine/nvidia/torchtrt.py(23): __call__
/workspace/luor/FlagPerf/inference/benchmarks/stable_diffusion_v1_4/pytorch/forward.py(209): engine_forward
/workspace/luor/FlagPerf/inference/run_inference.py(80): main
/workspace/luor/FlagPerf/inference/run_inference.py(137):
Traceback (most recent call last):
File "/workspace/luor/FlagPerf/inference/run_inference.py", line 137, in
config, p_forward, p_infer, p_forward_core, p_infer_core, val_acc, infer_acc = main(
File "/workspace/luor/FlagPerf/inference/run_inference.py", line 80, in main
p_infer, p_infer_core, infer_acc = benchmark_module.engine_forward(
File "/workspace/luor/FlagPerf/inference/benchmarks/stable_diffusion_v1_4/pytorch/forward.py", line 209, in engine_forward
outputs = model(inputs)
File "/workspace/luor/FlagPerf/inference/inference_engine/nvidia/torchtrt.py", line 25, in __call__
self.trt_model = torchtrt.compile(
File "/root/miniconda3/envs/fperf/lib/python3.10/site-packages/torch_tensorrt/_compile.py", line 185, in compile
compiled_ts_module: torch.jit.ScriptModule = torchscript_compile(
File "/root/miniconda3/envs/fperf/lib/python3.10/site-packages/torch_tensorrt/ts/_compiler.py", line 151, in compile
compiled_cpp_mod = _C.compile_graph(module._c, _parse_compile_spec(spec))
RuntimeError: [Error thrown at core/compiler.cpp:413] Expected conversion::VerifyConverterSupportForBlock(g->block()) to be true but got false
Not all operations in graph are supported by the compiler
请问测试时,官方使用的软件版本是多少?
以下是我测试时的版本:
libtorch 2.4.0
torch 2.2.2
torch-tensorrt 2.2.0
torchmetrics 1.4.2
tensorrt 8.6.1.post1
tensorrt-bindings 8.6.1
tensorrt-libs 8.6.1
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with inference/benchmarks/stable_diffusion_v1_4/pytorch/model_utils/unet2d.py at line 854 and trace the call through inference/inference_engine/nvidia/torchtrt.py line 23. Reproduce the failure with the listed PyTorch, Torch-TensorRT, and TensorRT versions, then compare the supported software versions and verify that stable-diffusion inference compiles without the unsupported aten::to.dtype_layout operator.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100