lllyasviel / lllyasviel/stable-diffusion-webui-forge

[BUG] TensorRT dont work anymore

Open
#879 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
13k
Forks
1.7k
PR merge metrics
No merged PRs in 30d

Description

i dont know 2 month ago its was still create a working ONNX model, new version
I have 16GBVram and 14GB free if i push the button "Export Engine"
the directory Unet-onnx and Unet-trt are empty

Error:

Exporting Smart_Evolution_Sevenof9 to TensorRT using - Batch Size: 1-1-1
Height: 576-1024-1408
Width: 576-1024-1408
Token Count: 75-75-150
ERROR:root:Exporting to ONNX failed. Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument mat1 in method wrapper_CUDA_addmm)
Building TensorRT engine... This can take a while, please check the progress in the terminal.
Building TensorRT engine for E:\WebUI_Forge\webui\models\Unet-onnx\Smart_Evolution_Sevenof9.onnx: E:\WebUI_Forge\webui\models\Unet-trt\Smart_Evolution_Sevenof9_0e3a40ad_cc89_sample=1x4x72x72+1x4x128x128+1x4x176x176-timesteps=1+1+1-encoder_hidden_states=1x77x2048+1x77x2048+1x154x2048-y=1x2816+1x2816+1x2816.trt
Could not open file E:\WebUI_Forge\webui\models\Unet-onnx\Smart_Evolution_Sevenof9.onnx
Could not open file E:\WebUI_Forge\webui\models\Unet-onnx\Smart_Evolution_Sevenof9.onnx
[E] ModelImporter.cpp:773: Failed to parse ONNX model from file: E:\WebUI_Forge\webui\models\Unet-onnx\Smart_Evolution_Sevenof9.onnx
[!] Failed to parse ONNX model. Does the model file exist and contain a valid ONNX model?
Traceback (most recent call last):
File "e:\WebUI_Forge\system\python\lib\site-packages\gradio\queueing.py", line 536, in process_events
response = await route_utils.call_process_api(
File "e:\WebUI_Forge\system\python\lib\site-packages\gradio\route_utils.py", line 276, in call_process_api
output = await app.get_blocks().process_api(
File "e:\WebUI_Forge\system\python\lib\site-packages\gradio\blocks.py", line 1923, in process_api
result = await self.call_function(
File "e:\WebUI_Forge\system\python\lib\site-packages\gradio\blocks.py", line 1508, in call_function
prediction = await anyio.to_thread.run_sync( # type: ignore
File "e:\WebUI_Forge\system\python\lib\site-packages\anyio\to_thread.py", line 33, in run_sync
return await get_asynclib().run_sync_in_worker_thread(
File "e:\WebUI_Forge\system\python\lib\site-packages\anyio\_backends\_asyncio.py", line 877, in run_sync_in_worker_thread
return await future
File "e:\WebUI_Forge\system\python\lib\site-packages\anyio\_backends\_asyncio.py", line 807, in run
result = context.run(func, *args)
File "e:\WebUI_Forge\system\python\lib\site-packages\gradio\utils.py", line 818, in wrapper
response = f(*args, **kwargs)
File "E:\WebUI_Forge\webui\extensions\Stable-Diffusion-WebUI-TensorRT\ui_trt.py", line 126, in export_unet_to_trt
ret = export_trt(
File "E:\WebUI_Forge\webui\extensions\Stable-Diffusion-WebUI-TensorRT\exporter.py", line 231, in export_trt
ret = engine.build(
File "E:\WebUI_Forge\webui\extensions\Stable-Diffusion-WebUI-TensorRT\utilities.py", line 227, in build
network = network_from_onnx_path(
File "", line 3, in network_from_onnx_path
File "e:\WebUI_Forge\system\python\lib\site-packages\polygraphy\backend\base\loader.py", line 40, in __call__
return self.call_impl(*args, **kwargs)
File "e:\WebUI_Forge\system\python\lib\site-packages\polygraphy\util\util.py", line 710, in wrapped
return func(*args, **kwargs)
File "e:\WebUI_Forge\system\python\lib\site-packages\polygraphy\backend\trt\loader.py", line 247, in call_impl
trt_util.check_onnx_parser_errors(parser, success)
File "e:\WebUI_Forge\system\python\lib\site-packages\polygraphy\backend\trt\util.py", line 88, in check_onnx_parser_errors
G_LOGGER.critical(
File "e:\WebUI_Forge\system\python\lib\site-packages\polygraphy\logger\logger.py", line 605, in critical
raise ExceptionType(message) from None
polygraphy.exception.exception.PolygraphyException: Failed to parse ONNX model. Does the model file exist and contain a valid ONNX model?

Contributor guide

No contributing guide indexed for this repository

Research direction

Start at export_unet_to_trt in extensions/Stable-Diffusion-WebUI-TensorRT/ui_trt.py, then trace export_trt in exporter.py and build in utilities.py. Reproduce the Export Engine failure and inspect the ONNX export step before the TensorRT parser is invoked. Done means the export creates a valid ONNX file and TensorRT engine without the reported mixed-device error.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.