lllyasviel / lllyasviel/stable-diffusion-webui-forge
[BUG] TensorRT dont work anymore
- Dominant language
- Python
- Stars
- 13k
- Forks
- 1.7k
- PR merge metrics
- No merged PRs in 30d
Description
i dont know 2 month ago its was still create a working ONNX model, new version
I have 16GBVram and 14GB free if i push the button "Export Engine"
the directory Unet-onnx and Unet-trt are empty
Error:
Exporting Smart_Evolution_Sevenof9 to TensorRT using - Batch Size: 1-1-1
Height: 576-1024-1408
Width: 576-1024-1408
Token Count: 75-75-150
ERROR:root:Exporting to ONNX failed. Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument mat1 in method wrapper_CUDA_addmm)
Building TensorRT engine... This can take a while, please check the progress in the terminal.
Building TensorRT engine for E:\WebUI_Forge\webui\models\Unet-onnx\Smart_Evolution_Sevenof9.onnx: E:\WebUI_Forge\webui\models\Unet-trt\Smart_Evolution_Sevenof9_0e3a40ad_cc89_sample=1x4x72x72+1x4x128x128+1x4x176x176-timesteps=1+1+1-encoder_hidden_states=1x77x2048+1x77x2048+1x154x2048-y=1x2816+1x2816+1x2816.trt
Could not open file E:\WebUI_Forge\webui\models\Unet-onnx\Smart_Evolution_Sevenof9.onnx
Could not open file E:\WebUI_Forge\webui\models\Unet-onnx\Smart_Evolution_Sevenof9.onnx
[E] ModelImporter.cpp:773: Failed to parse ONNX model from file: E:\WebUI_Forge\webui\models\Unet-onnx\Smart_Evolution_Sevenof9.onnx
[!] Failed to parse ONNX model. Does the model file exist and contain a valid ONNX model?
Traceback (most recent call last):
File "e:\WebUI_Forge\system\python\lib\site-packages\gradio\queueing.py", line 536, in process_events
response = await route_utils.call_process_api(
File "e:\WebUI_Forge\system\python\lib\site-packages\gradio\route_utils.py", line 276, in call_process_api
output = await app.get_blocks().process_api(
File "e:\WebUI_Forge\system\python\lib\site-packages\gradio\blocks.py", line 1923, in process_api
result = await self.call_function(
File "e:\WebUI_Forge\system\python\lib\site-packages\gradio\blocks.py", line 1508, in call_function
prediction = await anyio.to_thread.run_sync( # type: ignore
File "e:\WebUI_Forge\system\python\lib\site-packages\anyio\to_thread.py", line 33, in run_sync
return await get_asynclib().run_sync_in_worker_thread(
File "e:\WebUI_Forge\system\python\lib\site-packages\anyio\_backends\_asyncio.py", line 877, in run_sync_in_worker_thread
return await future
File "e:\WebUI_Forge\system\python\lib\site-packages\anyio\_backends\_asyncio.py", line 807, in run
result = context.run(func, *args)
File "e:\WebUI_Forge\system\python\lib\site-packages\gradio\utils.py", line 818, in wrapper
response = f(*args, **kwargs)
File "E:\WebUI_Forge\webui\extensions\Stable-Diffusion-WebUI-TensorRT\ui_trt.py", line 126, in export_unet_to_trt
ret = export_trt(
File "E:\WebUI_Forge\webui\extensions\Stable-Diffusion-WebUI-TensorRT\exporter.py", line 231, in export_trt
ret = engine.build(
File "E:\WebUI_Forge\webui\extensions\Stable-Diffusion-WebUI-TensorRT\utilities.py", line 227, in build
network = network_from_onnx_path(
File "", line 3, in network_from_onnx_path
File "e:\WebUI_Forge\system\python\lib\site-packages\polygraphy\backend\base\loader.py", line 40, in __call__
return self.call_impl(*args, **kwargs)
File "e:\WebUI_Forge\system\python\lib\site-packages\polygraphy\util\util.py", line 710, in wrapped
return func(*args, **kwargs)
File "e:\WebUI_Forge\system\python\lib\site-packages\polygraphy\backend\trt\loader.py", line 247, in call_impl
trt_util.check_onnx_parser_errors(parser, success)
File "e:\WebUI_Forge\system\python\lib\site-packages\polygraphy\backend\trt\util.py", line 88, in check_onnx_parser_errors
G_LOGGER.critical(
File "e:\WebUI_Forge\system\python\lib\site-packages\polygraphy\logger\logger.py", line 605, in critical
raise ExceptionType(message) from None
polygraphy.exception.exception.PolygraphyException: Failed to parse ONNX model. Does the model file exist and contain a valid ONNX model?
Contributor guide
No contributing guide indexed for this repository
Research direction
Start at export_unet_to_trt in extensions/Stable-Diffusion-WebUI-TensorRT/ui_trt.py, then trace export_trt in exporter.py and build in utilities.py. Reproduce the Export Engine failure and inspect the ONNX export step before the TensorRT parser is invoked. Done means the export creates a valid ONNX file and TensorRT engine without the reported mixed-device error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100