lllyasviel / lllyasviel/stable-diffusion-webui-forge

Integrated ControlNet Missing SDXL FaceID+ Compatible Preprocessor

Open
#1,854 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
13k
Forks
1.7k
PR merge metrics
No merged PRs in 30d

Description

There is currently no ControlNet preprocessor that is compatible with the IP-Adapter model ip-adapter-faceid-plusv2, Automatic1111 has compatible preprocessors that are missing in Forge. All 3 IP-Adapater preprocessors error out, the one preprocessor labeled InsightFace+CLIP-H gave this error:

Traceback (most recent call last):
  File "C:\webui_forge_cu121_torch231\webui\extensions-builtin\sd_forge_ipadapter\lib_ipadapter\IPAdapterPlus.py", line 566, in load_insight_face
    from insightface.app import FaceAnalysis
  File "C:\webui_forge_cu121_torch231\webui\venv\lib\site-packages\insightface\__init__.py", line 16, in <module>
    from . import model_zoo
  File "C:\webui_forge_cu121_torch231\webui\venv\lib\site-packages\insightface\model_zoo\__init__.py", line 1, in <module>
    from .model_zoo import get_model
  File "C:\webui_forge_cu121_torch231\webui\venv\lib\site-packages\insightface\model_zoo\model_zoo.py", line 11, in <module>
    from .arcface_onnx import *
  File "C:\webui_forge_cu121_torch231\webui\venv\lib\site-packages\insightface\model_zoo\arcface_onnx.py", line 10, in <module>
    import onnx
  File "C:\webui_forge_cu121_torch231\webui\venv\lib\site-packages\onnx\__init__.py", line 77, in <module>
    from onnx.onnx_cpp2py_export import ONNX_ML
ImportError: DLL load failed while importing onnx_cpp2py_export: A dynamic link library (DLL) initialization routine failed.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\webui_forge_cu121_torch231\webui\modules\scripts.py", line 844, in process
    script.process(p, *script_args)
  File "C:\webui_forge_cu121_torch231\webui\venv\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "C:\webui_forge_cu121_torch231\webui\extensions-builtin\sd_forge_controlnet\scripts\controlnet.py", line 554, in process
    self.process_unit_after_click_generate(p, unit, params, *args, **kwargs)
  File "C:\webui_forge_cu121_torch231\webui\venv\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "C:\webui_forge_cu121_torch231\webui\extensions-builtin\sd_forge_controlnet\scripts\controlnet.py", line 328, in process_unit_after_click_generate
    preprocessor_output = preprocessor(
  File "C:\webui_forge_cu121_torch231\webui\extensions-builtin\sd_forge_ipadapter\scripts\forge_ipadapter.py", line 46, in __call__
    insightface=self.load_insightface(),
  File "C:\webui_forge_cu121_torch231\webui\extensions-builtin\sd_forge_ipadapter\scripts\forge_ipadapter.py", line 40, in load_insightface
    self.cached_insightface = opInsightFaceLoader()[0]
  File "C:\webui_forge_cu121_torch231\webui\extensions-builtin\sd_forge_ipadapter\lib_ipadapter\IPAdapterPlus.py", line 568, in load_insight_face
    raise Exception(e)
Exception: DLL load failed while importing onnx_cpp2py_export: A dynamic link library (DLL) initialization routine failed.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the failure using the IP-Adapter preprocessors, then read extensions-builtin/sd_forge_ipadapter/scripts/forge_ipadapter.py and lib_ipadapter/IPAdapterPlus.py around load_insightface, along with the ControlNet call in scripts/controlnet.py. Compare the available preprocessors with the ip-adapter-faceid-plusv2 requirements and verify that the compatible preprocessor runs without the reported ONNX or DLL error.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
ai, computer-vision
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.