lllyasviel / lllyasviel/stable-diffusion-webui-forge

InstantX ControlNet Canny built for Flux not working.

Open
#1,067 4 comments 4 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
13k
Forks
1.7k
PR merge metrics
No merged PRs in 30d

Description

A ControlNet Model that is currently not working with Flux in Forge:

https://huggingface.co/InstantX/FLUX.1-dev-Controlnet-Canny

An error message shows:

`Recognizing Control Model failed: C:\Forge\stable-diffusion-webui-forge\models\ControlNet\InstantXFLUX.1-dev-Controlnet-Canny.safetensors
*** Error running process: C:\Forge\stable-diffusion-webui-forge\extensions-builtin\sd_forge_controlnet\scripts\controlnet.py
Traceback (most recent call last):
File "C:\Forge\stable-diffusion-webui-forge\modules\scripts.py", line 844, in process
script.process(p, *script_args)
File "C:\Forge\stable-diffusion-webui-forge\venv\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "C:\Forge\stable-diffusion-webui-forge\extensions-builtin\sd_forge_controlnet\scripts\controlnet.py", line 554, in process
self.process_unit_after_click_generate(p, unit, params, *args, **kwargs)
File "C:\Forge\stable-diffusion-webui-forge\venv\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "C:\Forge\stable-diffusion-webui-forge\extensions-builtin\sd_forge_controlnet\scripts\controlnet.py", line 414, in process_unit_after_click_generate
assert params.model is not None, logger.error(f"Recognizing Control Model failed: {model_filename}")
AssertionError: None`

Thank you for all your work on Forge, It seems like a huge undertaking but with the steps you've taken, I'm super excited for it's future :)

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with extensions-builtin/sd_forge_controlnet/scripts/controlnet.py, especially the model-processing paths at lines 414 and 554, and reproduce the reported failure with the InstantX FLUX.1-dev-Controlnet-Canny.safetensors model. Done means the ControlNet model is recognized without the shown AssertionError and can be used with Flux.

Written by the indexing model from the issue text.

Assessment

Tech stack
huggingface, python, pytorch
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.