deforum / deforum/sd-webui-deforum

[Bug]: Error: local variable 'h' referenced before assignment.

Open
#913 3 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
2.9k
Forks
407
PR merge metrics
No merged PRs in 30d

Description

### Have you read the latest version of the FAQ?

- [X] I have visited the FAQ page right now and my issue is not present there

### Is there an existing issue for this?

- [X] I have searched the existing issues and checked the recent builds/commits of both this extension and the webui

### Are you using the latest version of the Deforum extension?

- [X] I have Deforum updated to the lastest version and I still have the issue.

### What happened?

I am using deforum with controlnet.
I have 5 controlnets all set to reference only.

It seems to generate the first frame but then after that it throw an error in k-diffusion?

### Steps to reproduce the problem

1. Goto deforum tab,
2. enable 5 controlnets with different images, reference only
3. have cadence set to 4 and coherance set to dis fine for both flow cadence and flow generation
4. set 500 frames and hit generate

### What should have happened/how would you fix it?

not to have an error thrown

### Torch version

2.0.1+cu118

### On which platform are you launching the webui with the extension?

Local PC setup (Windows)

### Deforum settings

https://gist.github.com/nicholasVANDAL/62ae5bab1f79eab258c7bb9d1a2e395b

### Webui core settings

https://gist.github.com/nicholasVANDAL/1cf02d15a8e4a8932d71b0c9d2ec80b2

### Console logs

```Shell
venv "C:\Users\va-ai.render\stable-diffusion-webui-sdxl\venv\Scripts\Python.exe"
Python 3.10.9 (tags/v3.10.9:1dd9be6, Dec 6 2022, 20:01:21) [MSC v.1934 64 bit (AMD64)]
Version: v1.6.0
Commit hash: 5ef669de080814067961f28357256e8fe27544f4
Launching Web UI with arguments: --xformers --no-half --no-half-vae
2023-10-19 10:47:26,739 - ControlNet - INFO - ControlNet v1.1.411
ControlNet preprocessor location: C:\Users\va-ai.render\stable-diffusion-webui-sdxl\extensions\sd-webui-controlnet\annotator\downloads
2023-10-19 10:47:26,833 - ControlNet - INFO - ControlNet v1.1.411
Loading weights [31e35c80fc] from C:\Users\va-ai.render\stable-diffusion-webui-sdxl\models\Stable-diffusion\sd_xl_base_1.0.safetensors
*Deforum ControlNet support: enabled*
Running on local URL: http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
Creating model from config: C:\Users\va-ai.render\stable-diffusion-webui-sdxl\repositories\generative-models\configs\inference\sd_xl_base.yaml
Startup time: 12.9s (prepare environment: 4.8s, import torch: 2.8s, import gradio: 0.8s, setup paths: 0.7s, initialize shared: 0.2s, other imports: 0.5s, setup codeformer: 0.2s, load scripts: 1.4s, create ui: 0.7s, gradio launch: 0.7s).
Applying attention optimization: xformers... done.
Model loaded in 7.4s (load weights from disk: 1.4s, create model: 1.1s, apply weights to model: 2.9s, apply float(): 0.9s, move model to device: 0.4s, calculate empty prompt: 0.7s).
reading custom settings from D:\_STABLE_DIFFUSION_LOCAL\Deforum_20231019102806_NYE_GLOW-HOOP\20231019102806_settings.txt
started run_deforum
times_to_run: 1
Deforum extension for auto1111 webui
Git commit: 43158f9c
Starting job task(6emk69e46gqk4uz)-0...
Copied CN Model 1's single input image to inputframes folder!
Copied CN Model 2's single input image to inputframes folder!
Copied CN Model 3's single input image to inputframes folder!
Copied CN Model 4's single input image to inputframes folder!
Copied CN Model 5's single input image to inputframes folder!
Saving animation frames to:
D:\_STABLE_DIFFUSION_LOCAL\Deforum_20231019104802_NYE_GLOW-HOOP
Animation frame: 0/500
Seed: 3050216332
Prompt: neon lines, neon shapes, glow, neon circles, circular, circle, vibes, vibrant, stunningly beautiful, crisp, sleek, ultramodern, high contrast, cinematic, professional, shpere, circles a picture of a colorful flower on a black background, infinity glyph waves, redshift, neon ligh, avatar image, spinning, centered and symmetrical, the energy is released, stylised design, phasmophobia, girih, depth blur, interconnected, joytoy, looping
Neg Prompt: painting, drawing, illustration, glitch, deformed, mutated, cross-eyed, ugly, disfigured, nsfw, nude, ugly, words, text, asymmetric text, logo, watermark, words, branding, word, dark, person, people, human, face nsfw, nude
Not using an init image (doing pure txt2img)
╭─────┬────┬───────┬────┬────┬────┬────┬────┬────╮
│Steps│CFG │Denoise│Tr X│Tr Y│Tr Z│Ro X│Ro Y│Ro Z│
├─────┼────┼───────┼────┼────┼────┼────┼────┼────┤
│ 50 │12.0│ 0 │ 0 │ 0 │ 1 │ 0 │ 0 │0.3 │
╰─────┴────┴───────┴────┴────┴────┴────┴────┴────╯
Reading ControlNet *static* base frame at D:\_STABLE_DIFFUSION_LOCAL\Deforum_20231019104802_NYE_GLOW-HOOP\controlnet_1_inputframes\000000000.jpg
Reading ControlNet *static* base frame at D:\_STABLE_DIFFUSION_LOCAL\Deforum_20231019104802_NYE_GLOW-HOOP\controlnet_2_inputframes\000000000.jpg
Reading ControlNet *static* base frame at D:\_STABLE_DIFFUSION_LOCAL\Deforum_20231019104802_NYE_GLOW-HOOP\controlnet_3_inputframes\000000000.jpg
Reading ControlNet *static* base frame at D:\_STABLE_DIFFUSION_LOCAL\Deforum_20231019104802_NYE_GLOW-HOOP\controlnet_4_inputframes\000000000.jpg
Reading ControlNet *static* base frame at D:\_STABLE_DIFFUSION_LOCAL\Deforum_20231019104802_NYE_GLOW-HOOP\controlnet_5_inputframes\000000000.jpg
2023-10-19 10:48:03,992 - ControlNet - INFO - Loading preprocessor: reference_only
2023-10-19 10:48:03,992 - ControlNet - INFO - preprocessor resolution = 64
2023-10-19 10:48:04,059 - ControlNet - INFO - Loading preprocessor: reference_only
2023-10-19 10:48:04,059 - ControlNet - INFO - preprocessor resolution = 64
2023-10-19 10:48:04,109 - ControlNet - INFO - Loading preprocessor: reference_only
2023-10-19 10:48:04,109 - ControlNet - INFO - preprocessor resolution = 64
2023-10-19 10:48:04,175 - ControlNet - INFO - Loading preprocessor: reference_only
2023-10-19 10:48:04,175 - ControlNet - INFO - preprocessor resolution = 64
2023-10-19 10:48:04,259 - ControlNet - INFO - Loading preprocessor: reference_only
2023-10-19 10:48:04,259 - ControlNet - INFO - preprocessor resolution = 64
2023-10-19 10:48:04,342 - ControlNet - INFO - ControlNet Hooked - Time = 0.5237596035003662
0%| | 0/50 [00:00 - local variable 'h' referenced before assignment

*START OF TRACEBACK*
Traceback (most recent call last):
File "C:\Users\va-ai.render\stable-diffusion-webui-sdxl\extensions\deforum-for-automatic1111-webui\scripts\deforum_helpers\run_deforum.py", line 116, in run_deforum
render_animation(args, anim_args, video_args, parseq_args, loop_args, controlnet_args, root)
File "C:\Users\va-ai.render\stable-diffusion-webui-sdxl\extensions\deforum-for-automatic1111-webui\scripts\deforum_helpers\render.py", line 550, in render_animation
disposable_image = generate(args, keys, anim_args, loop_args, controlnet_args, root, parseq_adapter, frame_idx, sampler_name=scheduled_sampler_name)
File "C:\Users\va-ai.render\stable-diffusion-webui-sdxl\extensions\deforum-for-automatic1111-webui\scripts\deforum_helpers\generate.py", line 80, in generate
image, caught_vae_exception = generate_with_nans_check(args, keys, anim_args, loop_args, controlnet_args, root, parseq_adapter, frame, sampler_name)
File "C:\Users\va-ai.render\stable-diffusion-webui-sdxl\extensions\deforum-for-automatic1111-webui\scripts\deforum_helpers\generate.py", line 114, in generate_with_nans_check
raise e
File "C:\Users\va-ai.render\stable-diffusion-webui-sdxl\extensions\deforum-for-automatic1111-webui\scripts\deforum_helpers\generate.py", line 108, in generate_with_nans_check
image = generate_inner(args, keys, anim_args, loop_args, controlnet_args, root, parseq_adapter, frame, sampler_name)
File "C:\Users\va-ai.render\stable-diffusion-webui-sdxl\extensions\deforum-for-automatic1111-webui\scripts\deforum_helpers\generate.py", line 283, in generate_inner
processed = processing.process_images(p)
File "C:\Users\va-ai.render\stable-diffusion-webui-sdxl\modules\processing.py", line 732, in process_images
res = process_images_inner(p)
File "C:\Users\va-ai.render\stable-diffusion-webui-sdxl\extensions\sd-webui-controlnet\scripts\batch_hijack.py", line 42, in processing_process_images_hijack
return getattr(processing, '__controlnet_original_process_images_inner')(p, *args, **kwargs)
File "C:\Users\va-ai.render\stable-diffusion-webui-sdxl\modules\processing.py", line 867, in process_images_inner
samples_ddim = p.sample(conditioning=p.c, unconditional_conditioning=p.uc, seeds=p.seeds, subseeds=p.subseeds, subseed_strength=p.subseed_strength, prompts=p.prompts)
File "C:\Users\va-ai.render\stable-diffusion-webui-sdxl\extensions\sd-webui-controlnet\scripts\hook.py", line 451, in process_sample
return process.sample_before_CN_hack(*args, **kwargs)
File "C:\Users\va-ai.render\stable-diffusion-webui-sdxl\modules\processing.py", line 1528, in sample
samples = self.sampler.sample_img2img(self, self.init_latent, x, conditioning, unconditional_conditioning, image_conditioning=self.image_conditioning)
File "C:\Users\va-ai.render\stable-diffusion-webui-sdxl\modules\sd_samplers_kdiffusion.py", line 188, in sample_img2img
samples = self.launch_sampling(t_enc + 1, lambda: self.func(self.model_wrap_cfg, xi, extra_args=self.sampler_extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs))
File "C:\Users\va-ai.render\stable-diffusion-webui-sdxl\modules\sd_samplers_common.py", line 261, in launch_sampling
return func()
File "C:\Users\va-ai.render\stable-diffusion-webui-sdxl\modules\sd_samplers_kdiffusion.py", line 188, in
samples = self.launch_sampling(t_enc + 1, lambda: self.func(self.model_wrap_cfg, xi, extra_args=self.sampler_extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs))
File "C:\Users\va-ai.render\stable-diffusion-webui-sdxl\venv\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "C:\Users\va-ai.render\stable-diffusion-webui-sdxl\repositories\k-diffusion\k_diffusion\sampling.py", line 701, in sample_dpmpp_3m_sde
h_1, h_2 = h, h_1
UnboundLocalError: local variable 'h' referenced before assignment
*END OF TRACEBACK*

User friendly error message:
Error: local variable 'h' referenced before assignment. Please, check your schedules/ init values.
Deforum progress: 3%|█▉ | 51/1662 [01:09<36:45, 1.37s/it]
```

### Additional information

_No response_

Contributor guide

Open the contributing guide

Research direction

Start with the traceback entry points in scripts/deforum_helpers/run_deforum.py, render.py, and generate.py, then reproduce the five-ControlNet reference-only animation using the linked Deforum and WebUI settings. Trace the second animation frame and optical-flow generation path to identify why h is unassigned; done means the 500-frame run completes without the UnboundLocalError.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.