lllyasviel / lllyasviel/sd-forge-layerdiffuse

Incorrect results when using img2img batch mode

Open
#115 1 comment 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
4.1k
Forks
352
PR merge metrics
No merged PRs in 30d

Description

Now I've tested again and can start generating, but the resulting image is only the first image. It's like the loop isn't working properly.

https://github.com/lllyasviel/stable-diffusion-webui-forge/commit/f94bcae7b00f30a9ef0540277770e5acdbd12c72

```[LayerDiffuse] LayerMethod.FG_ONLY_ATTN_SD15
Reuse 1 loaded models
To load target model BaseModel
Begin to load 1 model
[Memory Management] Current Free GPU Memory (MB) = 6634.41796875
[Memory Management] Model Memory (MB) = 0.0
[Memory Management] Minimal Inference Memory (MB) = 1024.0
[Memory Management] Estimated Remaining GPU Memory (MB) = 5610.41796875
Moving model(s) has taken 0.07 seconds

*** Error running before_process_init_images: D:\webui_forge_cu121_torch21\webui\extensions\sd-forge-layerdiffuse\scripts\forge_layerdiffusion.py
Traceback (most recent call last):
File "D:\webui_forge_cu121_torch21\webui\modules\scripts.py", line 868, in before_process_init_images
script.before_process_init_images(p, pp, *script_args, **kwargs)
File "D:\webui_forge_cu121_torch21\webui\extensions\sd-forge-layerdiffuse\scripts\forge_layerdiffusion.py", line 429, in before_process_init_images
latent_offset = vae_transparent_encoder.encode(image)
File "D:\webui_forge_cu121_torch21\system\python\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "D:\webui_forge_cu121_torch21\webui\extensions\sd-forge-layerdiffuse\lib_layerdiffusion\models.py", line 319, in encode
list_of_np_rgb_padded = [pad_rgb(x) for x in list_of_np_rgba_hwc_uint8]
File "D:\webui_forge_cu121_torch21\webui\extensions\sd-forge-layerdiffuse\lib_layerdiffusion\models.py", line 319, in
list_of_np_rgb_padded = [pad_rgb(x) for x in list_of_np_rgba_hwc_uint8]
File "D:\webui_forge_cu121_torch21\webui\extensions\sd-forge-layerdiffuse\lib_layerdiffusion\models.py", line 207, in pad_rgb
pyramid = build_alpha_pyramid(color=np_rgba_hwc[..., :3], alpha=np_rgba_hwc[..., 3:])
File "D:\webui_forge_cu121_torch21\webui\extensions\sd-forge-layerdiffuse\lib_layerdiffusion\models.py", line 190, in build_alpha_pyramid
current_premultiplied_color = color * alpha
ValueError: operands could not be broadcast together with shapes (512,512,3) (512,512,0)```

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the img2img batch case and start in scripts/forge_layerdiffusion.py at before_process_init_images, then trace into lib_layerdiffusion/models.py through encode, pad_rgb, and build_alpha_pyramid. Check why the batch input reaches build_alpha_pyramid with an empty alpha dimension. Done means batch generation completes without the broadcasting error and produces a result for each image.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.