lllyasviel / lllyasviel/sd-forge-layerdiffuse

Transparency with Highres Fix Doesn't work anymore

Open
#119 12 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
4.1k
Forks
352
PR merge metrics
No merged PRs in 30d

Description

Hello

When i'm using Highres Fix, the image doesn't get transparency. That worked few day ago with the same setting.

I got this error (i don't know if this related) :

Error running postprocess_image_after_composite: /content/drive/MyDrive/stable-diffusion-webui-forge/extensions/sd-forge-layerdiffuse/scripts/forge_layerdiffusion.py
Traceback (most recent call last):
File "/content/drive/MyDrive/stable-diffusion-webui-forge/modules/scripts.py", line 956, in postprocess_image_after_composite
script.postprocess_image_after_composite(p, pp, *script_args)
File "/content/drive/MyDrive/stable-diffusion-webui-forge/extensions/sd-forge-layerdiffuse/scripts/forge_layerdiffusion.py", line 376, in postprocess_image_after_composite
png, vis = vae_transparent_decoder.decode(latent, pixel)
File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/content/drive/MyDrive/stable-diffusion-webui-forge/extensions/sd-forge-layerdiffuse/lib_layerdiffusion/models.py", line 278, in decode
y = self.estimate_augmented(pixel, latent)
File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/content/drive/MyDrive/stable-diffusion-webui-forge/extensions/sd-forge-layerdiffuse/lib_layerdiffusion/models.py", line 259, in estimate_augmented
eps = self.estimate_single_pass(feed_pixel, feed_latent).clip(0, 1)
File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/content/drive/MyDrive/stable-diffusion-webui-forge/extensions/sd-forge-layerdiffuse/lib_layerdiffusion/models.py", line 237, in estimate_single_pass
y = self.model.model(pixel, latent)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1541, in _call_impl
return forward_call(*args, **kwargs)
File "/content/drive/MyDrive/stable-diffusion-webui-forge/extensions/sd-forge-layerdiffuse/lib_layerdiffusion/models.py", line 159, in forward
sample_latent = self.latent_conv_in(latent)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1541, in _call_impl
return forward_call(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/conv.py", line 460, in forward
return self._conv_forward(input, self.weight, self.bias)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/conv.py", line 456, in _conv_forward
return F.conv2d(input, weight, bias, self.stride,
RuntimeError: Given groups=1, weight of size [64, 4, 1, 1], expected input[1, 3, 268, 268] to have 4 channels, but got 3 channels instead

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in extensions/sd-forge-layerdiffuse/scripts/forge_layerdiffusion.py at postprocess_image_after_composite, then trace the decode path through lib_layerdiffusion/models.py at lines 278, 259, 237, and 159. Reproduce the issue with Highres Fix and inspect the tensor shapes reaching latent_conv_in. Done means transparency works again without the reported channel-mismatch error.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
ai, computer-vision
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.