lllyasviel / lllyasviel/sd-forge-layerdiffuse

[Intel Arc] [Windows] Exceptions both with SD 1.5 and SDXL generation

Open
#60 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
4.1k
Forks
352
PR merge metrics
No merged PRs in 30d

Description

Hello! As the title states, extension does not seem to work correctly on Intel Arc GPUs. Specifically, when I try to run inference with SDXL image and extension enabled, I get the following exception:

File "c:\Forge\modules_forge\main_thread.py", line 37, in loop
    task.work()
  File "c:\Forge\modules_forge\main_thread.py", line 26, in work
    self.result = self.func(*self.args, **self.kwargs)
  File "C:\Forge\modules\txt2img.py", line 111, in txt2img_function
    processed = processing.process_images(p)
  File "C:\Forge\modules\processing.py", line 752, in process_images
    res = process_images_inner(p)
  File "C:\Forge\modules\processing.py", line 938, in process_images_inner
    x_samples_ddim = decode_latent_batch(p.sd_model, samples_ddim, target_device=devices.cpu, check_for_nans=True)
  File "C:\Forge\modules\processing.py", line 638, in decode_latent_batch
    sample = decode_first_stage(model, batch[i:i + 1])[0]
  File "C:\Forge\modules\sd_samplers_common.py", line 74, in decode_first_stage
    return samples_to_images_tensor(x, approx_index, model)
  File "C:\Forge\modules\sd_samplers_common.py", line 57, in samples_to_images_tensor
    x_sample = model.decode_first_stage(sample)
  File "c:\Forge\venv\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "C:\Forge\modules_forge\forge_loader.py", line 239, in patched_decode_first_stage
    sample = sd_model.forge_objects.vae.decode(sample).movedim(-1, 1) * 2.0 - 1.0
  File "C:\Forge\ldm_patched\modules\sd.py", line 288, in decode
    return wrapper(self.decode_inner, samples_in)
  File "c:\Forge\venv\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "C:\Forge\extensions\sd-forge-layerdiffuse\lib_layerdiffusion\models.py", line 256, in wrapper
    y = self.estimate_augmented(pixel[i:i+1], latent[i:i+1])
  File "c:\Forge\venv\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "C:\Forge\extensions\sd-forge-layerdiffuse\lib_layerdiffusion\models.py", line 234, in estimate_augmented
    median = torch.median(result, dim=0).values
RuntimeError: Provided range is out of integer limits. Pass `-fno-sycl-id-queries-fit-in-int' to disable range check. -30 (PI_ERROR_INVALID_VALUE)

With SD 1.5, I run into the following (excerpt):

File "C:\Forge\ldm_patched\ldm\modules\attention.py", line 447, in forward
    return checkpoint(self._forward, (x, context, transformer_options), self.parameters(), self.checkpoint)
  File "C:\Forge\ldm_patched\ldm\modules\diffusionmodules\util.py", line 194, in checkpoint
    return func(*inputs)
  File "C:\Forge\ldm_patched\ldm\modules\attention.py", line 507, in _forward
    n = self.attn1(n, context=context_attn1, value=value_attn1)
  File "c:\Forge\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "C:\Forge\extensions\sd-forge-layerdiffuse\lib_layerdiffusion\attention_sharing.py", line 92, in forward
    framed_cond_mark = einops.rearrange(transformer_options['cond_mark'], '(b f) -> f b', f=self.frames).to(modified_hidden_states)
KeyError: 'cond_mark'

Forge runs on Windows natively, GPU in use is Intel Arc A770 16GB. Happy to provide any other details as necessary.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing SDXL and SD 1.5 generation on Windows with an Intel Arc A770 and the extension enabled. Inspect lib_layerdiffusion/models.py around estimate_augmented and attention_sharing.py around the cond_mark lookup, using the reported tracebacks as entry points. Done means both generation paths complete without the reported exceptions.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.