lllyasviel / lllyasviel/stable-diffusion-webui-forge

[Bug] 4GB fix does not appear to work on Intel Arc

Open
#523 5 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
13k
Forks
1.7k
PR merge metrics
No merged PRs in 30d

Description

### Checklist

- [X] The issue exists after disabling all extensions
- [X] The issue exists on a clean installation of webui
- [ ] The issue is caused by an extension, but I believe it is caused by a bug in the webui
- [X] The issue exists in the current version of the webui
- [X] The issue has not been reported before recently
- [ ] The issue has been reported before but has not been fixed yet

### What happened?

I am running latest Forge on Windows natively with Intel Arc A770 16GB GPU. Whenever I add a slightly beefier task to the queue (minimal example being batch of 4 images at 512x768, no Loras, SD 1.5), inference will crash with

```
RuntimeError: Current platform can NOT allocate memory block with size larger than 4GB! Tried to allocate 4.50 GiB (GPU 0; 15.56 GiB total capacity; 487.30 MiB already allocated; 510.00 MiB reserved in total by PyTorch)
```

From the looks of it, `xpu_specific.py` file containing the fix for 4GB limitation problem is never invoked (tested by adding `print` statements to the script, both at the start of the file and within the device checking/SD hijacking functions).

### Steps to reproduce the problem

On Arc A770, start a batch of 4 images at 512x768 on any SD 1.5 model, then wait until you see an exception in the console.

### What should have happened?

Inference should have finished normally.

### What browsers do you use to access the UI ?

Google Chrome

### Sysinfo

[sysinfo-2024-03-09-20-15.json](https://github.com/lllyasviel/stable-diffusion-webui-forge/files/14547696/sysinfo-2024-03-09-20-15.json)

### Console logs

```Shell
File "C:\Forge\modules_forge\main_thread.py", line 37, in loop
task.work()
File "C:\Forge\modules_forge\main_thread.py", line 26, in work
self.result = self.func(*self.args, **self.kwargs)
File "C:\Forge\modules\txt2img.py", line 111, in txt2img_function
processed = processing.process_images(p)
File "C:\Forge\modules\processing.py", line 752, in process_images
res = process_images_inner(p)
File "C:\Forge\modules\processing.py", line 922, in process_images_inner
samples_ddim = p.sample(conditioning=p.c, unconditional_conditioning=p.uc, seeds=p.seeds, subseeds=p.subseeds, subseed_strength=p.subseed_strength, prompts=p.prompts)
File "C:\Forge\modules\processing.py", line 1275, in sample
samples = self.sampler.sample(self, x, conditioning, unconditional_conditioning, image_conditioning=self.txt2img_image_conditioning(x))
File "C:\Forge\modules\sd_samplers_kdiffusion.py", line 251, in sample
samples = self.launch_sampling(steps, lambda: self.func(self.model_wrap_cfg, x, extra_args=self.sampler_extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs))
File "C:\Forge\modules\sd_samplers_common.py", line 263, in launch_sampling
return func()
File "C:\Forge\modules\sd_samplers_kdiffusion.py", line 251, in
samples = self.launch_sampling(steps, lambda: self.func(self.model_wrap_cfg, x, extra_args=self.sampler_extra_args, disable=False, callback=self.callback_state, **extra_params_kwargs))
File "C:\Forge\venv\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "C:\Forge\repositories\k-diffusion\k_diffusion\sampling.py", line 594, in sample_dpmpp_2m
denoised = model(x, sigmas[i] * s_in, **extra_args)
File "C:\Forge\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "C:\Forge\modules\sd_samplers_cfg_denoiser.py", line 182, in forward
denoised = forge_sampler.forge_sample(self, denoiser_params=denoiser_params,
File "C:\Forge\modules_forge\forge_sampler.py", line 88, in forge_sample
denoised = sampling_function(model, x, timestep, uncond, cond, cond_scale, model_options, seed)
File "C:\Forge\ldm_patched\modules\samplers.py", line 289, in sampling_function
cond_pred, uncond_pred = calc_cond_uncond_batch(model, cond, uncond_, x, timestep, model_options)
File "C:\Forge\ldm_patched\modules\samplers.py", line 258, in calc_cond_uncond_batch
output = model.apply_model(input_x, timestep_, **c).chunk(batch_chunks)
File "C:\Forge\ldm_patched\modules\model_base.py", line 90, in apply_model
model_output = self.diffusion_model(xc, t, context=context, control=control, transformer_options=transformer_options, **extra_conds).float()
File "C:\Forge\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "C:\Forge\ldm_patched\ldm\modules\diffusionmodules\openaimodel.py", line 867, in forward
h = forward_timestep_embed(module, h, emb, context, transformer_options, time_context=time_context, num_video_frames=num_video_frames, image_only_indicator=image_only_indicator)
File "C:\Forge\ldm_patched\ldm\modules\diffusionmodules\openaimodel.py", line 55, in forward_timestep_embed
x = layer(x, context, transformer_options)
File "C:\Forge\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "C:\Forge\ldm_patched\ldm\modules\attention.py", line 620, in forward
x = block(x, context=context[i], transformer_options=transformer_options)
File "C:\Forge\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "C:\Forge\ldm_patched\ldm\modules\attention.py", line 447, in forward
return checkpoint(self._forward, (x, context, transformer_options), self.parameters(), self.checkpoint)
File "C:\Forge\ldm_patched\ldm\modules\diffusionmodules\util.py", line 194, in checkpoint
return func(*inputs)
File "C:\Forge\ldm_patched\ldm\modules\attention.py", line 507, in _forward
n = self.attn1(n, context=context_attn1, value=value_attn1)
File "C:\Forge\venv\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "C:\Forge\ldm_patched\ldm\modules\attention.py", line 399, in forward
out = optimized_attention(q, k, v, self.heads)
File "C:\Forge\ldm_patched\ldm\modules\attention.py", line 332, in attention_pytorch
out = torch.nn.functional.scaled_dot_product_attention(q, k, v, attn_mask=mask, dropout_p=0.0, is_causal=False)
RuntimeError: Current platform can NOT allocate memory block with size larger than 4GB! Tried to allocate 4.50 GiB (GPU 0; 15.56 GiB total capacity; 487.30 MiB already allocated; 510.00 MiB reserved in total by PyTorch)
```

### Additional information

_No response_

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with xpu_specific.py and trace whether its device-checking and SD-hijacking paths are reached during the reported Arc A770 run. Follow the stack from modules/txt2img.py through modules_forge/forge_sampler.py and ldm_patched/modules/attention.py, then reproduce the 512x768 batch-of-4 SD 1.5 case. Done means inference completes without the 4GB allocation error.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.