lllyasviel / lllyasviel/stable-diffusion-webui-forge
A new error occurs in xformers on flux model
- Dominant language
- Python
- Stars
- 13k
- Forks
- 1.7k
- PR merge metrics
- No merged PRs in 30d
Description
torch: 2.4.0+cu124
xformers: 0.0.28.dev877
"NoneType" error fixed, but new error occurs
Here`s log
```
Traceback (most recent call last):
File "D:\SDWebUIForge\modules_forge\main_thread.py", line 30, in work
self.result = self.func(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\SDWebUIForge\modules\txt2img.py", line 110, in txt2img_function
processed = processing.process_images(p)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\SDWebUIForge\modules\processing.py", line 809, in process_images
res = process_images_inner(p)
^^^^^^^^^^^^^^^^^^^^^^^
File "D:\SDWebUIForge\modules\processing.py", line 922, in process_images_inner
p.setup_conds()
File "D:\SDWebUIForge\modules\processing.py", line 1507, in setup_conds
super().setup_conds()
File "D:\SDWebUIForge\modules\processing.py", line 494, in setup_conds
self.c = self.get_conds_with_caching(prompt_parser.get_multicond_learned_conditioning, prompts, total_steps, [self.cached_c], self.extra_network_data)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\SDWebUIForge\modules\processing.py", line 463, in get_conds_with_caching
cache[1] = function(shared.sd_model, required_prompts, steps, hires_steps, shared.opts.use_old_scheduling)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\SDWebUIForge\modules\prompt_parser.py", line 262, in get_multicond_learned_conditioning
learned_conditioning = get_learned_conditioning(model, prompt_flat_list, steps, hires_steps, use_old_scheduling)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\SDWebUIForge\modules\prompt_parser.py", line 189, in get_learned_conditioning
conds = model.get_learned_conditioning(texts)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\SDWebUIForge\venv\Lib\site-packages\torch\utils\_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "D:\SDWebUIForge\backend\diffusion_engine\flux.py", line 79, in get_learned_conditioning
cond_t5 = self.text_processing_engine_t5(prompt)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\SDWebUIForge\backend\text_processing\t5_engine.py", line 123, in __call__
z = self.process_tokens([tokens], [multipliers])[0]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\SDWebUIForge\backend\text_processing\t5_engine.py", line 134, in process_tokens
z = self.encode_with_transformers(tokens)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\SDWebUIForge\backend\text_processing\t5_engine.py", line 60, in encode_with_transformers
z = self.text_encoder(
^^^^^^^^^^^^^^^^^^
File "D:\SDWebUIForge\venv\Lib\site-packages\torch\nn\modules\module.py", line 1553, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\SDWebUIForge\venv\Lib\site-packages\torch\nn\modules\module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\SDWebUIForge\backend\nn\t5.py", line 205, in forward
return self.encoder(x, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\SDWebUIForge\venv\Lib\site-packages\torch\nn\modules\module.py", line 1553, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\SDWebUIForge\venv\Lib\site-packages\torch\nn\modules\module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\SDWebUIForge\backend\nn\t5.py", line 186, in forward
x, past_bias = l(x, mask, past_bias)
^^^^^^^^^^^^^^^^^^^^^
File "D:\SDWebUIForge\venv\Lib\site-packages\torch\nn\modules\module.py", line 1553, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\SDWebUIForge\venv\Lib\site-packages\torch\nn\modules\module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\SDWebUIForge\backend\nn\t5.py", line 162, in forward
x, past_bias = self.layer[0](x, mask, past_bias)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\SDWebUIForge\venv\Lib\site-packages\torch\nn\modules\module.py", line 1553, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\SDWebUIForge\venv\Lib\site-packages\torch\nn\modules\module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\SDWebUIForge\backend\nn\t5.py", line 149, in forward
output, past_bias = self.SelfAttention(self.layer_norm(x), mask=mask, past_bias=past_bias)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\SDWebUIForge\venv\Lib\site-packages\torch\nn\modules\module.py", line 1553, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\SDWebUIForge\venv\Lib\site-packages\torch\nn\modules\module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\SDWebUIForge\backend\nn\t5.py", line 138, in forward
out = attention_function(q, k * ((k.shape[-1] / self.num_heads) ** 0.5), v, self.num_heads, mask)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\SDWebUIForge\backend\attention.py", line 314, in attention_xformers
mask_out[:, :, :mask.shape[-1]] = mask
~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: The expanded size of the tensor (1) must match the existing size (64) at non-singleton dimension 0. Target sizes: [1, 256, 256]. Tensor sizes: [64, 256, 256]
The expanded size of the tensor (1) must match the existing size (64) at non-singleton dimension 0. Target sizes: [1, 256, 256]. Tensor sizes: [64, 256, 256]
```
Contributor guide
No contributing guide indexed for this repository
Research direction
Reproduce the Flux generation failure with torch 2.4.0+cu124 and xformers 0.0.28.dev877, then follow the traceback from backend/diffusion_engine/flux.py through backend/text_processing/t5_engine.py and backend/nn/t5.py to backend/attention.py:314. Compare the mask and attention tensor shapes at the failing assignment; done means Flux generation completes without the reported RuntimeError.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- backend, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100