lllyasviel / lllyasviel/ControlNet
Complex types are unsupported on MPS
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 34.1k
- Forks
- 3k
- PR merge metrics
- No merged PRs in 30d
Description
I'm trying to use inpaint+lama and get the following error during preview:
````python
Traceback (most recent call last):
File "/Users/mariodian/Source/AI/automatic/venv/lib/python3.10/site-packages/gradio/routes.py", line 422, in run_predict
output = await app.get_blocks().process_api(
File "/Users/mariodian/Source/AI/automatic/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1323, in process_api
result = await self.call_function(
File "/Users/mariodian/Source/AI/automatic/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1051, in call_function
prediction = await anyio.to_thread.run_sync(
File "/Users/mariodian/Source/AI/automatic/venv/lib/python3.10/site-packages/anyio/to_thread.py", line 33, in run_sync
return await get_asynclib().run_sync_in_worker_thread(
File "/Users/mariodian/Source/AI/automatic/venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 877, in run_sync_in_worker_thread
return await future
File "/Users/mariodian/Source/AI/automatic/venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 807, in run
result = context.run(func, *args)
File "/Users/mariodian/Source/AI/automatic/extensions-builtin/sd-webui-controlnet/scripts/controlnet_ui/controlnet_ui_group.py", line 636, in run_annotator
result, is_image = preprocessor(
File "/Users/mariodian/Source/AI/automatic/extensions-builtin/sd-webui-controlnet/scripts/utils.py", line 76, in decorated_func
return cached_func(*args, **kwargs)
File "/Users/mariodian/Source/AI/automatic/extensions-builtin/sd-webui-controlnet/scripts/utils.py", line 64, in cached_func
return func(*args, **kwargs)
File "/Users/mariodian/Source/AI/automatic/extensions-builtin/sd-webui-controlnet/scripts/global_state.py", line 35, in unified_preprocessor
return preprocessor_modules[preprocessor_name](*args, **kwargs)
File "/Users/mariodian/Source/AI/automatic/extensions-builtin/sd-webui-controlnet/scripts/processor.py", line 489, in lama_inpaint
prd_color = model_lama(img_res)
File "/Users/mariodian/Source/AI/automatic/extensions-builtin/sd-webui-controlnet/annotator/lama/__init__.py", line 54, in __call__
result = self.model(image_feed)[0]
File "/Users/mariodian/Source/AI/automatic/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1502, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/Users/mariodian/Source/AI/automatic/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _call_impl
return forward_call(*args, **kwargs)
File "/Users/mariodian/Source/AI/automatic/extensions-builtin/sd-webui-controlnet/annotator/lama/saicinpainting/training/modules/ffc.py", line 419, in forward
return self.model(input)
File "/Users/mariodian/Source/AI/automatic/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1502, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/Users/mariodian/Source/AI/automatic/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _call_impl
return forward_call(*args, **kwargs)
File "/Users/mariodian/Source/AI/automatic/venv/lib/python3.10/site-packages/torch/nn/modules/container.py", line 217, in forward
input = module(input)
File "/Users/mariodian/Source/AI/automatic/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1502, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/Users/mariodian/Source/AI/automatic/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _call_impl
return forward_call(*args, **kwargs)
File "/Users/mariodian/Source/AI/automatic/extensions-builtin/sd-webui-controlnet/annotator/lama/saicinpainting/training/modules/ffc.py", line 337, in forward
x_l, x_g = self.conv1((x_l, x_g))
File "/Users/mariodian/Source/AI/automatic/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1502, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/Users/mariodian/Source/AI/automatic/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _call_impl
return forward_call(*args, **kwargs)
File "/Users/mariodian/Source/AI/automatic/extensions-builtin/sd-webui-controlnet/annotator/lama/saicinpainting/training/modules/ffc.py", line 304, in forward
x_l, x_g = self.ffc(x)
File "/Users/mariodian/Source/AI/automatic/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1502, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/Users/mariodian/Source/AI/automatic/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _call_impl
return forward_call(*args, **kwargs)
File "/Users/mariodian/Source/AI/automatic/extensions-builtin/sd-webui-controlnet/annotator/lama/saicinpainting/training/modules/ffc.py", line 275, in forward
out_xg = self.convl2g(x_l) * l2g_gate + self.convg2g(x_g)
File "/Users/mariodian/Source/AI/automatic/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1502, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/Users/mariodian/Source/AI/automatic/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _call_impl
return forward_call(*args, **kwargs)
File "/Users/mariodian/Source/AI/automatic/extensions-builtin/sd-webui-controlnet/annotator/lama/saicinpainting/training/modules/ffc.py", line 198, in forward
output = self.fu(x)
File "/Users/mariodian/Source/AI/automatic/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1502, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/Users/mariodian/Source/AI/automatic/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _call_impl
return forward_call(*args, **kwargs)
File "/Users/mariodian/Source/AI/automatic/extensions-builtin/sd-webui-controlnet/annotator/lama/saicinpainting/training/modules/ffc.py", line 86, in forward
ffted = torch.fft.rfftn(x, dim=fft_dim, norm=self.fft_norm)
TypeError: Complex types are unsupported on MPS
````
and this error during image generation:
````
13:18:06-665653 ERROR Running script process: /Users/mariodian/Source/AI/automatic/extensions-builtin/sd-webui-controlnet/scripts/controlnet.py: TypeError
╭───────────────────────────────────────────────────────────────────────────────── Traceback (most recent call last) ──────────────────────────────────────────────────────────────────────────────────╮
│ /Users/mariodian/Source/AI/automatic/modules/scripts.py:406 in process │
│ │
│ 405 │ │ │ │ args = p.per_script_args.get(script.title(), p.script_args[script.args_f │
│ ❱ 406 │ │ │ │ script.process(p, *args, **kwargs) │
│ 407 │ │ │ │ s.append(f'{script.title()}:{round(time.time()-t0, 2)}s') │
│ │
│ /Users/mariodian/Source/AI/automatic/extensions-builtin/sd-webui-controlnet/scripts/controlnet.py:829 in process │
│ │
│ 828 │ │ │ logger.debug(f"Use numpy seed {seed}.") │
│ ❱ 829 │ │ │ detected_map, is_image = preprocessor( │
│ 830 │ │ │ │ input_image, │
│ │
│ ... 24 frames hidden ... │
│ │
│ /Users/mariodian/Source/AI/automatic/venv/lib/python3.10/site-packages/torch/nn/modules/module.py:1511 in _call_impl │
│ │
│ 1510 │ │ │ │ or _global_forward_hooks or _global_forward_pre_hooks): │
│ ❱ 1511 │ │ │ return forward_call(*args, **kwargs) │
│ 1512 │ │ # Do not call functions when jit is used │
│ │
│ /Users/mariodian/Source/AI/automatic/extensions-builtin/sd-webui-controlnet/annotator/lama/saicinpainting/training/modules/ffc.py:86 in forward │
│ │
│ 85 │ │ fft_dim = (-3, -2, -1) if self.ffc3d else (-2, -1) │
│ ❱ 86 │ │ ffted = torch.fft.rfftn(x, dim=fft_dim, norm=self.fft_norm) │
│ 87 │ │ ffted = torch.stack((ffted.real, ffted.imag), dim=-1) │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
TypeError: Complex types are unsupported on MPS
````
# Specs
HW: mac mini m2 pro, 32gb ram
OS: macOS ventura 13.4
SD webui: latest [vladmandic/automatic](https://github.com/vladmandic/automatic)
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the inpaint+lama preview and image-generation paths on the reported MPS setup. Start at extensions-builtin/sd-webui-controlnet/annotator/lama/saicinpainting/training/modules/ffc.py:86, where torch.fft.rfftn raises the reported error, and trace how that path is reached from the ControlNet preprocessor. Done means both reported operations no longer fail with the MPS complex-type error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100