lllyasviel / lllyasviel/ControlNet
depth_leres++ can't run in M1 Pro
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 34.1k
- Forks
- 3k
- PR merge metrics
- No merged PRs in 30d
Description
A1111 version: v1.5
Controlnet version: v1.1.233
torch version: v2.0.1
torchversion version: 0.15.2
MetalPerformanceShaders version: v1.0
2023-07-27 10:32:50,095 - ControlNet - INFO - Preview Resolution = 512
Traceback (most recent call last):
File "/Users/simplemin/stable-diffusion-webui/venv/lib/python3.10/site-packages/gradio/routes.py", line 422, in run_predict
output = await app.get_blocks().process_api(
File "/Users/simplemin/stable-diffusion-webui/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1323, in process_api
result = await self.call_function(
File "/Users/simplemin/stable-diffusion-webui/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1051, in call_function
prediction = await anyio.to_thread.run_sync(
File "/Users/simplemin/stable-diffusion-webui/venv/lib/python3.10/site-packages/anyio/to_thread.py", line 33, in run_sync
return await get_asynclib().run_sync_in_worker_thread(
File "/Users/simplemin/stable-diffusion-webui/venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 877, in run_sync_in_worker_thread
return await future
File "/Users/simplemin/stable-diffusion-webui/venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 807, in run
result = context.run(func, *args)
File "/Users/simplemin/stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/controlnet_ui/controlnet_ui_group.py", line 606, in run_annotator
result, is_image = preprocessor(
File "/Users/simplemin/stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/utils.py", line 79, in decorated_func
return cached_func(*args, **kwargs)
File "/Users/simplemin/stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/utils.py", line 67, in cached_func
return func(*args, **kwargs)
File "/Users/simplemin/stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/global_state.py", line 35, in unified_preprocessor
return preprocessor_modules[preprocessor_name](*args, **kwargs)
File "/Users/simplemin/stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/processor.py", line 211, in leres
result = model_leres(img, thr_a, thr_b, boost=boost)
File "/Users/simplemin/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/leres/init.py", line 79, in apply_leres
depth = estimateboost(input_image, model, 0, pix2pixmodel, max(width, height))
File "/Users/simplemin/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/leres/leres/depthmap.py", line 418, in estimateboost
whole_estimate = doubleestimate(img, net_receptive_field_size, whole_image_optimal_size, pix2pixsize, model, model_type, pix2pixmodel)
File "/Users/simplemin/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/leres/leres/depthmap.py", line 132, in doubleestimate
estimate1 = singleestimate(img, size1, model, net_type)
File "/Users/simplemin/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/leres/leres/depthmap.py", line 156, in singleestimate
return estimateleres(img, model, msize, msize)
File "/Users/simplemin/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/leres/leres/depthmap.py", line 43, in estimateleres
prediction = model.depth_model(img_torch)
File "/Users/simplemin/stable-diffusion-webui/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/Users/simplemin/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/leres/leres/multi_depth_model_woauxi.py", line 32, in forward
lateral_out = self.encoder_modules(x)
File "/Users/simplemin/stable-diffusion-webui/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/Users/simplemin/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/leres/leres/network_auxi.py", line 96, in forward
x = self.encoder(x) # 1/32, 1/16, 1/8, 1/4
File "/Users/simplemin/stable-diffusion-webui/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/Users/simplemin/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/leres/leres/Resnext_torch.py", line 223, in forward
return self._forward_impl(x)
File "/Users/simplemin/stable-diffusion-webui/extensions/sd-webui-controlnet/annotator/leres/leres/Resnext_torch.py", line 199, in _forward_impl
x = self.conv1(x)
File "/Users/simplemin/stable-diffusion-webui/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/Users/simplemin/stable-diffusion-webui/extensions-builtin/Lora/networks.py", line 371, in network_Conv2d_forward
return torch.nn.Conv2d_forward_before_network(self, input)
File "/Users/simplemin/stable-diffusion-webui/venv/lib/python3.10/site-packages/torch/nn/modules/conv.py", line 463, in forward
return self._conv_forward(input, self.weight, self.bias)
File "/Users/simplemin/stable-diffusion-webui/venv/lib/python3.10/site-packages/torch/nn/modules/conv.py", line 459, in _conv_forward
return F.conv2d(input, weight, bias, self.stride,
RuntimeError: Input type (MPSFloatType) and weight type (torch.FloatTensor) should be the same
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with annotator/leres/leres/depthmap.py and annotator/leres/leres/Resnext_torch.py, following the traceback into the first convolution in the MPS execution path. Reproduce depth_leres++ on an M1 Pro with the listed versions; done means the preprocessor completes without the MPSFloatType and torch.FloatTensor mismatch.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100