huggingface / huggingface/swift-coreml-diffusers

Stable Diffusion 2.1-based models fail to load or generate blank black images

Open
#43 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Swift
Stars
2.8k
Forks
235
PR merge metrics
No merged PRs in 30d

Description

Hi šŸ‘‹

I’ve been doing some experiments with the app and, for the most part, it works. However, I’ve noticed that the `Diffusion` app fails to work with models that are based on **Stable Diffusion 2.1**.
- For `split_einsum_compiled` CoreML models, the app reports that it’s ā€œPreparing the model...ā€ or ā€œGenerating...ā€ in the console and never (after waiting hours) finishes or starts generating an image.
- For `original_compiled` CoreML models, the app does load the model but it produces images that are completely black; no images are generated.

This behavior is consistent among a variety of SD-2.1 checkpoints. Here are some that I’ve confirmed to have this issue:
- [Artius V2.1](https://civitai.com/models/27739/artius-v21)
- [rMada Merge - SD 2.1 768](https://civitai.com/models/15303/rmada-merge-sd-21-768)
- [Isometric Dreams SD2.1](https://civitai.com/models/8013/isometric-dreams-sd21) (GPU is fine; ANE isn’t)

Strangely, the `stable-diffusion-2-1-base` does work so I’m pretty confused and wanted to report this issue because this would heavily restrict users who want to use models beyond SD 1.x.

Even more peculiarly, this behavior isn’t unique to just this app. Other Stable Diffusion apps are exhibiting similar issues ([see here](https://github.com/godly-devotion/MochiDiffusion/issues/227))

—

System Info:
- Mac mini (M1, 2020)
- RAM: 16 GB
- macOS 13.3 (22E252)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.