Comfy-Org / Comfy-Org/ComfyUI

Auto Detection support for "more" SDXL VPred Models

Open
#12,579 0 comments 0 reactions 0 assignees View on GitHub
Feature
Dominant language
Python
Stars
133k
Forks
15.7k
Avg merge
1d 7h
Merged PRs (30d)
158

Description

### Feature Idea

- [Location as Git Blame](https://github.com/Comfy-Org/ComfyUI/blame/master/comfy/supported_models.py#L214)

- I see the "some zsnr anime checkpoints" were mentioned. However I don't find any code reference for any popular model trainer suchas [Kohyas](https://github.com/kohya-ss/sd-scripts/blob/main/library/train_util.py#L6232) and [NaifuDIffusion](https://github.com/Mikubill/naifu/blob/main/modules/sdxl_model.py#L273) (maybe Diffusion Pipe or else).

- Instead of **placeholder layers**, I found that **model metadata** has been explictly set with `prediction_type="v"` or `prediction_type="v_prediction"`. Hopefully it doesn't rely on checking model layers, which VPred Models up to SDXL was relying `alpha_cumprods` layers instead of `v_pred` or `ztsnr`. Ref: [DDPMScheduler](https://github.com/huggingface/diffusers/blob/main/src/diffusers/schedulers/scheduling_ddpm.py#L598) in `diffusers`.

- Affected model: [CivitAI link](https://civitai.com/models/1898715?modelVersionId=2705799) which is not NoobAI VPred.

### Existing Solutions

- Just use [Model Sampling Discrete](https://comfyui-wiki.com/en/comfyui-nodes/advanced/model/model-sampling-discrete) to override the auto detection result.

### Other

*It works in both A1111 and ReForge.*

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.