Comfy-Org / Comfy-Org/ComfyUI

Universal was to get state_dict keys for svd model?

Open
#4,145 2 comments 0 reactions 0 assignees View on GitHub
Stale User Support
Dominant language
Python
Stars
133k
Forks
15.7k
Avg merge
1d 6h
Merged PRs (30d)
155

Description

### Your question

On a custom node I am making, I have spent months trying to get the actual `state_dict` keys for the svd model as they don't match the huggingface names when you click the little layer icon. I need the comfyui post prefixed names for clip_vision, unet. I want it to print all of the names used and even the open clip ones after `positional.embedding`.

I noticed comfyui renames a bunch of the `state_dict` key prefixes and this makes it very difficult for targeting weight and bias layers directly when using comfyui. I think I am definitely missing something here. I see some classes that include `layer_idx` at least, but when I use their methods to get the state_dict keys (after importing) it doesn't work.

In addition, can we get some comments for sd1_clip.py, the sdxl_clip.py, the class SDXLClipG, the SDXLClipGTokenizer, and other classes like that. These seems to be exactly what I am looking but it took 4 months to finally discover them. I need to be able to get the actual `state_dict` names comfyui renames everything in the svd model to.

I don't mind the `state_dict_prefix_replace` in the sd.py if it's necessary, but there needs to still be a way to get the prefixed names. Unless I am missing something here. Thanks!

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.