Using SD 3.5 Large Fine Tuned Models with Embedded Clip L + Clip G but with External T5
- Dominant language
- Python
- Stars
- 133k
- Forks
- 15.7k
- Avg merge
- 1d 6h
- Merged PRs (30d)
- 155
Description
### Feature Idea
Currently it either loads all Clip L + Clip G + T5 from single safetensors file or
It loads all 3 externally
However what I want is loading Clip L + Clip G from trained safetensors file and loading T5 from external file
Because training T5 is usually useless. But training Clip models helps
Kohya supporting all these options but I talked with SwarmUI developer and checked from ComfyUI back-end, ComfyUI doesn't support such separate loading
You can see Kohya latest pull request here : https://github.com/kohya-ss/sd-scripts/pull/1960/files#diff-6b605dc86514575eab565c7ba50409469c1b6d4e88013e38ece90d9735340125
It works perfect I tested genearting a combined safetensors file with
DiT + Clip L + Clip G - ComfyUI can't use T5 in this case
and
DiT + Clip L + Clip G + T5 - ComfyUI uses T5 in this case but it makes files even larger totally unnecessarily
Thank you
Contributor guide
Assessment
This issue has not been assessed yet.