Comfy-Org / Comfy-Org/ComfyUI

Using SD 3.5 Large Fine Tuned Models with Embedded Clip L + Clip G but with External T5

Open
#7,019 3 comments 0 reactions 0 assignees View on GitHub
Feature
Dominant language
Python
Stars
133k
Forks
15.7k
Avg merge
1d 6h
Merged PRs (30d)
155

Description

### Feature Idea

Currently it either loads all Clip L + Clip G + T5 from single safetensors file or

It loads all 3 externally

However what I want is loading Clip L + Clip G from trained safetensors file and loading T5 from external file

Because training T5 is usually useless. But training Clip models helps

Kohya supporting all these options but I talked with SwarmUI developer and checked from ComfyUI back-end, ComfyUI doesn't support such separate loading

You can see Kohya latest pull request here : https://github.com/kohya-ss/sd-scripts/pull/1960/files#diff-6b605dc86514575eab565c7ba50409469c1b6d4e88013e38ece90d9735340125

It works perfect I tested genearting a combined safetensors file with

DiT + Clip L + Clip G - ComfyUI can't use T5 in this case
and
DiT + Clip L + Clip G + T5 - ComfyUI uses T5 in this case but it makes files even larger totally unnecessarily

Thank you

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.