Comfy-Org / Comfy-Org/ComfyUI

[Feature Request] Support for bundled LoRA/embedding files

Open
#3,659 2 comments 2 reactions 0 assignees View on GitHub
Feature
Dominant language
Python
Stars
133k
Forks
15.7k
Avg merge
1d 7h
Merged PRs (30d)
158

Description

Onetrainer recently added an option to simultaneously train embeddings and LoRAs (I think this is called pivotal tuning?) and bundle them into a single safetensor file. I believe the format for this was actually created a while ago (see the first link). I experimented a bit with training using this method and got good results, and it seems like it can be an effective method for multi-concept LoRAs.

In A1111/web UI these work as expected, just include " embeddingname" in the prompt. However I can't find a way to activate the embedding in a bundled file in Stable Swarm or the ComfyUI backend, only the LoRA. Separate embedding files work when loaded with but bundled files are a lot more convenient for large multi-concept LoRAs. Would support for this be something that can be implemented by ComfyUI?

Ideally, bundled embeddings should only be "accessible" when the associated LoRA is active, which is how web UI appears to work from my testing. Or there may be a better way to avoid conflicts between identically named embeddings in multiple bundle files.

Some links for reference:
[Original web UI pull request for the bundle format](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/13568)
[OneTrainer pull request](https://github.com/Nerogar/OneTrainer/pull/278)
[Relevant section of the OneTrainer code](https://github.com/Nerogar/OneTrainer/blob/master/modules/modelSaver/stableDiffusion/StableDiffusionEmbeddingSaver.py#L121)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.