kohya-ss / kohya-ss/sd-scripts

[Feature request] --caption_dropout_rate when cache te output

Open
#1,583 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
7.2k
Forks
1.2k
Avg merge
11m
Merged PRs (30d)
2

Description

[flux-style-captioning-differences-training-diary](https://civitai.com/articles/6792/flux-style-captioning-differences-training-diary)
[flux-style-captioning-differences-pt2-4-new-caption-tools-training-diary](https://civitai.com/articles/7146/flux-style-captioning-differences-pt2-4-new-caption-tools-training-diary)
[flux-is-smarter-than-you-and-other-surprising-findings-on-making-the-model-your-own](https://civitai.com/articles/6982/flux-is-smarter-than-you-and-other-surprising-findings-on-making-the-model-your-own)

Some FLUX training experience comparisons point out that FLUX performs well even when trained without captions. However, when testing with entirely captionless data, it becomes difficult to subdivide the concepts one wishes to add. When using detailed captions, however, adding some probability of no captions can increase the similarity of the subject when using short prompts or simple trigger words. In actual tests, mixing with a 1:1 ratio is too high, showing signs of overfixing the main features, but it still produces significantly better results. It would be better if the ratio could be freely controlled.

Currently, if you want to use a caption dropout rate when caching the te output, you need to duplicate all the images and delete the text files, naming their directory something like "1_". This process generates an 8MB te cache for each image, which is identical. It would be preferable to integrate this with the existing caption dropout rate function to achieve a similar effect with just a single te cache.

Contributor guide

No contributing guide indexed for this repository

Research direction

The issue names no files or tests; begin by locating the existing caption-dropout implementation and the text-encoder output caching entry points. Done means allowing a configurable caption dropout rate while reusing a single text-encoder cache per image instead of duplicating images and caches.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.