kohya-ss / kohya-ss/sd-scripts
[Feature request] --caption_dropout_rate when cache te output
- Dominant language
- Python
- Stars
- 7.2k
- Forks
- 1.2k
- Avg merge
- 11m
- Merged PRs (30d)
- 2
Description
[flux-style-captioning-differences-training-diary](https://civitai.com/articles/6792/flux-style-captioning-differences-training-diary)
[flux-style-captioning-differences-pt2-4-new-caption-tools-training-diary](https://civitai.com/articles/7146/flux-style-captioning-differences-pt2-4-new-caption-tools-training-diary)
[flux-is-smarter-than-you-and-other-surprising-findings-on-making-the-model-your-own](https://civitai.com/articles/6982/flux-is-smarter-than-you-and-other-surprising-findings-on-making-the-model-your-own)
Some FLUX training experience comparisons point out that FLUX performs well even when trained without captions. However, when testing with entirely captionless data, it becomes difficult to subdivide the concepts one wishes to add. When using detailed captions, however, adding some probability of no captions can increase the similarity of the subject when using short prompts or simple trigger words. In actual tests, mixing with a 1:1 ratio is too high, showing signs of overfixing the main features, but it still produces significantly better results. It would be better if the ratio could be freely controlled.
Currently, if you want to use a caption dropout rate when caching the te output, you need to duplicate all the images and delete the text files, naming their directory something like "1_". This process generates an 8MB te cache for each image, which is identical. It would be preferable to integrate this with the existing caption dropout rate function to achieve a similar effect with just a single te cache.
Contributor guide
No contributing guide indexed for this repository
Research direction
The issue names no files or tests; begin by locating the existing caption-dropout implementation and the text-encoder output caching entry points. Done means allowing a configurable caption dropout rate while reusing a single text-encoder cache per image instead of duplicating images and caches.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100