kohya-ss / kohya-ss/sd-scripts

Caption dropout at high noise

Open
#1,400 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
7.2k
Forks
1.2k
Avg merge
11m
Merged PRs (30d)
2

Description

Does caption dropout make sense at high noise levels? At low noise levels caption dropout might make a lot of sense because it should be possible to infer the correct image by looking at a slightly noised image. But I would think that captionless training at high noise levels would cause the model to learn weird priors.

Would it make sense to have an option to threshold caption dropout at certain noise levels? For example, 0.05 probability of caption dropout if noise level is below threshold X. At noise levels higher than X, caption dropout doesn't happen.

Contributor guide

No contributing guide indexed for this repository

Research direction

No file or test is named. First locate where caption dropout and noise levels are handled, then clarify the threshold semantics and expected behavior. The issue is done when the option limits caption dropout above the chosen noise level and that behavior is tested.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.