kohya-ss / kohya-ss/sd-scripts

Allow more granular control over which timesteps are chosen during training

Open
#2,247 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
7.2k
Forks
1.2k
Avg merge
11m
Merged PRs (30d)
2

Description

Timesteps are randomly chosen during training within a specified range, defaulting to 0 to 1000. You can only adjust min and max range for timesteps chosen. However, because the min and max range apply to all training steps/epochs, it would only result in undertrained loras if anything besides the default settings are used. It would be better if there's a piecewise function parameter that allows you to choose a timestep for each step similar to the piecewise function scheduler for the learning rate. This would help with generalization and reduce overcooking since not all timesteps need to have the same emphasis.

Contributor guide

No contributing guide indexed for this repository

Research direction

The issue names no files or tests; start by locating the training timestep selection and the existing piecewise learning-rate scheduler. Compare their interfaces and define how a per-step timestep schedule should work, with completion demonstrated by configurable timestep choices across training steps without relying on one global range.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.