kohya-ss / kohya-ss/sd-scripts
Strange part in calculating the number of steps.
- Dominant language
- Python
- Stars
- 7.2k
- Forks
- 1.2k
- Avg merge
- 11m
- Merged PRs (30d)
- 2
Description
I found some strange part in flux_train.py to calculate the number of steps.
The number of steps is calculated in this code.
```
args.max_train_steps = args.max_train_epochs * math.ceil(
# len(train_dataloader) / accelerator.num_processes / args.gradient_accumulation_steps
#)
train_dataloader = torch.utils.data.DataLoader(
train_dataset_group,
batch_size=1,
shuffle=True,
collate_fn=collator,
num_workers=n_workers,
persistent_workers=args.persistent_data_loader_workers,
)
```
So, I checked the len(train_dataloader), and I found that batch_size of dataloader is set to 1.
It is strange that no matter what batch size is used during fine-tuning, it is always set to 1
And, if you set this part to the fine-tuning batch size, for example, 4, len(train_dataloader) becomes small and the number of steps calculation becomes strange.
Isn't it correct to change it like the code below?
For example, with the code above, when the number of datasets is 400, the epoch is 100, gradident_accumulation=1, and the batch size is 4, the number of steps is 2500, which is strange, but if you use the modified code below, it becomes 10000 steps.
Isn't it correct that it's 10000 steps?
```
num_examples = train_dataset_group.num_train_images
effective_batch_size = train_batch_size*accelerator.num_processes*args.gradient_accumulation_steps
args.max_train_steps = math.ceil((num_examples*args.max_train_epochs)/effective_batch_size)
```
Contributor guide
No contributing guide indexed for this repository
Research direction
Read flux_train.py around the commented max_train_steps calculation and the DataLoader definition, then trace how the configured training batch size, num_train_images, process count, and gradient accumulation are used. Run a fine-tuning configuration matching the issue's 400-image, 100-epoch, batch-size-4 example and verify that the reported step count matches the project's intended training semantics.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100