kohya-ss / kohya-ss/sd-scripts
About Regularization Image
- Dominant language
- Python
- Stars
- 7.2k
- Forks
- 1.2k
- Avg merge
- 11m
- Merged PRs (30d)
- 2
Description
There are two things I'm not very clear about reg image in DreamBooth and LoRA training.
1. The current implementation will discard parts of reg image if it exceeds the amount of train image (after repeat). Is there a specific reason for this limitation? Since I check the original DreamBooth paper, they used 200x amout of reg images. Theoretically you should use more image to better approximate the prior class distribution.
2. If you provided caption files for all images (including tran and reg) then the caption will be used directly for text conditioning vector generation right? I read through the code but am not sure if I missed something. If that's the case, there is another problem when you use folder name but not text caption. E.g. if you used "10_aaa 1girl" for both train and reg images, then "aaa 1girl" will be used for both train and reg subset. However the correct way to text conditioning should be "aaa 1girl" for train images and "1girl" for reg images.
I'm not sure if any of these two problems really exists since I had a hard time read through the code, especially for the dataset pipeline part. I cound miss some important code. If that's the case please let me know :)
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by tracing the dataset pipeline for DreamBooth and LoRA training, focusing on regularization-image balancing and caption or folder-name prompt handling. Verify whether the two suspected behaviors occur, then document the findings or define the required change and tests; the issue provides no specific file or test path.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100