kohya-ss / kohya-ss/sd-scripts
TOML questions - validation loss and DoRA
- Dominant language
- Python
- Stars
- 7.2k
- Forks
- 1.2k
- Avg merge
- 11m
- Merged PRs (30d)
- 2
Description
**1. Validation Dataset Setup (`sd3` Branch Method):**
* To set up a single folder of validation images with the https://github.com/kohya-ss/sd-scripts/pull/1864 updates, is configuring a `[[datasets]]` block with `validation_split = 1.0` the correct approach? (I saw mentions of an older `is_validation = true` flag elsewhere, so wanted to confirm the intended method for this branch/feature).
* Should the validation dataset use the same `batch_size` as the main training dataset?
* I'm using the sd3 branch because of the validation loss, does it still work fine to train SDXL?
* Do captions have any effect or purpose for images within the validation dataset block when calculating validation loss?
Here's my current `dataset.toml` structure:
```toml
# --- Training Dataset Definition ---
[[datasets]]
batch_size = 4
resolution = [1024, 1024]
enable_bucket = true
min_bucket_reso = 512
max_bucket_reso = 2048
bucket_reso_steps = 64
bucket_no_upscale = true
[[datasets.subsets]]
image_dir = "path"
caption_extension = ".txt"
num_repeats = 10
shuffle_caption = true
keep_tokens = 1
flip_aug = true
random_crop = false
# --- Validation Dataset Definition ---
[[datasets]]
validation_split = 1.0
batch_size = 4
resolution = [1024, 1024]
enable_bucket = true
min_bucket_reso = 512
max_bucket_reso = 2048
bucket_reso_steps = 64
bucket_no_upscale = true
[[datasets.subsets]]
image_dir = "val_path"
caption_extension = ".txt"
```
**2. Correct DoRA Implementation:**
* I've seen examples using LyCORIS (`network_module="lycoris.kohya", network_args=["algo=dora", ...]`) but encountered a `KeyError: 'dora'` when trying that approach after installing `lycoris-lora`.
* Is using the standard LoRA module with the `use_dora=True` argument, like below, the correct and currently intended way to enable DoRA within the `sd3` branch?
```toml
# From config.toml
network_module = "networks.lora"
network_args = ["use_dora=True"]
# (other network_dim, network_alpha settings...)
```
Thanks for any clarification!
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reviewing PR #1864 and the sd3 branch's dataset configuration and networks.lora entry points, using the posted dataset.toml and config.toml examples as scope. Confirm the supported validation and DoRA settings for SDXL, captions, and batch size, then document the answers and any required configuration changes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100