Flux Lora training seems not to converge with big dataset(140 images)

I use one NVIDIA L40S(48GB VRAM) to train a Lora for Flux, and here is my training script:
`./sd-scripts/flux_train_network.py --pretrained_model_name_or_path ./model/flux1-dev.safetensors --clip_l ./model/clip_l.safetensors --t5xxl ./model/t5xxl_fp16.safetensors --ae ./model/ae.safetensors --cache_latents_to_disk --save_model_as safetensors --sdpa --persistent_data_loader_workers --max_data_loader_n_workers 2 --gradient_checkpointing --mixed_precision bf16 --full_bf16 --save_precision bf16 --network_module networks.lora_flux --network_dim 64 --network_alpha 32 --learning_rate 1 --lr_scheduler cosine_with_restarts --lr_scheduler_num_cycles 1 --optimizer_type prodigy --network_train_unet_only --cache_text_encoder_outputs --cache_text_encoder_outputs_to_disk --highvram --max_train_steps 3000 --save_every_n_steps 500 --dataset_config ./dataset.toml --output_dir ./lora_weight --output_name flux-lora-demo --timestep_sampling sigmoid --model_prediction_type raw --guidance_scale 1.0 --loss_type l2 --t5xxl_max_token_length 512 --min_snr_gamma 5 --sample_every_n_steps 500 --sample_prompts ./sample_prompt.toml --sample_sampler euler_a --logging_dir ./logs --log_with all --log_tracker_name flux_lora_komoru --wandb_api_key xxxxxxxxxxxxxxxx`

And my training dataset includes 140 images with below `dataset.toml` configuration:
```
[general]
enable_bucket = true
caption_extension = '.txt'
keep_tokens = 0

# DreamBooth caption based character datasets
[[datasets]]
resolution = 1024
min_bucket_reso = 640
max_bucket_reso = 1536
bucket_reso_steps = 32
batch_size = 4

[[datasets.subsets]]
image_dir = './dataset'
```
The final loss/average rate is around 0.38xxxx which is much higher than the nearly same configuration(less batch_size and less steps) with a small dataset(12 images, 0.08xxx loss/average rate).

Any advice for my training configurations? Thanks.

PS: I am not quite sure that `min_snr_gamma=5` would work well for the Flux training ,but it seemed to be a little bit improvement for the convergence.

Start with flux_train_network.py and the supplied dataset.toml, then compare the 140-image run with the 12-image configuration and inspect the logged loss. The issue does not define a code change or a reproducible expected result, so a newcomer would need to establish whether this is expected training behavior before a fix can be considered.

Written by the indexing model from the issue text.