If GGPO is NOT used for flux lora training, all "avg_key_norm" are NaN starting from step0 in Tensorboard

Tried different optimizers, lower/higher lr, rank/alpha combinations, datasets, base models. If i do not enable GGPO for Flux lora training the average key norms are always NaN

Start by reproducing Flux LoRA training with GGPO disabled and inspect the TensorBoard avg_key_norm values from step 0. Compare the training behavior with GGPO enabled while narrowing down where the NaN values first appear; done means avg_key_norm remains finite without GGPO.

Written by the indexing model from the issue text.