Question about SLat flow fine-tuning

Hi, thank you for open-sourcing your great work.

I am trying to fine-tune the SLAT flow model with LoRA + ModulatedSLATMultiViewCond on a ~10k ProObjaverse subset, due to limited GPU resources. I would like to ask whether my training setup and observed loss behavior are expected.

I am running on 2 A6000 GPUs and Ubuntu22.04-cu124.

As shown in the below loss curve, the training loss decreases only slightly and appears to converge quite early. Around step ~11.7k, I also observed NaN in the loss.

I would really appreciate if you could answer few questions:
1. Is early loss convergence expected when fine-tuning only LoRA + vggt_cond for SLAT flow?
2. What should the expected SLAT flow training loss curve look like?
3. What batch size did you use when fine-tuning SLAT flow?

**Training setup:**
```
Dataset: ~10k ProObjaverse subset
num_views: 6
batch_size: 4
num_workers: 1
accum_batches: 4
Precision: 16-mixed
Learning rate: 1e-4
swa_lrs: 1e-2
training step: ~12k steps
```

**The training initialization shows:**

```
W0428 22:23:42.806000 2159815 site-packages/torch/distributed/run.py:792]
W0428 22:23:42.806000 2159815 site-packages/torch/distributed/run.py:792] *****************************************
W0428 22:23:42.806000 2159815 site-packages/torch/distributed/run.py:792] Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.
W0428 22:23:42.806000 2159815 site-packages/torch/distributed/run.py:792] *****************************************
[SPARSE] Backend: spconv, Attention: flash_attn[SPARSE] Backend: spconv, Attention: flash_attn

[SPARSE][CONV] spconv algo: native[SPARSE][CONV] spconv algo: native

[SPARSE] Conv backend: flex_gemm; Attention backend: flash_attn
[SPARSE] Conv backend: flex_gemm; Attention backend: flash_attn
[ATTENTION] Using backend: flash_attn
[TarDataset] 10841 samples found.
[rank 0] -------------- Build_models start --------------
[rank 0] Loading TrellisImageTo3DPipeline from microsoft/TRELLIS-image-large...
[ATTENTION] Using backend: flash_attn
[TarDataset] 10841 samples found.
[rank 1] -------------- Build_models start --------------
[rank 1] Loading TrellisImageTo3DPipeline from microsoft/TRELLIS-image-large...
trainable params: 20,447,232 || all params: 620,879,752 || trainable%: 3.2933
trainable params: 20,447,232 || all params: 620,879,752 || trainable%: 3.2933
Using 16bit Automatic Mixed Precision (AMP)
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
ed model size in MB will not be accurate. Using 32 bits instead.
┏━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━┓
┃ ┃ Name ┃ Type ┃ Params ┃ Mode ┃ FLOPs ┃
┡━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━┩
│ 0 │ flow_model │ PeftModel │ 620 M │ train │ 0 │
│ 1 │ vggt_cond │ ModulatedSLATMultiViewCond │ 25.2 M │ train │ 0 │
│ 2 │ image_cond_model_dinov2 │ DinoVisionTransformer │ 304 M │ eval │ 0 │
│ 3 │ vggt_model │ VGGT │ 909 M │ eval │ 0 │
└───┴─────────────────────────┴────────────────────────────┴────────┴───────┴───────┘
Trainable params: 45.6 M
Non-trainable params: 1.8 B
Total params: 1.9 B
Total estimated model params size (MB): 7.4 K
Modules in train mode: 1229
Modules in eval mode: 2333
Total FLOPs: 0
```

Thank you very much for your time and for sharing this work!

No source files, tests, or entry points are named. Start by reviewing the fine-tuning configuration for SLAT flow, LoRA, and ModulatedSLATMultiViewCond, then compare the reported loss behavior and NaN occurrence with the documented or reproducible training setup; done means resolving whether the configuration and observed behavior are expected.

Written by the indexing model from the issue text.