Integrate NeMo Automodel Qwen3.5-35B-A3B recipe with MFSDP v2
Open
- Dominant language
- Python
- Stars
- 17.9k
- Forks
- 4.5k
- Avg merge
- 4d 6h
- Merged PRs (30d)
- 271
Description
## Summary
The NeMo Automodel Qwen3.5-35B-A3B VLM fine-tuning recipe currently uses PyTorch FSDP2 and needs a validated Megatron Core path using experimental MFSDP v2.
Reference recipe:
- https://github.com/NVIDIA-NeMo/Automodel/blob/main/examples/vlm_finetune/qwen3_5_moe/qwen3_5_35b.yaml
## Reproduction / starting point
The reference recipe runs with:
```bash
automodel examples/vlm_finetune/qwen3_5_moe/qwen3_5_35b.yaml --nproc-per-node 8
```
Use the same model and the closest equivalent data, parallelism, and checkpoint settings through the MFSDP v2 path.
Contributor guide
Assessment
This issue has not been assessed yet.