[QUESTION] A few questions on expert data parallel

**Your question**

Hey Folks,
Please help me understand the observation.
I'm fine-tuning(SFT) Qwen3.5-122B with 32GPUs(H200).
As the length of longest sequence in training data is ~42K, I set expert-model-parallel-size=16(EMP) and CP=16. PP=1, TP=1 and DP=2. Since 16 GPUs form a DP group, I set EMP=16 as I was under the impression that it can't exceed DP ranks. However, the job runs without issues even if I set EMP=32.

* How exactly training works when expert data parallel and data parallel are not equal, specifically data flow? For ex. DP=16 and EDP=1 in a 32 GPU setup with TP=2 and EMP=32; each DP group process a part of global batch size, how does experts process the data when EDP=1?

* Although possible, is it recommended to have a state where EDP != DP?

* While PP and TP operates within DP group, why EMP is allowed to exceed DP ranks?

* When distributed optimizer flag is set, how would the optimizer state of experts distributed when they sharded across GPUs more than DP ranks?

Please let me know if I'm missing something or correct my understanding.

Tag @NVIDIA/mcore-oncall
to get oncall's attention to this issue.

The issue names no files, tests, or entry points. Start by tracing the reported 32-GPU configuration and the expert data-parallel, data-parallel, tensor-parallel, and distributed-optimizer concepts. Done means providing a clear explanation of the data flow, EDP-versus-DP recommendation, EMP sizing, and expert optimizer-state sharding.

Written by the indexing model from the issue text.