Issues with the MS-Swift Pipeline Layout Feature When Used with MTP

Checklist / 检查清单

• I have searched existing issues, and this is a new bug report. / 我已经搜索过现有的 issues，确认这是一个新的 bug report。

Bug Description / Bug 描述

Swift’s Pipeline Layout is very similar to that of Megatron-LM/Bridge, but the following issues arise when MTP is used:

1. Unlike Swift, Megatron requires you to enter as many “m”s in the pipeline layout as there are MTP layers. However, Swift only requires a single “m” regardless of the number of layers.
2. In Swift, a VP stage containing MTP layers must include a transformer layer; otherwise (e.g. E(|tt)*30,|mL for Qwen 3.5 397B-A17B), the following error will occur:

[rank237]: Traceback (most recent call last):
[rank237]:   File "/opt/venv-mbridge/lib/python3.12/site-packages/swift/cli/_megatron/sft.py", line 7, in <module>
[rank237]:     megatron_sft_main()
[rank237]:   File "/opt/venv-mbridge/lib/python3.12/site-packages/swift/megatron/pipelines/train/sft.py", line 97, in megatron_sft_main
[rank237]:     return MegatronSft(args).main()
[rank237]:            ^^^^^^^^^^^^^^^^^^^^^^^^
[rank237]:   File "/opt/venv-mbridge/lib/python3.12/site-packages/swift/pipelines/base.py", line 52, in main
[rank237]:     result = self.run()
[rank237]:              ^^^^^^^^^^
[rank237]:   File "/opt/venv-mbridge/lib/python3.12/site-packages/swift/megatron/pipelines/train/sft.py", line 70, in run
[rank237]:     trainer = self.prepare_trainer()
[rank237]:               ^^^^^^^^^^^^^^^^^^^^^^
[rank237]:   File "/opt/venv-mbridge/lib/python3.12/site-packages/swift/megatron/pipelines/train/sft.py", line 37, in prepare_trainer
[rank237]:     return MegatronTrainer(self.args, self.template)
[rank237]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank237]:   File "/opt/venv-mbridge/lib/python3.12/site-packages/swift/megatron/trainers/base.py", line 69, in __init__
[rank237]:     self.prepare_model()
[rank237]:   File "/opt/venv-mbridge/lib/python3.12/site-packages/swift/megatron/trainers/base.py", line 186, in prepare_model
[rank237]:     self.unwrapped_models = get_mcore_model(args, self.template.config)
[rank237]:                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank237]:   File "/opt/venv-mbridge/lib/python3.12/site-packages/swift/megatron/model/utils.py", line 85, in get_mcore_model
[rank237]:     models = _get_mcore_model(config)
[rank237]:              ^^^^^^^^^^^^^^^^^^^^^^^^
[rank237]:   File "/opt/venv-mbridge/lib/python3.12/site-packages/mcore_bridge/model/register.py", line 230, in get_mcore_model
[rank237]:     model = loader.build_model(pre_process, post_process, vp_stage=i)
[rank237]:             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank237]:   File "/opt/venv-mbridge/lib/python3.12/site-packages/mcore_bridge/model/gpts/qwen3_next_gdn.py", line 139, in build_model
[rank237]:     model = super().build_model(pre_process, post_process, vp_stage)
[rank237]:             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank237]:   File "/opt/venv-mbridge/lib/python3.12/site-packages/mcore_bridge/model/register.py", line 193, in build_model
[rank237]:     mtp_block_spec = self.get_mtp_block_spec(transformer_layer_spec, vp_stage=vp_stage)
[rank237]:                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank237]:   File "/opt/venv-mbridge/lib/python3.12/site-packages/mcore_bridge/model/register.py", line 139, in get_mtp_block_spec
[rank237]:     mtp_block_spec = get_gpt_mtp_block_spec(
[rank237]:                      ^^^^^^^^^^^^^^^^^^^^^^^
[rank237]:   File "/opt/venv-mbridge/lib/python3.12/site-packages/megatron/core/models/gpt/gpt_layer_specs.py", line 742, in get_gpt_mtp_block_spec
[rank237]:     return get_gpt_mtp_block_spec_for_backend(
[rank237]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank237]:   File "/opt/venv-mbridge/lib/python3.12/site-packages/megatron/core/models/gpt/gpt_layer_specs.py", line 761, in get_gpt_mtp_block_spec_for_backend
[rank237]:     transformer_layer_spec = spec.layer_specs[-1]
[rank237]:                              ~~~~~~~~~~~~~~~~^^^^
[rank237]: IndexError: list index out of range

Are these features or bugs?

How to Reproduce / 如何复现

• Model: Qwen 3.5 397B-A17B
• --pipeline_model_parallel_layout 'E(|tt)*30,|mL'

Additional Information / 补充信息

N/A

Reproduce with Qwen 3.5 397B-A17B and --pipeline_model_parallel_layout 'E(|tt)*30,|mL'. Start at swift/megatron/model/utils.py, then inspect the reported mcore_bridge/model/register.py and Megatron gpt_layer_specs.py MTP path to compare layout handling with the traceback. Done means the layout behaves consistently with the intended MTP semantics and no empty transformer specification causes the reported error.