modelscope / modelscope/ms-swift

[Bug]: Megatron arg mtp_shared_weights is defined but never consumed (no effect)

Open
#9,936 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
15.7k
Forks
1.7k
Avg merge
1d 16h
Merged PRs (30d)
136

Description

[Bug]: Megatron arg mtp_shared_weights is defined but never consumed (no effect)

Describe the bug

mtp_shared_weights is declared as a Megatron training argument but is never read anywhere in the codebase, so setting --mtp_shared_weights true has no effect. Users expecting it to make the N MTP layers share one physical layer (a single repeated MTP layer, as some inference stacks deploy) will silently get N independent layers instead.

Verification

In both 4.4.2 and the latest 4.5.2, the only occurrence of the symbol in the whole package is its definition:

swift/megatron/arguments/megatron_args.py:    mtp_shared_weights: bool = False
# reproduce
pip download ms-swift==4.5.2 --no-deps --no-binary :all: -d /tmp/sw
cd /tmp/sw && tar xzf ms_swift-4.5.2.tar.gz
grep -rn "mtp_shared_weights" ms_swift-4.5.2/swift
# -> only the single definition line; no consumer, no plumbing into the mcore MTP block

There is no code path passing this flag into get_gpt_mtp_block_spec / the Megatron MultiTokenPredictionBlock, and no branch that builds a single shared layer when it is True.

Expected behavior

Either:

  1. Wire mtp_shared_weights through to the mcore MTP block so a single physical MTP layer is built and reused across depths (mcore ≥ 0.19 exposes mtp_use_repeated_layer in TransformerConfig for exactly this), or
  2. Remove the dead argument to avoid the silent no-op.
Context / motivation

Speculative-decoding deployment for MTP often runs layer0 autoregressively K times (one physical draft layer, applied K times). Training with a single repeated MTP layer matches that deployment structure exactly, whereas N independent layers do not — the extra layers are never executed by the serving engine. A working mtp_shared_weights would let users train the deployment-aligned single-layer drafter directly.

Environment
  • ms-swift 4.4.2 and 4.5.2
  • megatron backend (mcore-bridge)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with swift/megatron/arguments/megatron_args.py and trace the Megatron path around get_gpt_mtp_block_spec and MultiTokenPredictionBlock. Verify how TransformerConfig is built and whether mcore exposes mtp_use_repeated_layer. Done means the flag has an effective consumer that creates shared MTP weights, or the unused argument is removed, with the grep check showing no silent definition-only behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.