Megatron-LM Monthly Activity Report — August 2026

# Megatron-LM Monthly Activity Report — August 2026

> Period: 2026-08-01 through 2026-08-31 (Asia/Shanghai); source: [`NVIDIA/Megatron-LM`](https://github.com/NVIDIA/Megatron-LM).

## Overview

The delivered portfolio emphasizes Megatron-FSDP v2, GTP, dynamic inference, MoE execution, sparse attention, hybrid architectures, THD sequence packing, multimodal workloads, optimizer efficiency, and RL rollout reliability. Ongoing work extends these directions through finer-grained FSDP overlap, DeepSeek-V4 attention, disaggregated serving, new MoE kernels, hybrid sequence models, Muon variants, multimodal execution, and reproducibility infrastructure.

- PRs opened: 743; merged during the period: 397; closed without merge: 221; still open at period end: 322.
- Excluded from the narrative: test-only 41, CI-only 44, test/CI-only 0, format-only 0.

## Delivered

### Megatron-FSDP v2: Sharding, Optimizer, and Distributed Checkpoints

Megatron-FSDP v2 gained a dedicated optimizer, multiple data-parallel sharding strategies, independent expert sharding, distributed checkpoint support, and tighter gradient and storage lifecycle handling.

- No-shard and ZeRO-1/2/3 layouts now cover replicated and progressively sharded parameters, gradients, and optimizer states, with a separate policy available for expert parameters.
- Distributed checkpoint handling covers uneven DTensor shards, optimizer-state materialization, compute-weight synchronization, and fused MLA and MTP parameter mappings.
- Expert-gradient scaling, stream-owned main-gradient buffers, DTensor-aware clipping, and stable post-step weight synchronization address correctness across expert and dense meshes.
- Persistent communication-buffer pools and weak-reference ownership improve combined-1F1B capacity management and allow sharded-model CUDA storage to be released.

Related PRs: [#5865](https://github.com/NVIDIA/Megatron-LM/pull/5865), [#5956](https://github.com/NVIDIA/Megatron-LM/pull/5956), [#6024](https://github.com/NVIDIA/Megatron-LM/pull/6024), [#6137](https://github.com/NVIDIA/Megatron-LM/pull/6137), [#6187](https://github.com/NVIDIA/Megatron-LM/pull/6187), [#6230](https://github.com/NVIDIA/Megatron-LM/pull/6230), [#6271](https://github.com/NVIDIA/Megatron-LM/pull/6271), [#6316](https://github.com/NVIDIA/Megatron-LM/pull/6316), [#6336](https://github.com/NVIDIA/Megatron-LM/pull/6336), [#6489](https://github.com/NVIDIA/Megatron-LM/pull/6489), [#6798](https://github.com/NVIDIA/Megatron-LM/pull/6798)

### GTP: CUDA-Graph Communication, MTP Weight Reuse, and Checkpoint Resumption

Generalized tensor parallelism was hardened for CUDA-graph replay, repeated MTP parameter use, mixed-precision reduction, evaluation, Muon integration, and heterogeneous checkpoint resumption.

- Per-graph communication ownership restored reduce-scatter overlap while fixing weight-gradient buffer lifetime and stale-gradient races.
- Repeated MTP and recomputation consumers now retain distinct communication tickets or gather buffers, preventing missing gradients and weight corruption.
- Parameter-readiness callbacks and main-gradient dtype propagation prevent stale weight gathers and BF16 rounding before FP32 reduction.
- GTP-aware evaluation, heterogeneous MIMO checkpoint groups, and MXFP8-aware Muon gathering improve correctness across rematerialized and low-precision configurations.

Related PRs: [#6060](https://github.com/NVIDIA/Megatron-LM/pull/6060), [#6242](https://github.com/NVIDIA/Megatron-LM/pull/6242), [#6388](https://github.com/NVIDIA/Megatron-LM/pull/6388), [#6407](https://github.com/NVIDIA/Megatron-LM/pull/6407), [#6624](https://github.com/NVIDIA/Megatron-LM/pull/6624), [#6717](https://github.com/NVIDIA/Megatron-LM/pull/6717), [#6743](https://github.com/NVIDIA/Megatron-LM/pull/6743), [#6754](https://github.com/NVIDIA/Megatron-LM/pull/6754)

### Disaggregated Dynamic Inference: KV/SSM Handoff, Prefix Caching, and ReFIT

Dynamic inference added disaggregated KV and SSM state transfer, stronger prefix-cache lifecycle handling, asynchronous scheduling, progressive tool calls, and model-refit support.

- Asynchronous prefill-to-decode handoff transfers attention KV state and hybrid SSM state over NIXL or NCCL while protecting cache blocks and avoiding prompt replay on decode workers.
- Prefix-cache admission, KV-write isolation, GDP state reuse, and Mamba edge-case alignment improve behavior under saturation and chunked prefill.
- NCCL M-to-N refitting and in-place MXFP8 updates support topology changes while preserving CUDA-graph-captured storage; eager communicator initialization prevents idle-rank hangs.
- The serving stack also gained an inference metadata ledger, streamed tool-call deltas, default asynchronous scheduling, subscriber rendezvous, and server-controlled chat-template handling.

Related PRs: [#5995](https://github.com/NVIDIA/Megatron-LM/pull/5995), [#6125](https://github.com/NVIDIA/Megatron-LM/pull/6125), [#6221](https://github.com/NVIDIA/Megatron-LM/pull/6221), [#6222](https://github.com/NVIDIA/Megatron-LM/pull/6222), [#6226](https://github.com/NVIDIA/Megatron-LM/pull/6226), [#6584](https://github.com/NVIDIA/Megatron-LM/pull/6584), [#6711](https://github.com/NVIDIA/Megatron-LM/pull/6711), [#6712](https://github.com/NVIDIA/Megatron-LM/pull/6712), [#6779](https://github.com/NVIDIA/Megatron-LM/pull/6779), [#6780](https://github.com/NVIDIA/Megatron-LM/pull/6780), [#6818](https://github.com/NVIDIA/Megatron-LM/pull/6818), [#6826](https://github.com/NVIDIA/Megatron-LM/pull/6826), [#6851](https://github.com/NVIDIA/Megatron-LM/pull/6851), [#6895](https://github.com/NVIDIA/Megatron-LM/pull/6895), [#6930](https://github.com/NVIDIA/Megatron-LM/pull/6930), [#6955](https://github.com/NVIDIA/Megatron-LM/pull/6955)

### MoE Batch Invariance, CUDA Graphs, and MXFP8 Decode

MoE execution expanded across batch-invariant routing, graph-compatible dispatch, paged activation storage, MXFP8 communication and refitting, and device-initiated expert kernels.

- Batch-invariant execution now spans Nemotron routing, dispatch, fused experts, SSM kernels, and dynamic inference; the fused-MoE path reported roughly 25,100 tokens/s versus roughly 4,070 previously and roughly 25,400 without invariance.
- Paged stash supports whole-MoE CUDA graphs, while shared vLLM grouped-GEMM buffers reduced reported graph memory from more than 20 GB to approximately 3 GB.
- The NCCL dispatcher supports dynamic eager capacity and static graph capacity with overflow recovery, plus BF16 or MXFP8 token movement and an optional FP32 NVLS combine.
- Device-initiated GroupedLinear removes host token-count synchronization, and FlashInfer routed MXFP8 decode reported 35.2% lower time per token and 1.54× end-to-end throughput.

Related PRs: [#4871](https://github.com/NVIDIA/Megatron-LM/pull/4871), [#6022](https://github.com/NVIDIA/Megatron-LM/pull/6022), [#6229](https://github.com/NVIDIA/Megatron-LM/pull/6229), [#6432](https://github.com/NVIDIA/Megatron-LM/pull/6432), [#6521](https://github.com/NVIDIA/Megatron-LM/pull/6521), [#6524](https://github.com/NVIDIA/Megatron-LM/pull/6524), [#6588](https://github.com/NVIDIA/Megatron-LM/pull/6588), [#6618](https://github.com/NVIDIA/Megatron-LM/pull/6618), [#6806](https://github.com/NVIDIA/Megatron-LM/pull/6806), [#6847](https://github.com/NVIDIA/Megatron-LM/pull/6847), [#6864](https://github.com/NVIDIA/Megatron-LM/pull/6864), [#6870](https://github.com/NVIDIA/Megatron-LM/pull/6870)

### DeepSeek-V4 CSA/DSA: THD Kernels, Teacher Loss, and Fused RoPE

Compressed and sparse attention gained a native SBHD reference, corrected teacher-loss semantics, fused THD compression and RoPE paths, and more efficient sparse-indexer backward execution.

- Teacher distributions now use the complete CSA denominator and full attention LSE while excluding padded THD entries and compacting shared attention indices.
- The unfused BF16 SBHD implementation covers compression, indexing, sparse and dense attention, sliding windows, attention sinks, and teacher-loss normalization.
- Fused THD pooling reduces the documented eager launch sequence to one compute kernel per direction, while fused MLA/DSA RoPE reduced a targeted long-sequence region from 85 kernels to 2.
- Sparse-indexer padding changes reduced the reported kernel from about 12.824 ms to 1.273 ms, with median iteration latency down 9.9% and throughput up 11.0% on the stated proxy.

Related PRs: [#5960](https://github.com/NVIDIA/Megatron-LM/pull/5960), [#5984](https://github.com/NVIDIA/Megatron-LM/pull/5984), [#6166](https://github.com/NVIDIA/Megatron-LM/pull/6166), [#6279](https://github.com/NVIDIA/Megatron-LM/pull/6279), [#6343](https://github.com/NVIDIA/Megatron-LM/pull/6343), [#6349](https://github.com/NVIDIA/Megatron-LM/pull/6349), [#6400](https://github.com/NVIDIA/Megatron-LM/pull/6400), [#6842](https://github.com/NVIDIA/Megatron-LM/pull/6842), [#6846](https://github.com/NVIDIA/Megatron-LM/pull/6846)

### Hybrid Models and CUDA-Graph Recomputation: mHC, GDP/GDN, and KDA–MLA

Hybrid-model support grew through manifold hyper-connections, Gated Delta Product and Gated DeltaNet execution, Ling-V3 Tiny KDA–MLA training, and broader CUDA-graph and recomputation coverage.

- mHC gained core and fused implementations, selective recomputation, hybrid-model integration, and FP32 cuTile mapping; the reported residual-mapping error improved from 1.89e-3 to 1.09e-5.
- Gated Delta Product now spans training, static and dynamic inference, TP/CP, packed sequences, checkpointing, optimizer integration, activation controls, CuTe kernels, CUDA graphs, and chunkwise context parallelism.
- Ling-V3 Tiny support covers its 7.9B-parameter sparse-MoE KDA–MLA architecture, while Gated DeltaNet gained dynamic inference and recurrent-mixer execution.
- Graph and recomputation fixes cover aliased checkpoint tensors, expert-overlap layer segments, nested checkpoint parameters, deterministic stream assignment, mHC capture behavior, and partial hybrid MTP capture.

Related PRs: [#4497](https://github.com/NVIDIA/Megatron-LM/pull/4497), [#4531](https://github.com/NVIDIA/Megatron-LM/pull/4531), [#5841](https://github.com/NVIDIA/Megatron-LM/pull/5841), [#5869](https://github.com/NVIDIA/Megatron-LM/pull/5869), [#6074](https://github.com/NVIDIA/Megatron-LM/pull/6074), [#6172](https://github.com/NVIDIA/Megatron-LM/pull/6172), [#6311](https://github.com/NVIDIA/Megatron-LM/pull/6311), [#6373](https://github.com/NVIDIA/Megatron-LM/pull/6373), [#6401](https://github.com/NVIDIA/Megatron-LM/pull/6401), [#6429](https://github.com/NVIDIA/Megatron-LM/pull/6429), [#6499](https://github.com/NVIDIA/Megatron-LM/pull/6499), [#6553](https://github.com/NVIDIA/Megatron-LM/pull/6553), [#6556](https://github.com/NVIDIA/Megatron-LM/pull/6556), [#6583](https://github.com/NVIDIA/Megatron-LM/pull/6583), [#6595](https://github.com/NVIDIA/Megatron-LM/pull/6595), [#6598](https://github.com/NVIDIA/Megatron-LM/pull/6598), [#6643](https://github.com/NVIDIA/Megatron-LM/pull/6643), [#6661](https://github.com/NVIDIA/Megatron-LM/pull/6661), [#6704](https://github.com/NVIDIA/Megatron-LM/pull/6704), [#6815](https://github.com/NVIDIA/Megatron-LM/pull/6815)

### THD Sequence Packing and Hybrid Context-Parallel Layouts

Sequence-packed training gained reusable scheduling, THD batch distribution, MTP rolling, data-parallel balancing, and per-layer conversion between contiguous and zigzag context-parallel layouts.

- Core and data-parallel schedulers now pack, reroute, broadcast, and fetch variable microbatches with PackedSeqParams and physical padding metadata.
- Packed context-parallel MTP prefetches successor rows once per microbatch and reuses them across prediction depths.
- Data-parallel rerouting avoided fully connected NCCL P2P allocation and reduced reported memory by 1,152–1,154 MiB per GPU in a four-rank test, at the cost of greater communication volume.
- Qwen3.5 hybrid layouts reported a 1.29× overall proxy speedup, while MagiAttention adds attention-area-balanced context parallelism for variable-length packed Qwen3 MoE SFT.

Related PRs: [#5903](https://github.com/NVIDIA/Megatron-LM/pull/5903), [#5904](https://github.com/NVIDIA/Megatron-LM/pull/5904), [#6246](https://github.com/NVIDIA/Megatron-LM/pull/6246), [#6378](https://github.com/NVIDIA/Megatron-LM/pull/6378), [#6387](https://github.com/NVIDIA/Megatron-LM/pull/6387), [#6478](https://github.com/NVIDIA/Megatron-LM/pull/6478), [#6515](https://github.com/NVIDIA/Megatron-LM/pull/6515), [#6625](https://github.com/NVIDIA/Megatron-LM/pull/6625), [#6626](https://github.com/NVIDIA/Megatron-LM/pull/6626), [#6684](https://github.com/NVIDIA/Megatron-LM/pull/6684), [#6764](https://github.com/NVIDIA/Megatron-LM/pull/6764), [#6776](https://github.com/NVIDIA/Megatron-LM/pull/6776)

### Bagel and MIMO Multimodal Training with Image/Video Inference

Multimodal coverage expanded through Bagel training, dynamic image and video inference, MTP modality masking, heterogeneous MIMO parallelism, and per-modality precision control.

- Bagel support includes MCore and Hugging Face LLM wrappers, a ViT wrapper, diffusion components, data handling, checkpoint utilities, and training integration.
- Dynamic batching now accepts image-bearing requests, and inference APIs and OpenAI-compatible endpoints also accept video bytes, frame manifests, or preprocessed tensors.
- MTP masking excludes modality embeddings and cumulatively blocks prediction paths after image, audio, or other placeholders.
- MIMO gained dense and expert GTP topology, heterogeneous DDP overlap, batched pipeline communication, and independent precision policies for modality submodules.

Related PRs: [#3635](https://github.com/NVIDIA/Megatron-LM/pull/3635), [#6010](https://github.com/NVIDIA/Megatron-LM/pull/6010), [#6256](https://github.com/NVIDIA/Megatron-LM/pull/6256), [#6260](https://github.com/NVIDIA/Megatron-LM/pull/6260), [#6284](https://github.com/NVIDIA/Megatron-LM/pull/6284), [#6570](https://github.com/NVIDIA/Megatron-LM/pull/6570), [#6606](https://github.com/NVIDIA/Megatron-LM/pull/6606), [#6745](https://github.com/NVIDIA/Megatron-LM/pull/6745), [#6770](https://github.com/NVIDIA/Megatron-LM/pull/6770), [#6809](https://github.com/NVIDIA/Megatron-LM/pull/6809)

### Muon Shard Balancing and Chunked Optimizer Offload

Optimizer work reduced GPU residency and improved layer-wise parameter assignment through pinned CPU storage, bounded state chunks, shard-aware packing, and per-weight Muon execution choices.

- Chunked optimizer-state and master-weight offload replaces full-state GPU restoration with bounded chunks backed by pinned CPU storage.
- Re-pinning FP32-converted offloaded parameters preserves pinned-memory host-to-device transfers instead of falling back to pageable memory.
- Layer-wise bucket packing removed the documented 25% slot overhead, while compute-cost-aware Muon balancing improved a reported 1,024-GB200 workload from 2,080 to 1,707 ms per iteration.
- Explicit Muon ownership exclusions and automatic per-weight TP mode selection prevent unowned matrices and adapt Newton–Schulz execution to shape, precision, group size, and hardware costs.

Related PRs: [#4547](https://github.com/NVIDIA/Megatron-LM/pull/4547), [#5415](https://github.com/NVIDIA/Megatron-LM/pull/5415), [#6244](https://github.com/NVIDIA/Megatron-LM/pull/6244), [#6379](https://github.com/NVIDIA/Megatron-LM/pull/6379), [#6642](https://github.com/NVIDIA/Megatron-LM/pull/6642), [#6664](https://github.com/NVIDIA/Megatron-LM/pull/6664)

### RL Context Parallelism, Unified Rollouts, and Durable Recovery

Reinforcement-learning execution gained context-parallel packed sequences, shared prefill, a unified rollout pipeline, durable rollout storage, and recovery protections for infrastructure failures.

- Context parallelism now spans RL utilities, packed-sequence handling, Transformer Engine integration, and the training entry point, while shared prefill allows packed samples to reuse common prompt work.
- A unified pipeline centralizes rollout generation and scheduling across environments, with automatic lag tuning reducing manual configuration.
- Checksummed, fsync-backed rollout storage supports torn-write recovery, restoration, consumption tracking, and checkpoint-boundary compaction.
- Infrastructure-failure placeholders are excluded from GRPO rewards, and all-placeholder rollout groups are detected and regenerated.

Related PRs: [#4127](https://github.com/NVIDIA/Megatron-LM/pull/4127), [#4362](https://github.com/NVIDIA/Megatron-LM/pull/4362), [#5882](https://github.com/NVIDIA/Megatron-LM/pull/5882), [#5887](https://github.com/NVIDIA/Megatron-LM/pull/5887), [#6352](https://github.com/NVIDIA/Megatron-LM/pull/6352), [#6475](https://github.com/NVIDIA/Megatron-LM/pull/6475), [#6480](https://github.com/NVIDIA/Megatron-LM/pull/6480), [#6495](https://github.com/NVIDIA/Megatron-LM/pull/6495)

## In Progress

### Megatron-FSDP v2: 1F1B Overlap, Expert Sharding, and CPU Offload

Ongoing Megatron-FSDP v2 work is developing fine-grained 1F1B lifecycle control, optimizer checkpoints, MXFP8 primary weights, expert-aware shard planning, CPU offload, and more robust asynchronous checkpoint handling.

- VPP and combined-1F1B paths are being wired with explicit unshard, reshard, reduction, prefetch, no-sync, delayed weight-gradient, shared-context, UBR, and cross-stream lifetime controls.
- Optimizer checkpoints, GroupedTensor expert storage, composed EP/DP placements, MXFP8 payloads, reproducible re-quantization, and reusable shard plans extend low-precision and expert workflows.
- Routed-expert weights and chunked Adam moments, master weights, initialization, and checkpoint saves are being moved through pinned CPU storage to bound GPU residency.
- A reported hybrid outer-optimizer strategy reached 1,450 TFLOP/s/GPU versus 1,195 for uniform ZeRO-3, while an initialization-stream change reduced reported reserved and device memory by 20.47 GiB.

Related PRs: [#6197](https://github.com/NVIDIA/Megatron-LM/pull/6197), [#6215](https://github.com/NVIDIA/Megatron-LM/pull/6215), [#6228](https://github.com/NVIDIA/Megatron-LM/pull/6228), [#6484](https://github.com/NVIDIA/Megatron-LM/pull/6484), [#6485](https://github.com/NVIDIA/Megatron-LM/pull/6485), [#6486](https://github.com/NVIDIA/Megatron-LM/pull/6486), [#6494](https://github.com/NVIDIA/Megatron-LM/pull/6494), [#6544](https://github.com/NVIDIA/Megatron-LM/pull/6544), [#6594](https://github.com/NVIDIA/Megatron-LM/pull/6594), [#6597](https://github.com/NVIDIA/Megatron-LM/pull/6597), [#6665](https://github.com/NVIDIA/Megatron-LM/pull/6665), [#6666](https://github.com/NVIDIA/Megatron-LM/pull/6666), [#6694](https://github.com/NVIDIA/Megatron-LM/pull/6694), [#6773](https://github.com/NVIDIA/Megatron-LM/pull/6773), [#6785](https://github.com/NVIDIA/Megatron-LM/pull/6785), [#6885](https://github.com/NVIDIA/Megatron-LM/pull/6885), [#6948](https://github.com/NVIDIA/Megatron-LM/pull/6948), [#6971](https://github.com/NVIDIA/Megatron-LM/pull/6971)

### GTP: On-Demand Materialization, Process Groups, and Persistent CUDA-Graph Buffers

Generalized tensor parallelism is being extended as an end-to-end weight-sharding system with explicit process groups, asynchronous materialization, fixed-address graph storage, repeated-parameter replay, and ReFIT support.

- The proposed GTP stack spans training, optimization, checkpoints, inference, CUDA graphs, low-precision parameters, MIMO, and MoE integration.
- Explicit process-group collections are being required across language models and related MCore paths to prevent collectives from silently using the wrong parallel grid.
- Persistent arenas replace specialized gradient rings, size storage by domain and dtype, and protect reusable CUDA-graph generations with replay-completion events.
- Repeated parameter use records every gradient-ready notification, while rematerialized tied embeddings and inference shards are being repaired for reshard and ReFIT workflows.

Related PRs: [#6293](https://github.com/NVIDIA/Megatron-LM/pull/6293), [#6303](https://github.com/NVIDIA/Megatron-LM/pull/6303), [#6713](https://github.com/NVIDIA/Megatron-LM/pull/6713), [#6862](https://github.com/NVIDIA/Megatron-LM/pull/6862), [#6867](https://github.com/NVIDIA/Megatron-LM/pull/6867), [#6882](https://github.com/NVIDIA/Megatron-LM/pull/6882), [#6940](https://github.com/NVIDIA/Megatron-LM/pull/6940)

### DeepSeek-V4 CSA/DSA: Hybrid Attention, MLA Latent CP, and NVFP4

DeepSeek-V4 work is integrating native and fused compressed sparse attention, GQA-based DSA, per-layer context layouts, MLA latent context parallelism, tensor and sequence parallelism, and NVFP4 training.

- Hybrid-attention orchestration and layer construction connect CSA, window attention, grouped output projection, MTP identity, indexer loss, and validated compression-ratio semantics.
- The fused SBHD CSA path targets SM90-or-newer GPUs, while GQA-based DSA adds reference and Triton implementations with memory-oriented execution.
- Packed-THD layout metadata, per-layer conversions, dynamic-CP graph sharing, communicator reuse, and DSA state reuse across MTP steps target lower repeated layout and indexer work.
- MLA latent CP, shared K-RoPE gradients, TP-plus-CP execution, NVFP4 training, compilation caches, and pipeline prewarming are being developed; the reported prewarm test reduced startup through the first iteration from 866.0 to 370.8 seconds.

Related PRs: [#6206](https://github.com/NVIDIA/Megatron-LM/pull/6206), [#6233](https://github.com/NVIDIA/Megatron-LM/pull/6233), [#6397](https://github.com/NVIDIA/Megatron-LM/pull/6397), [#6402](https://github.com/NVIDIA/Megatron-LM/pull/6402), [#6404](https://github.com/NVIDIA/Megatron-LM/pull/6404), [#6405](https://github.com/NVIDIA/Megatron-LM/pull/6405), [#6426](https://github.com/NVIDIA/Megatron-LM/pull/6426), [#6472](https://github.com/NVIDIA/Megatron-LM/pull/6472), [#6479](https://github.com/NVIDIA/Megatron-LM/pull/6479), [#6699](https://github.com/NVIDIA/Megatron-LM/pull/6699), [#6801](https://github.com/NVIDIA/Megatron-LM/pull/6801), [#6829](https://github.com/NVIDIA/Megatron-LM/pull/6829), [#6839](https://github.com/NVIDIA/Megatron-LM/pull/6839), [#6841](https://github.com/NVIDIA/Megatron-LM/pull/6841), [#6848](https://github.com/NVIDIA/Megatron-LM/pull/6848), [#6850](https://github.com/NVIDIA/Megatron-LM/pull/6850), [#6869](https://github.com/NVIDIA/Megatron-LM/pull/6869), [#6874](https://github.com/NVIDIA/Megatron-LM/pull/6874), [#6887](https://github.com/NVIDIA/Megatron-LM/pull/6887)

### MoE Decode Kernels, SeqTopK Routing, and Batch-Invariant Reduction

Ongoing MoE work targets sequence-level routing, fused decode kernels, graph-safe expert-parallel initialization, new execution backends, batch-invariant reduction, load balancing, and reusable activation buffers.

- SeqTopK proposes a fixed sequence-level expert budget with a configurable per-token cap, while dense routing indices avoid reconstructing supported Flex-dispatch inputs.
- Decode optimizations fuse FC1 activation, router selection, Q/K RMSNorm, routing-table construction, and padding handling and add separate FC1/FC2 tiling; individual reported gains range from kernel-level reductions to double-digit end-to-end improvement.
- MOK and Sonic-MoE add experimental fused backends, while device-initiated GroupedTensor experts are being integrated with paged stash and reusable HybridEP buffers.
- Batch-invariant NVLS reduction, global-batch quantile balancing, SiTU-GLU, grouped-MLP synchronization, allocator hand-back, and DDP-preserved fused expert views address training and colocated RL execution.

Related PRs: [#6208](https://github.com/NVIDIA/Megatron-LM/pull/6208), [#6412](https://github.com/NVIDIA/Megatron-LM/pull/6412), [#6452](https://github.com/NVIDIA/Megatron-LM/pull/6452), [#6453](https://github.com/NVIDIA/Megatron-LM/pull/6453), [#6454](https://github.com/NVIDIA/Megatron-LM/pull/6454), [#6456](https://github.com/NVIDIA/Megatron-LM/pull/6456), [#6457](https://github.com/NVIDIA/Megatron-LM/pull/6457), [#6458](https://github.com/NVIDIA/Megatron-LM/pull/6458), [#6459](https://github.com/NVIDIA/Megatron-LM/pull/6459), [#6572](https://github.com/NVIDIA/Megatron-LM/pull/6572), [#6601](https://github.com/NVIDIA/Megatron-LM/pull/6601), [#6614](https://github.com/NVIDIA/Megatron-LM/pull/6614), [#6615](https://github.com/NVIDIA/Megatron-LM/pull/6615), [#6634](https://github.com/NVIDIA/Megatron-LM/pull/6634), [#6637](https://github.com/NVIDIA/Megatron-LM/pull/6637), [#6638](https://github.com/NVIDIA/Megatron-LM/pull/6638), [#6654](https://github.com/NVIDIA/Megatron-LM/pull/6654), [#6673](https://github.com/NVIDIA/Megatron-LM/pull/6673), [#6674](https://github.com/NVIDIA/Megatron-LM/pull/6674), [#6800](https://github.com/NVIDIA/Megatron-LM/pull/6800), [#6828](https://github.com/NVIDIA/Megatron-LM/pull/6828), [#6845](https://github.com/NVIDIA/Megatron-LM/pull/6845), [#6883](https://github.com/NVIDIA/Megatron-LM/pull/6883), [#6963](https://github.com/NVIDIA/Megatron-LM/pull/6963)

### Disaggregated Inference, Prefix-Affinity Routing, and ReFIT Safety

Inference work is developing independently scaled prefill and decode workers, direct state handoff, prefix-affinity routing, lease-based cache management, asynchronous scheduling, and safer model refits.

- The disaggregated path targets direct NCCL or NIXL transfer of KV and SSM state between independently parallelized prefill and decode shards.
- Weight-version-salted prefix hashes, load-aware affinity routing, cache leases, and lifecycle fixes aim to prevent stale reuse and coordinate cache protection and eviction.
- Local-copy stream ordering, Mamba cache refresh, and direct BF16-to-MXFP8 conversion into persistent buffers address stale parameters and refit allocation overhead.
- Data-parallel HTTP frontends, ZMQ subscription rendezvous, asynchronous scheduling, request cancellation, and ReplaySSM speculative rollback extend serving scalability and resilience.

Related PRs: [#6196](https://github.com/NVIDIA/Megatron-LM/pull/6196), [#6223](https://github.com/NVIDIA/Megatron-LM/pull/6223), [#6497](https://github.com/NVIDIA/Megatron-LM/pull/6497), [#6617](https://github.com/NVIDIA/Megatron-LM/pull/6617), [#6668](https://github.com/NVIDIA/Megatron-LM/pull/6668), [#6675](https://github.com/NVIDIA/Megatron-LM/pull/6675), [#6715](https://github.com/NVIDIA/Megatron-LM/pull/6715), [#6723](https://github.com/NVIDIA/Megatron-LM/pull/6723), [#6823](https://github.com/NVIDIA/Megatron-LM/pull/6823), [#6852](https://github.com/NVIDIA/Megatron-LM/pull/6852), [#6853](https://github.com/NVIDIA/Megatron-LM/pull/6853), [#6916](https://github.com/NVIDIA/Megatron-LM/pull/6916), [#6932](https://github.com/NVIDIA/Megatron-LM/pull/6932), [#6952](https://github.com/NVIDIA/Megatron-LM/pull/6952)

### GatedDeltaNet, Mamba2, and Wide/Attention Residual Hybrid Models

Hybrid architecture work is adding native and fused GatedDeltaNet kernels, Mamba2 state-passing context parallelism, wide residual streams, Attention Residuals, KDA–MLA variants, and broader CUDA-graph compatibility.

- Internal CuTe and Transformer Engine paths provide experimental native and fused GatedDeltaNet execution, including replay-safe variable-length THD capture.
- Mamba2 context parallelism exchanges convolution halos and recurrent-state summaries, while selective replay reduces retained wide-boundary activations.
- Streamwise wide residuals keep attention, MLP, and MoE branches at ordinary width and are being composed with MTP, MIMO, Mamba, and mixed HybridStack models.
- Attention Residuals, KDA low-rank projections, MLA NOPE, runtime CP groups, allocator stream affinity, and full-iteration graph fixes expand hybrid model choices and graph coverage.

Related PRs: [#6511](https://github.com/NVIDIA/Megatron-LM/pull/6511), [#6645](https://github.com/NVIDIA/Megatron-LM/pull/6645), [#6716](https://github.com/NVIDIA/Megatron-LM/pull/6716), [#6796](https://github.com/NVIDIA/Megatron-LM/pull/6796), [#6805](https://github.com/NVIDIA/Megatron-LM/pull/6805), [#6808](https://github.com/NVIDIA/Megatron-LM/pull/6808), [#6821](https://github.com/NVIDIA/Megatron-LM/pull/6821), [#6827](https://github.com/NVIDIA/Megatron-LM/pull/6827), [#6840](https://github.com/NVIDIA/Megatron-LM/pull/6840), [#6855](https://github.com/NVIDIA/Megatron-LM/pull/6855), [#6877](https://github.com/NVIDIA/Megatron-LM/pull/6877), [#6958](https://github.com/NVIDIA/Megatron-LM/pull/6958)

### Muon Head-Wise, Hyperball, and Layer-Sharded Tensor Parallelism

Muon development is adding head-wise QKV handling, Hyperball projection, per-domain execution modes, gated-projection correctness, deterministic parameter-group ordering, and layer-sharded Newton–Schulz work.

- Head-wise orthogonalization covers MHA, GQA, gated attention, MLA, fragmented TP, and GTP-rematerialized layouts while excluding padding from head boundaries.
- TensorParallelMuonHT introduces fixed-radius initialization and Hyperball updates with logical Frobenius norms across TP, expert-TP, and GTP shards.
- Layer-sharded Muon assigns each matrix to one home rank; the reported optimizer step fell from 264 ms to 139 ms and NCCL calls fell to four at the stated production shapes.
- Independent dense and expert mode selection, gated-FC1 splitting, and stable expert-aware parameter ordering address execution choice and collective consistency.

Related PRs: [#6326](https://github.com/NVIDIA/Megatron-LM/pull/6326), [#6422](https://github.com/NVIDIA/Megatron-LM/pull/6422), [#6616](https://github.com/NVIDIA/Megatron-LM/pull/6616), [#6683](https://github.com/NVIDIA/Megatron-LM/pull/6683), [#6688](https://github.com/NVIDIA/Megatron-LM/pull/6688), [#6761](https://github.com/NVIDIA/Megatron-LM/pull/6761)

### MTP Alignment, Variable-Length THD Packing, and Durable RL Rollouts

Ongoing work is aligning MTP across packed and context-parallel layouts, completing variable-length THD training, reducing RL logit materialization, and preserving partial rollout progress.

- Absolute-offset roll contexts and CP-group boundary exchanges aim to keep MTP inputs, targets, losses, and CUDA-graph values aligned across local, contiguous, zigzag, and dynamic layouts.
- A draft-only total-variation MTP objective is being added alongside fused kernels, while variable-length sequence packing is being wired through training and evaluation.
- Vocabulary-parallel selected-token log probabilities avoid full-vocabulary logits for eligible RL consumers.
- Per-rollout ledgers preserve completed members of partial groups, while FSDP and Transformer Engine microbatch-state fixes prevent duplicate finalization and accumulated-gradient overwrites.

Related PRs: [#6205](https://github.com/NVIDIA/Megatron-LM/pull/6205), [#6473](https://github.com/NVIDIA/Megatron-LM/pull/6473), [#6605](https://github.com/NVIDIA/Megatron-LM/pull/6605), [#6706](https://github.com/NVIDIA/Megatron-LM/pull/6706), [#6741](https://github.com/NVIDIA/Megatron-LM/pull/6741), [#6742](https://github.com/NVIDIA/Megatron-LM/pull/6742), [#6746](https://github.com/NVIDIA/Megatron-LM/pull/6746), [#6768](https://github.com/NVIDIA/Megatron-LM/pull/6768), [#6875](https://github.com/NVIDIA/Megatron-LM/pull/6875), [#6876](https://github.com/NVIDIA/Megatron-LM/pull/6876), [#6965](https://github.com/NVIDIA/Megatron-LM/pull/6965)

### Qwen3.5-VL, Nemotron Omni, and MIMO Multimodal Execution

Multimodal work is extending Qwen3.5-VL training and GTP, Nemotron image/video/audio inference and SFT masking, dynamic-resolution visual processing, and configurable MIMO placement and communication.

- Qwen3.5-VL is being migrated to HybridModel with multimodal RoPE, Energon data loading, packed-sequence and CP handling, and generalized tensor parallelism.
- Dynamic-resolution pixel shuffling uses actual patch grids and video tubelets to align visual feature ordering between policy training and inference.
- The Nemotron Omni draft targets fixed-address image, video, and audio embeddings across paged caching, chunked prefill, and CUDA graphs, but the input states that it has not been executed or tested.
- MIMO work adds optional projector placement, bridge communication policies, modality-aware checkpoint replica IDs, and Nemotron SFT masking for assistant responses and retained thinking traces.

Related PRs: [#6193](https://github.com/NVIDIA/Megatron-LM/pull/6193), [#6255](https://github.com/NVIDIA/Megatron-LM/pull/6255), [#6308](https://github.com/NVIDIA/Megatron-LM/pull/6308), [#6315](https://github.com/NVIDIA/Megatron-LM/pull/6315), [#6424](https://github.com/NVIDIA/Megatron-LM/pull/6424), [#6608](https://github.com/NVIDIA/Megatron-LM/pull/6608), [#6792](https://github.com/NVIDIA/Megatron-LM/pull/6792), [#6843](https://github.com/NVIDIA/Megatron-LM/pull/6843), [#6888](https://github.com/NVIDIA/Megatron-LM/pull/6888)

### Kernel Backend Selection, Tensor Metrics, and Determinism Tracing

Execution infrastructure is being developed for per-operation backend selection, distributed tensor diagnostics, rank-local determinism tracing, and reproducible low-level execution behavior.

- A rank-local JSONL tracer and offline comparator provide metadata, summaries, samples, and full-tensor evidence without distributed collectives.
- Tensor observation hooks, distributed metric definitions, and training-loop integration expose parameters, gradients, residuals, logits, and router data through TensorBoard, Weights & Biases, or stdout.
- A centralized backend registry defines availability, determinism, fusion declarations, defaults, and per-operation overrides, replacing duplicated model-spec selection logic.
- Rank-local RNG checkpoint offsets, corrected batch-invariant RMSNorm, explicit Triton cache requirements, and independently controlled MoE auxiliary-loss fusion target reproducible resume and execution.

Related PRs: [#6610](https://github.com/NVIDIA/Megatron-LM/pull/6610), [#6647](https://github.com/NVIDIA/Megatron-LM/pull/6647), [#6648](https://github.com/NVIDIA/Megatron-LM/pull/6648), [#6649](https://github.com/NVIDIA/Megatron-LM/pull/6649), [#6650](https://github.com/NVIDIA/Megatron-LM/pull/6650), [#6689](https://github.com/NVIDIA/Megatron-LM/pull/6689), [#6690](https://github.com/NVIDIA/Megatron-LM/pull/6690), [#6766](https://github.com/NVIDIA/Megatron-LM/pull/6766), [#6858](https://github.com/NVIDIA/Megatron-LM/pull/6858), [#6859](https://github.com/NVIDIA/Megatron-LM/pull/6859), [#6966](https://github.com/NVIDIA/Megatron-LM/pull/6966), [#6967](https://github.com/NVIDIA/Megatron-LM/pull/6967)

## Scope and Method

The narrative is grouped by technical theme. Related `dev` and `main` PRs are described once; the complete PR ledger, exclusion evidence, and state events remain in the local database. PRs closed without merge are counted but not narrated.

This issue is a monthly activity report for NVIDIA/Megatron-LM and does not name a source file, test, or specific change to make. Start by reviewing the report period and linked pull requests; completion criteria are not specified in the issue body.

Written by the indexing model from the issue text.