Track R3 RouterReplay support and validation for Megatron VPP
Open
Feature
- Dominant language
- Python
- Stars
- 2k
- Forks
- 562
- Avg merge
- 4d 5h
- Merged PRs (30d)
- 145
Description
## Background
R3 RouterReplay currently assigns replay tensors by Megatron global `layer_number`, which should be the right invariant for virtual pipeline parallelism
(VPP). However, VPP support is not yet available in main, so PR #2590 cannot fully validate real VPP behavior yet.
## Plan
Create a follow-up PR after VPP lands in main to validate and fix any VPP-specific RouterReplay behavior.
Contributor guide
Research direction
Wait for VPP support to land in main, then review PR #2590 and the RouterReplay handling of Megatron global layer_number. Reproduce or add validation for real virtual pipeline parallelism behavior, and use the observed results to identify whether any VPP-specific behavior needs fixing.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100