NVIDIA / NVIDIA/Megatron-LM

[QUESTION]Question about packed_seq_params in decoder intermediate ranks when using sequence packing

Open
#1,518 1 comment 1 reaction 0 assignees View on GitHub
module: multimodal question
Dominant language
Python
Stars
17.9k
Forks
4.5k
Avg merge
4d 3h
Merged PRs (30d)
272

Description

**Your question**

I noticed that the ```get_batch``` function (Lines 51–53 in the given figure) in ```examples/multimodal/train.py``` returns all None values without calling ```next(data_iterator)``` for the decoder's intermediate pipeline parallel ranks.
However, isn't packed_seq_params still necessary in those ranks for computing self-attention maps or calling ```apply_rotary_pos_emb``` In ```megatron/core/models/common/embeddings/rope_utils.py```, if we use sequence packing?
Just wanted to clarify if this is expected behavior or if something might be missing. Thanks!

Image

Contributor guide

Open the contributing guide

Research direction

Start with get_batch in examples/multimodal/train.py and trace how packed_seq_params reaches decoder intermediate pipeline-parallel ranks. Then inspect the packed-sequence consumers in megatron/core/models/common/embeddings/rope_utils.py, determining whether the observed None values are expected; done means documenting the behavior or identifying a confirmed missing path.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
distributed-systems, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.