[QUESTION]Question about packed_seq_params in decoder intermediate ranks when using sequence packing
- Dominant language
- Python
- Stars
- 17.9k
- Forks
- 4.5k
- Avg merge
- 4d 3h
- Merged PRs (30d)
- 272
Description
**Your question**
I noticed that the ```get_batch``` function (Lines 51–53 in the given figure) in ```examples/multimodal/train.py``` returns all None values without calling ```next(data_iterator)``` for the decoder's intermediate pipeline parallel ranks.
However, isn't packed_seq_params still necessary in those ranks for computing self-attention maps or calling ```apply_rotary_pos_emb``` In ```megatron/core/models/common/embeddings/rope_utils.py```, if we use sequence packing?
Just wanted to clarify if this is expected behavior or if something might be missing. Thanks!
Contributor guide
Research direction
Start with get_batch in examples/multimodal/train.py and trace how packed_seq_params reaches decoder intermediate pipeline-parallel ranks. Then inspect the packed-sequence consumers in megatron/core/models/common/embeddings/rope_utils.py, determining whether the observed None values are expected; done means documenting the behavior or identifying a confirmed missing path.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- distributed-systems, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100