[QUESTION] Is there any way to specify the model placement in PP manually?
Open
question
- Dominant language
- Python
- Stars
- 17.9k
- Forks
- 4.5k
- Avg merge
- 4d 3h
- Merged PRs (30d)
- 272
Description
Hi,
I'm experimenting with a "dualpipe-ish" pipeline scheduling idea and need to manually control how model stages are placed across different ranks. Specifically, I want to achieve a layout like this:
- **Model A**:
- First half on `rank0`
- Second half on `rank1`
- **Model B**:
- First half on `rank1`
- Second half on `rank0`
Like:
Is there a recommended way to manually assign pipeline stages to specific ranks in megatron? I've looked through the documentation for the pipeline scheduler but haven't found a direct approach for this kind of interleaved placement.
Thanks in advance!
Contributor guide
Assessment
This issue has not been assessed yet.