NVIDIA / NVIDIA/Megatron-LM

[QUESTION] Is there any way to specify the model placement in PP manually?

Open
#1,856 0 comments 0 reactions 1 assignee Claimed by @gautham-kollu View on GitHub
question
Dominant language
Python
Stars
17.9k
Forks
4.5k
Avg merge
4d 3h
Merged PRs (30d)
272

Description

Hi,

I'm experimenting with a "dualpipe-ish" pipeline scheduling idea and need to manually control how model stages are placed across different ranks. Specifically, I want to achieve a layout like this:

- **Model A**:
- First half on `rank0`
- Second half on `rank1`
- **Model B**:
- First half on `rank1`
- Second half on `rank0`

Like:
Image

Is there a recommended way to manually assign pipeline stages to specific ranks in megatron? I've looked through the documentation for the pipeline scheduler but haven't found a direct approach for this kind of interleaved placement.

Thanks in advance!

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.