NVIDIA / NVIDIA/Megatron-LM

Hard-coded RankGenerator arguments in megatron.core.parallel_state.create_all_gather_groups (used for independent AG).

Open
#4,261 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
17.9k
Forks
4.5k
Avg merge
4d 6h
Merged PRs (30d)
271

Description

**Describe the bug**

https://github.com/NVIDIA/Megatron-LM/pull/3249 was not finished with review before it was merged, which had a Claude comment that was unaddressed: https://github.com/NVIDIA/Megatron-LM/pull/3249#discussion_r2989277554

**Steps/Code to reproduce bug**

- Potential bug but mainly code quality.

**Expected behavior**

- Ranks should be generated based on the current PG collection rank assignments.

**Additional context**

Add any other context about the problem here.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.