NVIDIA / NVIDIA/Megatron-LM

docs: Fix typos, grammar inconsistencies, and missing information across multiple documentation files

Open
#4,875 2 comments 0 reactions 1 assignee Claimed by @Phlip79 View on GitHub
community-request waiting-on-customer
Dominant language
Python
Stars
17.9k
Forks
4.5k
Avg merge
4d 6h
Merged PRs (30d)
271

Description

## Summary

I've identified several documentation issues across multiple files that affect clarity, consistency, and completeness. These range from simple typos to missing prerequisite information.

## Issues Found

### `docs/developer/contribute.md`

1. **Typo:** "recieve" → should be "receive"
> "Older issues we believe still be relevant may **recieve** a request to re-test them"

2. **Grammar error:** "if you they should be re-opened" → stray "you"
> "if **you they** should be re-opened then just respond with a comment"

3. **Spelling inconsistency:** "licence" (British) used while rest of doc uses American English ("license agreement")
> "Submit code that's incompatible with the project **licence**."

4. **Possibly outdated:** "rebased on the master branch" — many repos have moved to `main` as default branch name. Should verify current default branch.

### `docs/get-started/overview.md`

5. **Inconsistent naming:** "Megatron-Core" (hyphenated) vs "Megatron Core" (no hyphen) used interchangeably throughout the document.

6. **Incorrect terminology:** Hugging Face referred to as "training model" — it's a training framework/library.
> "Megatron Bridge connects Megatron-Core and Megatron-LM to other popular training **models**, such as Hugging Face."

7. **Redundant text:** "expands the capability/capabilities of Megatron-LM" appears in both the Overview and Megatron Core sections.

8. **Awkward grammar:** "Megatron-LM is a reference implementation, with a lightweight large-scale LLM training framework." — comma creates confusing appositive.

9. **Hyphenation:** "multi-billion and trillion-parameter" — should use suspended hyphen: "multi-billion- and trillion-parameter".

### `docs/get-started/quickstart.md`

10. **Missing system requirements:** No CUDA version, GPU requirements, or Python version prerequisites listed. The LLaMA-3 example targets "8 GPUs" with FP8 on H100 but this isn't stated as a requirement.

11. **Vague tokenizer info:** `--tokenizer-type` lists options followed by "and so on" — incomplete and unhelpful.

12. **No format explanation:** Megatron expects `.bin`/`.idx` binary files but never explains what these contain or why.

13. **Ambiguous flag:** `--tokenizer-model` described for all tokenizer types, but `HuggingFaceTokenizer` uses a different format than `.model` files.

### `README.md`

14. **Potentially broken link:** Mamba example links to `tree/ssm/examples/mamba` — the `ssm` branch may no longer exist after merge.

15. **Assumed tool:** `uv pip install` used without explaining what `uv` is or offering standard `pip install` alternative.

16. **Version ambiguity:** Deprecation notice mentions Python 3.12 but nowhere states current minimum supported version.

17. **Missing prerequisites:** No system requirements section (CUDA, GPU, driver versions).

18. **Duplicate links:** MoE Roadmap link appears 3 times, Contributing Guide appears 3 times, Megatron Bridge appears 3 times.

## Proposed Fix

I'd like to submit PRs addressing these issues, starting with the straightforward typos and grammar fixes in `contribute.md`, then moving to the structural improvements in other files.

## Labels

`documentation`, `good first issue`

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.