[GTP] Documentation update
- Dominant language
- Python
- Stars
- 17.9k
- Forks
- 4.5k
- Avg merge
- 4d 3h
- Merged PRs (30d)
- 272
Description
What's need to update later:
- Best practices related
- `export NVTE_FWD_LAYERNORM_SM_MARGIN=32`, `export NVTE_BWD_LAYERNORM_SM_MARGIN=32` to make LN fwd/bwd can be overlapped with async GTP AG/RS
- add `MEGATRON_LOGGING_LEVEL=20` if you want to check the GTP prefetch chain construction details
- Don't set `CUDA_DEVICE_MAX_CONNECTIONS` for Blackwell+
- `--gtp-remat-opt-in-modules moe_latent_proj` when the latent size is large enough to amortize the all-gather for more memory saving.
- `export NVTE_INF_LAYERNORM_SM_MARGIN=32` for `gdp_in_proj`
- required for PartialCG:
- `export PYTORCH_CUDA_ALLOC_CONF=graph_capture_record_stream_reuse:True` for memory saving
- export CUDA_GRAPHS_USE_NODE_PRIORITY=1
- export CUDA_DEVICE_MAX_CONNECTIONS=32
- export NVTE_INF_LAYERNORM_SM_MARGIN=32
- GTP+DCP related dtype convetion explaination now that we have changed the workflow for bit-wise matching.
- tune gtp realted nccl comms: `--nccl-communicator-config-path `
- Latest perf / scalability on nemotron-next
- Update supporting matrix for now
- Others...
Contributor guide
Assessment
This issue has not been assessed yet.