NVIDIA / NVIDIA/Megatron-LM

fp8-param-gather for mxfp8

Open
#2,582 5 comments 0 reactions 1 assignee Claimed by @kunlunl View on GitHub
bug community-request waiting-on-maintainers
Dominant language
Python
Stars
17.9k
Forks
4.5k
Avg merge
4d 6h
Merged PRs (30d)
271

Description

Training on b200:
These errors seem to oppose each other

AssertionError: When `--fp8-param-gather` is enabled, the optimizer cpu offload must be used in conjunction with `--fp8-recipe delayed`.

(MegatronTrainRayActor pid=336553, ip=10.142.0.5) /root/Megatron-LM/megatron/core/optimizer/optimizer_config.py:212: UserWarning: mxfp8 without using reuse_grad_buf_for_mxfp8_param_ag and fp8_param_gatherwill use significant amount additional GPU memory.Setting --reuse-grad-buf-for-mxfp8-param-ag and --fp8-param-gather is recommended for mxfp8 training.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.