NVIDIA / NVIDIA/Megatron-LM

Support FP8 Param Gather with Optimizer CPU Offloading

Open
#2,407 1 comment 0 reactions 1 assignee Claimed by @kunlunl View on GitHub
enhancement
Dominant language
Python
Stars
17.9k
Forks
4.5k
Avg merge
4d 6h
Merged PRs (30d)
271

Description

**Is your feature request related to a problem? Please describe.**
Support FP8 Param Gather with Optimizer CPU Offloading

**Describe the solution you'd like**
A clear and concise description of what you want to happen.

**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.

**Additional context**
Add any other context or screenshots about the feature request here.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.