bigscience-workshop / bigscience-workshop/Megatron-DeepSpeed
DeepSpeedCheckpoint needs to support bf16 optimizer states.
Open
- Dominant language
- Python
- Stars
- 1.4k
- Forks
- 226
- PR merge metrics
- No merged PRs in 30d
Description
https://github.com/bigscience-workshop/Megatron-DeepSpeed/blob/a72225908e9bbda4d989bcdecd71c3c4a05a7f71/tools/convert_checkpoint/deepspeed_checkpoint.py#L5 seems wrong since the files generated using bf16 have `bf16_zero_pp_rank` as prefix.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.