Create a singular "to HF" checkpoint script
Open
enhancement
- Dominant language
- Python
- Stars
- 2k
- Forks
- 561
- Avg merge
- 4d 5h
- Merged PRs (30d)
- 145
Description
There are multiple checkpoint formats:
* dtensorv2 (safe tensors)
* dtensorv2 (DCP)
* mcore (DCP)
and we currently provide two checkpoint conversion scripts that users sometimes mistakenly apply (use mcore conversion script for Dtensor V2 DCP b/c the file looks the same).
Since there is enough metadata in the checkpoint directory, we should provide a single conversion entrypoint that automatically converts all checkpoint types.
Contributor guide
Assessment
This issue has not been assessed yet.