bigscience-workshop / bigscience-workshop/xmtf
bloomz-mt universal checkpoint
Open
- Dominant language
- Jupyter Notebook
- Stars
- 534
- Forks
- 43
- PR merge metrics
- No merged PRs in 30d
Description
Hello!
Thanks a lot for your job!
I want to finetune bloomz-mt by your Megatron-DeepSpeed，but I can not find a universal version checkpoint of bloomz-mt or bloomz. I only found the bloom universal checkpoint below.
https://huggingface.co/bigscience/bloom-optimizer-states/tree/global_step95000_universal
With limited GPUs，I have to use TP 4, PP 12 to finetune, but I found that you suggest not to merge TP in below document. So I want to find the bloomz-mt universal checkpoint
https://github.com/bigscience-workshop/bigscience/blob/master/train/tr11-176B-ml/finetune.md
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.