AnswerDotAI / AnswerDotAI/ModernBERT
checkpoints for further pretraining
Open
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 145
- PR merge metrics
- No merged PRs in 30d
Description
Hi, congratulations on the awesome work.
I was planning on further pretrain your models in another languages, since your version focus mainly in English. However, I am having trouble starting a train ( from main.py) initializing the model weights with the ones from moderbert-base. Are the composer checkpoints available somewhere? or maybe it is possible to start a train in composer using the weights at hugginface in some way? I would greatly appreciate any guidance regarding that.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.