AnswerDotAI / AnswerDotAI/ModernBERT

checkpoints for further pretraining

Open
#153 11 comments 2 reactions 1 assignee Claimed by @NohTow View on GitHub
Dominant language
Python
Stars
1.7k
Forks
145
PR merge metrics
No merged PRs in 30d

Description

Hi, congratulations on the awesome work.

I was planning on further pretrain your models in another languages, since your version focus mainly in English. However, I am having trouble starting a train ( from main.py) initializing the model weights with the ones from moderbert-base. Are the composer checkpoints available somewhere? or maybe it is possible to start a train in composer using the weights at hugginface in some way? I would greatly appreciate any guidance regarding that.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.