AnswerDotAI / AnswerDotAI/ModernBERT

How to run pre-training?

Open
#155 7 comments 14 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.7k
Forks
145
PR merge metrics
No merged PRs in 30d

Description

Thank you for the great work! I have a question regarding pre-training. Could you please clarify which YAML configuration file should be used to achieve a similar pre-training setup as ModernBert, but for a different language? I noticed that in the yamls folder, there doesn’t seem to be a specific file for this purpose. The only related script I found is generate_eval_config.py, which, if I understand correctly, generates a YAML configuration using ModernBert’s training params. Is my understanding correct, or am I missing something?

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.