AnswerDotAI / AnswerDotAI/ModernBERT
Is there a config file for ModernBERT-Large?
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 145
- PR merge metrics
- No merged PRs in 30d
Description
Hello,
I have been trying to replicate your pretraining results and I wanted to understand which model reported in the paper corresponds to which yaml file listed in the repository.
I think (based on this [pull request](https://github.com/AnswerDotAI/ModernBERT/pull/203)) that flex-bert-base.yaml is the ModernBERT base in the paper. I am not clear on what the difference is between flex-bert-base.yaml and flex-bert-base-parallel.yaml. I also do not see a config file for the ModernBERT Large.
Thank you for any assistance you can provide. I am interested in pretraining the ModernBERT Large on my own dataset using different tokenizers.
LeAnn
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by comparing flex-bert-base.yaml and flex-bert-base-parallel.yaml, then search the repository for references connecting configuration files to the paper's model variants. Check whether a ModernBERT Large configuration is present; done means the base mappings, parallel variant distinction, and Large configuration status are clearly documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, yaml
- Domain
- documentation
- Issue type
- Documentation
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100