AnswerDotAI / AnswerDotAI/ModernBERT
Are there minimum hardware requirements regarding pre-training?
Open
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 145
- PR merge metrics
- No merged PRs in 30d
Description
good job for this awesome work! I want to ask if there are _specific minimum hardware requirements_ regarding pre-training, something that I didn't find in the paper. thanks!
I want to pre-train ModernBERT from scratch but I am resource-constrained.
Contributor guide
No contributing guide indexed for this repository
Research direction
Review the paper and ModernBERT's pre-training setup to determine the minimum hardware needed to pre-train from scratch. Done means the resource requirements and relevant constraints are documented clearly enough for a resource-constrained user to assess feasibility.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100