AnswerDotAI / AnswerDotAI/ModernBERT

Continue pretraining on my dataset

Open
#245 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.7k
Forks
145
PR merge metrics
No merged PRs in 30d

Description

Thanks for your incredible models,
I want to fine tune the model using mlm on my dataset. Can i use the regular fine tuning using transformers package or i must use the current repo pretraining method?

Contributor guide

No contributing guide indexed for this repository

Research direction

The issue names no files or tests. Start by comparing the repository's current pretraining method with regular MLM fine-tuning through the Transformers package, then document which workflow applies to a user's dataset and what prerequisites or limitations determine that choice.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.