AnswerDotAI / AnswerDotAI/ModernBERT
How to Extract a Model from a Checkpoint
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 145
- PR merge metrics
- No merged PRs in 30d
Description
I used the /ModernBERT/yamls/main/modernbert-base.yaml file to perform pretraining, and as a result, checkpoint files were saved on the disk.
command: composer main.py /ModernBERT/yamls/main/modernbert-base.yaml
I tried to extract the model from checkpoint files using the following code, but the accuracy is low. Is there a specific method for extracting the model?
```
model, tokenizer = HuggingFaceModel.hf_from_composer_checkpoint(
checkpoint_path='./checkpoints/modernbert-base-en/ep0-ba1400-rank0.pt',
model_instantiation_class='transformers.ModernBertForMaskedLM',
model_config_kwargs={
'num_labels': 2,
"model_type": "modernbert",
}
)
```
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.