AnswerDotAI / AnswerDotAI/ModernBERT

How to Extract a Model from a Checkpoint

Open
#186 3 comments 2 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.7k
Forks
145
PR merge metrics
No merged PRs in 30d

Description

I used the /ModernBERT/yamls/main/modernbert-base.yaml file to perform pretraining, and as a result, checkpoint files were saved on the disk.
command: composer main.py /ModernBERT/yamls/main/modernbert-base.yaml
I tried to extract the model from checkpoint files using the following code, but the accuracy is low. Is there a specific method for extracting the model?

```
model, tokenizer = HuggingFaceModel.hf_from_composer_checkpoint(
checkpoint_path='./checkpoints/modernbert-base-en/ep0-ba1400-rank0.pt',
model_instantiation_class='transformers.ModernBertForMaskedLM',
model_config_kwargs={
'num_labels': 2,
"model_type": "modernbert",
}
)
```

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.