contextual embeddings of individual words

I need contextual embeddings of individual words for analysis. However, it turns out that ModernBERT performs much worse than BERT. I use the right version of transformer mentioned in Hugging Face and the following code (same for BERT which works well) to obtain the word-level embeddings. Could there be any problem?

```
from transformers import AutoTokenizer, AutoModel
llm = 'ModernBERT-large'
tokenizer = AutoTokenizer.from_pretrained(llm)
model = AutoModel.from_pretrained(llm)

def embedding(text, tokenizer, model):
input = tokenizer(text, padding="longest", truncation=True, max_length=512, return_tensors="pt")
with torch.no_grad():
outputs = model(**input)
return outputs.last_hidden_state

pos_adj=1
text = f"{textA}{word}{textB}{doc}"
start_pos = len(tokenizer.tokenize(textA)) + pos_adj
end_pos = len(tokenizer.tokenize(textA + word)) + pos_adj
all_embeddings = embedding(text, tokenizer, model)
word_embedding = torch.mean(all_embeddings[0, start_pos:end_pos], dim=0)
```