Low throughput with modernbert

Nobody has claimed this yet.

### System Info

Testing https://huggingface.co/Alibaba-NLP/gte-reranker-modernbert-base

```
INFO 2025-02-11 20:36:37,724 infinity_emb INFO: select_model.py:64
model=`Alibaba-NLP/gte-reranker-modernbert-base`
selected, using engine=`torch` and device=`cuda`
You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
INFO 2025-02-11 20:36:44,229 infinity_emb INFO: using torch.py:88
torch.compile(dynamic=True)
W0211 20:37:23.950000 1 torch/_inductor/utils.py:1137] [6/0] Not enough SMs to use max_autotune_gemm mode
INFO 2025-02-11 20:39:06,469 infinity_emb INFO: Getting select_model.py:97
timings for batch_size=32 and avg tokens per
sentence=3
2.62 ms tokenization
19.90 ms inference
0.04 ms post-processing
22.56 ms total
embeddings/sec: 1418.67

INFO 2025-02-11 20:40:19,740 infinity_emb INFO: Getting select_model.py:103
timings for batch_size=32 and avg tokens per
sentence=1025
52.67 ms tokenization
33388.80 ms inference
0.16 ms post-processing
33441.63 ms total
embeddings/sec: 0.96
```

On NVIDIA L4, seems quite low for ~150M param model?

### Information

- [x] Docker + cli
- [ ] pip + cli
- [ ] pip + usage of Python interface

### Tasks

- [x] An officially supported CLI command
- [ ] My own modifications

### Reproduction

v75 + torch + cuda
Alibaba-NLP/gte-reranker-modernbert-base

Reproduce the official Docker CLI setup using v75 with torch and CUDA, the Alibaba-NLP/gte-reranker-modernbert-base model, and an NVIDIA L4. Compare the reported timings for batches averaging 3 and 1025 tokens, including the Flash Attention and torch.compile messages. Done means explaining the low long-input throughput and verifying an appropriate resolution or documented limitation.

Written by the indexing model from the issue text.