michaelfeil / michaelfeil/infinity
Support for v2 version of Mixedbread Reranker Base
- Dominant language
- Python
- Stars
- 2.9k
- Forks
- 206
- PR merge metrics
- No merged PRs in 30d
Description
### Model description
Hi there :)
I have been using the `mixedbread-ai/mxbai-rerank-base-v1` served with Infinity via Runpod for some time now.
However, mixedbread has released a v2 version: https://huggingface.co/mixedbread-ai/mxbai-rerank-base-v2,
for which I get the following error when trying to use in the same way I am using v1:
`{
"error": {
"message": "ModelNotDeployedError: model=`mixedbread-ai/mxbai-rerank-base-v2` does not support `rerank`. Reason: the loaded moded cannot fullyfill `rerank`. Options are {'embed'}.",
"type": null,
"param": null,
"code": 400
}
}`
### Open source status & huggingface transformers.
- [ ] The model implementation is available on transformers
- [ ] The model weights are available on huggingface-hub
- [ ] I verified that the model is currently not running in the latest version `pip install infinity_emb[all] --upgrade`
- [ ] I made the authors of the model aware that I want to use it with infinity_emb & check if they are aware of the issue.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reviewing the mixedbread-ai/mxbai-rerank-base-v2 model page and reproduce the failure with the latest `infinity_emb[all]` installation. Compare its available transformers behavior with the existing v1 support. Done means the v2 model can be served through Infinity for reranking without the ModelNotDeployedError.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- huggingface, python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100