michaelfeil / michaelfeil/infinity

Support for v2 version of Mixedbread Reranker Base

Open
#566 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
2.9k
Forks
206
PR merge metrics
No merged PRs in 30d

Description

### Model description

Hi there :)

I have been using the `mixedbread-ai/mxbai-rerank-base-v1` served with Infinity via Runpod for some time now.
However, mixedbread has released a v2 version: https://huggingface.co/mixedbread-ai/mxbai-rerank-base-v2,

for which I get the following error when trying to use in the same way I am using v1:

`{
"error": {
"message": "ModelNotDeployedError: model=`mixedbread-ai/mxbai-rerank-base-v2` does not support `rerank`. Reason: the loaded moded cannot fullyfill `rerank`. Options are {'embed'}.",
"type": null,
"param": null,
"code": 400
}
}`

### Open source status & huggingface transformers.

- [ ] The model implementation is available on transformers
- [ ] The model weights are available on huggingface-hub
- [ ] I verified that the model is currently not running in the latest version `pip install infinity_emb[all] --upgrade`
- [ ] I made the authors of the model aware that I want to use it with infinity_emb & check if they are aware of the issue.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reviewing the mixedbread-ai/mxbai-rerank-base-v2 model page and reproduce the failure with the latest `infinity_emb[all]` installation. Compare its available transformers behavior with the existing v1 support. Done means the v2 model can be served through Infinity for reranking without the ModelNotDeployedError.

Written by the indexing model from the issue text.

Assessment

Tech stack
huggingface, python
Domain
machine-learning
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.