lm-sys / lm-sys/FastChat

Cannot use /v1/embeddings api with BGE-M3 model

Open
#3,355 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
39.5k
Forks
4.8k
PR merge metrics
No merged PRs in 30d

Description

fastchat version 0.2.36

First, start the services:

python3 -m fastchat.serve.controller --port 21002 --host 0.0.0.0
python3 -m fastchat.serve.model_worker --port 21003 --model-names bge-m3 --model-path /root/proj/models/bge-m3 --device cpu --controller-address http://0.0.0.0:21002 --host 0.0.0.0
python3 -m fastchat.serve.openai_api_server --controller-address http://0.0.0.0:21002 --port 8001 --host 0.0.0.0

Then, post requests to the 8001 port:

import requests

url = "http://10.99.106.214:8001/v1/embeddings"
headers = {
"Content-Type": "application/json"
}
data = {
"model": "bge-m3",
"input": "Hello world!"
}

response = requests.post(url, headers=headers, json=data)

print(response.text)_

and get the response:
{"object":"error","message":"Not Found","code":50001}

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the reported setup with the controller, model_worker, and openai_api_server commands using the bge-m3 model, then inspect how the /v1/embeddings endpoint routes requests. Done means the posted request no longer returns the Not Found error and produces an embedding response; no test or source file is named in the issue.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.