Cannot use /v1/embeddings api with BGE-M3 model
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 39.5k
- Forks
- 4.8k
- PR merge metrics
- No merged PRs in 30d
Description
fastchat version 0.2.36
First, start the services:
python3 -m fastchat.serve.controller --port 21002 --host 0.0.0.0
python3 -m fastchat.serve.model_worker --port 21003 --model-names bge-m3 --model-path /root/proj/models/bge-m3 --device cpu --controller-address http://0.0.0.0:21002 --host 0.0.0.0
python3 -m fastchat.serve.openai_api_server --controller-address http://0.0.0.0:21002 --port 8001 --host 0.0.0.0
Then, post requests to the 8001 port:
import requests
url = "http://10.99.106.214:8001/v1/embeddings"
headers = {
"Content-Type": "application/json"
}
data = {
"model": "bge-m3",
"input": "Hello world!"
}
response = requests.post(url, headers=headers, json=data)
print(response.text)_
and get the response:
{"object":"error","message":"Not Found","code":50001}
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the reported setup with the controller, model_worker, and openai_api_server commands using the bge-m3 model, then inspect how the /v1/embeddings endpoint routes requests. Done means the posted request no longer returns the Not Found error and produces an embedding response; no test or source file is named in the issue.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100