SGLang worker does not expose embeddings.
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 39.5k
- Forks
- 4.8k
- PR merge metrics
- No merged PRs in 30d
Description
Hi,
when running a model with the sglang worker, you get a 404 on the embeddings:
```
curl -X 'POST' \
'https://helmholtz-blablador.fz-juelich.de:8000/v1/embeddings' \
-H 'accept: application/json' \
-H 'Authorization: Bearer MY_API_KEY' \
-H 'Content-Type: application/json' \
-d '{
"model": "Mistral-7B-Instruct-v0.2",
"engine": "string",
"input": "string",
"user": "string",
"encoding_format": "float"
}'
```
The reply is
```
{"object":"error","message":"Not Found","code":50001}%
```
When you call the same model with the slower model_worker, you get the embeddings.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the SGLang worker's handling of the /v1/embeddings endpoint and compare it with the slower model_worker, which is reported to expose embeddings. Reproduce the request from the issue and confirm that the SGLang worker no longer returns 404 while the existing model_worker behavior remains available.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api, backend
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100