openvinotoolkit / openvinotoolkit/model_server
Add dimensions parameter support for embeddings endpoint (Matryoshka)
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 931
- Forks
- 277
- Avg merge
- 2d 13h
- Merged PRs (30d)
- 68
Description
The /v3/embeddings endpoint currently doesn't support the dimensions parameter from the OpenAI API.
Many modern embedding models (like ibm-granite/granite-embedding-311m-multilingual-r2, nomic-embed-text, text-embedding-3-small) support Matryoshka Representation Learning, which allows truncating embeddings to smaller dimensions without retraining.
Request
Add support for the dimensions parameter in EmbeddingsCalculatorOV to truncate and re-normalize the output embeddings.
Example request
{
"model": "my-embedding-model",
"input": "Hello world",
"dimensions": 256
}
Expected behavior
Return embeddings truncated to 256 dimensions (instead of full 768), re-normalized.
References
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at EmbeddingsCalculatorOV and the /v3/embeddings request handling, then trace the existing embedding calculation path. Done means a dimensions request returns truncated, re-normalized embeddings while requests without dimensions retain their full output.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- api, machine-learning
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 68/100