michaelfeil / michaelfeil/infinity
Pass kwargs to encoder
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 2.9k
- Forks
- 206
- PR merge metrics
- No merged PRs in 30d
Description
### Feature request
Models like https://huggingface.co/BAAI/bge-m3 and https://huggingface.co/jinaai/jina-embeddings-v3 can take extras kwargs as input of the `encode` function such as `task=...` for Jina v3 or `return_dense=False/True` for bge-m3
It would be great if we could pass these kwargs either when using the async engine via the Python API
`engine.embed(sentences=[...], additional_args=**kwargs)`
or when we are sending requests to an endpoint create using your docker image
`r = requests.post("http://0.0.0.0:7997/embeddings", json={"model":"test_model","input":["Two cute cats."], "task": "text-matching"})`
### Motivation
This would could also be used to handle `truncate_dim` for Matryoshka embeddings.
might be linked to: #476
### Your contribution
I could try to implement it on my free time but I do not have much currently plus I'm still navigating the code. Any pointers at where to start are welcome.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing the Python API entry point for engine.embed and the /embeddings request handler in the Docker-served endpoint through to the model's encode call. Confirm how request parameters are currently parsed and forwarded, then verify that kwargs such as task, return_dense, and truncate_dim reach both paths without changing existing behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, python
- Domain
- api, backend
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100