michaelfeil / michaelfeil/infinity
Implement Colbert for Optimum
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 2.9k
- Forks
- 206
- PR merge metrics
- No merged PRs in 30d
Description
### Feature request
Add logic for colbert in the optimum engine so it returns token embeddings
### Motivation
Since this is already supported for the torch engine, it will be useful to be able to use onnx versions of Colbert style models correctly as well
### Your contribution
I can potentially submit a PR
(This issue is converted from https://github.com/michaelfeil/infinity/issues/512)
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by comparing how ColBERT token embeddings are handled in the existing torch engine with the corresponding optimum engine path. Confirm the expected output for ONNX versions of ColBERT-style models, then verify that the optimum engine returns token embeddings consistently with the torch engine.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100