michaelfeil / michaelfeil/infinity
Reranker dynamic quantization
Open
help wanted
- Dominant language
- Python
- Stars
- 2.9k
- Forks
- 206
- PR merge metrics
- No merged PRs in 30d
Description
### Feature request
--dtype support for rerankers
### Motivation
Easily quantize cross encoder models
### Your contribution
. (will look into this)
Contributor guide
No contributing guide indexed for this repository
Research direction
The issue does not name any files, tests, or entry points. Start by locating the reranker and cross-encoder model handling, then trace how model dtype is selected and whether dynamic quantization is already supported elsewhere. Done means rerankers accept the requested dtype and quantized cross-encoder behavior is covered by tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100