michaelfeil/infinity

View on GitHub

Infinity is a high-throughput, low-latency serving engine for text-embeddings, reranking models, clip, clap and colpali

Stars: 2.9k
Forks: 206
Open beginner issues: 0
Indexed issues: 110

Dominant language: Python
License: MIT
Last GitHub push: Mar 24, 2026
Latest indexed: Sep 19, 2026
Contributing guide: No contributing guide
Code of conduct: No code of conduct
Beginner labels: help wanted good first issue
PR merge metrics: No merged PRs in 30d

Beginner friendly Everything

Open All

110 open issues indexed

is this repo deprecated? Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

michaelfeil/infinity#673 · Opened Sep 9, 2026
Expose a tokenizer API (/tokenize + /decode), aligned with TEI/Cohere Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

michaelfeil/infinity#667 · 1 comment · Opened Jun 18, 2026
allow transformers>=5.0.0rc3 to fix reported CVEs Open

Difficulty 2/5 1-3 hours Newbie friendliness 68/100

michaelfeil/infinity#664 · 1 comment · Opened Jun 9, 2026
Memory leak: bge-m3 RSS grows from 2 GiB → 28 GiB over hours on AMD W6800 (gfx1030, ROCm 6.4) Open

Difficulty 4/5 3-5 days Newbie friendliness 38/100

michaelfeil/infinity#661 · 1 comment · 1 reaction · Opened May 23, 2026
📋 Documentation Enhancement Suggestion Open

Difficulty 2/5 1-3 hours Newbie friendliness 35/100

michaelfeil/infinity#660 · 2 reactions · Opened Mar 7, 2026
The /classify method does not respond to the raw_scores parameter. Open

Difficulty 3/5 1-2 days Newbie friendliness 45/100

michaelfeil/infinity#658 · Opened Feb 22, 2026
Need support for transformers==5.0.0 or greater Open

Difficulty 3/5 1-2 days Newbie friendliness 42/100

michaelfeil/infinity#657 · 2 comments · 2 reactions · Opened Feb 12, 2026
[Bug]: ModuleNotFoundError: No module named 'optimum.bettertransformer' when running v2 CLI Open

Difficulty 3/5 1-2 days Newbie friendliness 45/100

michaelfeil/infinity#656 · 4 comments · 7 reactions · Opened Feb 11, 2026
Support Request: Add `voyage-4-nano` Model Integration (Qwen3 Architecture) Open

Difficulty 4/5 3-5 days Newbie friendliness 32/100

michaelfeil/infinity#655 · Opened Feb 9, 2026
Getting Errors during ONNX translation Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

michaelfeil/infinity#652 · 3 comments · Opened Dec 23, 2025
need httpx[http2]==0.28.1support for this package Open

Difficulty 2/5 1-3 hours Newbie friendliness 68/100

michaelfeil/infinity#651 · 2 comments · Opened Dec 1, 2025
Running infinity cli with click >8.2.1 fails with TypeError: Secondary flag is not valid for non-boolean flag Open

Difficulty 2/5 1-3 hours Newbie friendliness 45/100

michaelfeil/infinity#650 · 3 comments · 1 reaction · Opened Oct 31, 2025
Optimum 2.0.0: Bettertransformer deprecated Open

Difficulty 3/5 1-2 days Newbie friendliness 38/100

michaelfeil/infinity#649 · 5 comments · 2 reactions · Opened Oct 30, 2025
Support google/embeddinggemma-300m and Qwen/Qwen3-Reranker-0.6B Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

michaelfeil/infinity#642 · 3 comments · 2 reactions · Opened Sep 18, 2025
Support Customizable Logging Format in Infinity Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

michaelfeil/infinity#640 · 1 comment · Opened Aug 28, 2025
Pass parameter "providers=['CUAExecutionProvider','TensorrtExecutionProvider'] " to "optimum" Engine Open

Difficulty 4/5 3-5 days Newbie friendliness 45/100

michaelfeil/infinity#639 · 3 comments · Opened Aug 23, 2025
Discrepancy in embeddings similarity between Infinity and SentenceTransformer / HF TEI Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

michaelfeil/infinity#634 · 8 comments · Opened Aug 21, 2025
No support for arm64? Open

Difficulty 4/5 3-5 days Newbie friendliness 32/100

michaelfeil/infinity#616 · 8 comments · Opened Jul 10, 2025
Support for jinaai/jina-embeddings-v4 Open

Difficulty 4/5 3-5 days Newbie friendliness 32/100

michaelfeil/infinity#615 · 1 comment · 4 reactions · Opened Jul 3, 2025
Qwen3-Embedding models do not work Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

michaelfeil/infinity#611 · 14 comments · 1 reaction · Opened Jun 30, 2025
1x1 input image can crash /v1/embeddings endpoint until pod restart while health check endpoint continues to return 200 Open

Difficulty 4/5 3-5 days Newbie friendliness 28/100

michaelfeil/infinity#609 · 6 comments · Opened Jun 26, 2025
Allow to pass SentenceTransformers `task` and `prompt` Open

Difficulty 4/5 3-5 days Newbie friendliness 45/100

michaelfeil/infinity#607 · 10 comments · Opened Jun 25, 2025
The "embedding_dtype" feature is not supported for fp16. Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

michaelfeil/infinity#606 · 2 comments · Opened Jun 24, 2025
support new model: jinaai/jina-reranker-m0 Open

Difficulty 4/5 3-5 days Newbie friendliness 45/100

michaelfeil/infinity#599 · 1 reaction · Opened Jun 13, 2025
support Qwen3 embedding Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

michaelfeil/infinity#598 · 5 comments · 1 reaction · Opened Jun 13, 2025
Support user-defined instructions Open

Difficulty 5/5 Over a week Newbie friendliness 30/100

michaelfeil/infinity#596 · 1 reaction · Opened Jun 11, 2025
support VL embedding Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

michaelfeil/infinity#595 · 2 comments · Opened May 30, 2025
AMD MI50 does not work since version 0.069 Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

michaelfeil/infinity#593 · 2 comments · Opened May 29, 2025
Please support ImageBind Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

michaelfeil/infinity#592 · 1 reaction · Opened May 26, 2025
RuntimeError: FlashAttention only supports Ampere GPUs or newer，How to solve Open

Difficulty 4/5 3-5 days Newbie friendliness 28/100

michaelfeil/infinity#591 · 1 comment · Opened May 21, 2025
Latest version of transformers breaks Infinity Server Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

michaelfeil/infinity#590 · 5 comments · 1 reaction · Opened May 20, 2025
Support for nomic-ai/colnomic-embed-multimodal-7b Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

michaelfeil/infinity#589 · 3 comments · Opened May 20, 2025
support for siglip 2 models Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

michaelfeil/infinity#581 · 8 comments · Opened May 5, 2025
Docs on disabling vector disk cache on high throughput scenario Open

Difficulty 2/5 1-3 hours Newbie friendliness 38/100

michaelfeil/infinity#580 · Opened May 5, 2025
High memory usage while loading onnx model with optimum engine Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

michaelfeil/infinity#579 · 1 comment · Opened May 5, 2025
Warmup fails for embedding model Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

michaelfeil/infinity#577 · 1 comment · Opened Apr 29, 2025
How to set customize max_length via engineArgs? Open

Difficulty 3/5 1-2 days Newbie friendliness 15/100

michaelfeil/infinity#576 · 1 reaction · Opened Apr 27, 2025
Scaling improvement for CPU-bound embedding tasks Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

michaelfeil/infinity#575 · Opened Apr 25, 2025
How to limit memory usage? Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

michaelfeil/infinity#573 · Opened Apr 19, 2025
Support for v2 version of Mixedbread Reranker Base Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

michaelfeil/infinity#566 · 4 comments · Opened Apr 12, 2025
Support for Linq-AI-Research/Linq-Embed-Mistral Open

Difficulty 4/5 3-5 days Newbie friendliness 32/100

michaelfeil/infinity#565 · 5 comments · Opened Apr 9, 2025
Is ColBERT model provide the right result for query and document vector embedding and rerank? Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

michaelfeil/infinity#559 · 1 comment · Opened Mar 17, 2025
Slim down michaelf34/infinity:latest-amd Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

michaelfeil/infinity#555 · 2 comments · 2 reactions · Opened Mar 16, 2025
python engine failed with BGE-VL-base and optimum Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

michaelfeil/infinity#551 · 2 reactions · Opened Mar 13, 2025
Support late chunking Open

Difficulty 5/5 Over a week Newbie friendliness 30/100

michaelfeil/infinity#550 · 1 comment · Opened Mar 8, 2025
Support embedding with "instructions" Again Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

michaelfeil/infinity#548 · 5 comments · Opened Mar 6, 2025
Can we ddd a timeout to the requests in the request queue? Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

michaelfeil/infinity#546 · Opened Mar 4, 2025
mixedbread-ai/mxbai-embed-large-v1 model deployment problem Open

Difficulty 4/5 3-5 days Newbie friendliness 30/100

michaelfeil/infinity#545 · 2 comments · Opened Mar 4, 2025
When the inference process encounters an out-of-memory (OOM) error, can the service automatically recover？ Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

michaelfeil/infinity#544 · 2 comments · 2 reactions · Opened Mar 4, 2025
Disable model warmup Open

Difficulty 3/5 1-2 days Newbie friendliness 45/100

michaelfeil/infinity#542 · Opened Feb 28, 2025

michaelfeil/infinity

Get new issues in your inbox