michaelfeil/infinity

View on GitHub

Infinity is a high-throughput, low-latency serving engine for text-embeddings, reranking models, clip, clap and colpali

Stars: 2.9k
Forks: 206
Open beginner issues: 0
Indexed issues: 110

Dominant language: Python
License: MIT
Last GitHub push: Mar 24, 2026
Latest indexed: Sep 19, 2026
Contributing guide: No contributing guide
Code of conduct: No code of conduct
Beginner labels: help wanted good first issue
PR merge metrics: No merged PRs in 30d

Beginner friendly Everything

Open All

110 open issues indexed

OpenAI API compatibility Open

Difficulty 2/5 1-3 hours Newbie friendliness 38/100

michaelfeil/infinity#541 · 2 comments · 3 reactions · Opened Feb 26, 2025
unloading model after some timeout automatically and load the model again after first API inference call Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

michaelfeil/infinity#540 · Opened Feb 26, 2025
InsightFace embedding Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

michaelfeil/infinity#539 · Opened Feb 26, 2025
MRL different dimensions not normalized Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

michaelfeil/infinity#538 · 6 comments · Opened Feb 24, 2025
Late interaction models dimensions Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

michaelfeil/infinity#537 · 3 comments · 2 reactions · Opened Feb 21, 2025
The generated embedding contains accuracy errors. Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

michaelfeil/infinity#535 · 3 comments · Opened Feb 19, 2025
Docker Build Error: failed to create task for container: unable to start container process: exec:"v2" executable file not found in $PATH Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

michaelfeil/infinity#534 · 6 comments · Opened Feb 18, 2025
gme-Qwen2-VL-2B-Instruct Open

Difficulty 3/5 1-2 days Newbie friendliness 30/100

michaelfeil/infinity#532 · 1 comment · Opened Feb 13, 2025
Low throughput with modernbert Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

michaelfeil/infinity#531 · 7 comments · Opened Feb 11, 2025
matryoshka_dim=512 is not in a valid range. Select between 1 and 128. Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

michaelfeil/infinity#519 · 2 comments · Opened Jan 21, 2025
inference speed Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

michaelfeil/infinity#518 · 1 comment · Opened Jan 20, 2025
Infinity embed crashes too easily Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

michaelfeil/infinity#517 · 11 comments · 5 reactions · Opened Jan 15, 2025
GPU Memory is not freed after embedding operations Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

michaelfeil/infinity#514 · 3 comments · Opened Jan 14, 2025
Implement Colbert for Optimum Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

michaelfeil/infinity#512 · Opened Jan 6, 2025
Set default `CMD` to `v2` in Dockerfiles if no command is set Open

Difficulty 2/5 1-3 hours Newbie friendliness 50/100

michaelfeil/infinity#510 · 1 comment · Opened Jan 6, 2025
ColQwen on RunPOD. ValueError: The checkpoint you are trying to load has model type `paligemma` but Transformers does not recognize this architecture. Open

Difficulty 4/5 3-5 days Newbie friendliness 30/100

michaelfeil/infinity#507 · 1 comment · Opened Jan 2, 2025
Support for colbert style late interaction models in rerank endpoint Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

michaelfeil/infinity#503 · Opened Dec 28, 2024
Reranker detected as embedder Jina rerank tiny Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

michaelfeil/infinity#500 · 3 comments · Opened Dec 17, 2024
Support for models trained with SetFit for classification Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

michaelfeil/infinity#497 · 5 comments · Opened Dec 15, 2024
Binary quantization - evaluate quality Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

michaelfeil/infinity#484 · 1 comment · Opened Nov 29, 2024
health endpoint does not really provide insights about healthiness Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

michaelfeil/infinity#483 · 3 comments · 1 reaction · Opened Nov 28, 2024
Pass kwargs to encoder Open

Difficulty 4/5 3-5 days Newbie friendliness 38/100

michaelfeil/infinity#482 · 6 comments · Opened Nov 28, 2024
Support for custom SentenceTransformer models Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

michaelfeil/infinity#474 · 3 comments · Opened Nov 20, 2024
Missing Contribution.md file Open

Difficulty 2/5 1-3 hours Newbie friendliness 35/100

michaelfeil/infinity#433 · 3 comments · Opened Oct 20, 2024
Abstraction for `resolve_torch_dtype_device(dtype: Dtype, device: Device) -> tuple[quantization_type, torch.device, torch.dtype]` Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

michaelfeil/infinity#424 · 1 comment · Opened Oct 14, 2024
Maintainer: Breaking CI / Python installs Open

help wanted

Difficulty 4/5 3-5 days Newbie friendliness 25/100

michaelfeil/infinity#415 · 1 comment · Opened Oct 11, 2024
cli to set max_seq_length = 2048 Open

Difficulty 4/5 3-5 days Newbie friendliness 30/100

michaelfeil/infinity#411 · 1 comment · Opened Oct 10, 2024
[Doc] Documentation on how to run infinity on AMD GPU Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

michaelfeil/infinity#400 · 3 comments · Opened Oct 7, 2024
Feature : Support SPLADE pooling Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

michaelfeil/infinity#399 · 2 comments · 1 reaction · Opened Oct 7, 2024
Better Prometheus metrics: Idea collection thread Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

michaelfeil/infinity#398 · 1 comment · 1 reaction · Opened Oct 5, 2024
Add a End-to-end unit test for image embeddings and audio embeddings Open

help wanted

Difficulty 3/5 1-2 days Newbie friendliness 35/100

michaelfeil/infinity#378 · 2 comments · Opened Sep 24, 2024
when use engine optimum device tensorrt，startup fail Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

michaelfeil/infinity#372 · 5 comments · Opened Sep 23, 2024
Reranker dynamic quantization Open

help wanted

Difficulty 5/5 Over a week Newbie friendliness 25/100

michaelfeil/infinity#363 · 1 comment · Opened Sep 16, 2024
jinaai/jina-reranker-v1-*-en does not work with optimum Open

Difficulty 4/5 3-5 days Newbie friendliness 28/100

michaelfeil/infinity#362 · 1 comment · Opened Sep 13, 2024
Issue running cross-encoder onnx model exported with optimum-cli Open

Difficulty 3/5 1-2 days Newbie friendliness 42/100

michaelfeil/infinity#361 · 2 comments · Opened Sep 13, 2024
Write a custom flash-attention function for the deberta model. Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

michaelfeil/infinity#359 · 1 comment · Opened Sep 12, 2024
Question: Support for colbertv2.0 ? Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

michaelfeil/infinity#355 · 8 comments · Opened Sep 10, 2024
Support Integration with KServe Open

help wanted

Difficulty 5/5 Over a week Newbie friendliness 25/100

michaelfeil/infinity#352 · 1 comment · Opened Sep 6, 2024
Reranker API “top_k” Support Open

Difficulty 4/5 3-5 days Newbie friendliness 30/100

michaelfeil/infinity#342 · 5 comments · Opened Aug 19, 2024
Add Installation Option to Depend Only on ONNX, Excluding New Torch and CUDA Packages Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

michaelfeil/infinity#332 · 3 comments · 1 reaction · Opened Aug 9, 2024
Permit loading of models at different precision at load time for sentence_transformers Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

michaelfeil/infinity#331 · 2 comments · Opened Aug 2, 2024
Jina reranker(turbo/tiny) being classified as embedding models Open

Difficulty 4/5 3-5 days Newbie friendliness 38/100

michaelfeil/infinity#325 · 11 comments · Opened Jul 25, 2024
get asyncio.exceptions.CancelledError error Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

michaelfeil/infinity#317 · 4 comments · Opened Jul 22, 2024
Helm charts for K8s deployment of Infinity Open

enhancement good first issue help wanted

Difficulty 5/5 Over a week Newbie friendliness 25/100

michaelfeil/infinity#314 · 3 comments · Opened Jul 21, 2024
how to accelerate bge m3 sparse embeding module when inference？ Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

michaelfeil/infinity#294 · 3 comments · Opened Jul 2, 2024
[Benchmark] - clip embeddings Open

good first issue help wanted

Difficulty 3/5 1-2 days Newbie friendliness 35/100

michaelfeil/infinity#259 · Opened Jun 10, 2024
[Benchmark] - embedding quantization Open

help wanted

Difficulty 3/5 1-2 days Newbie friendliness 25/100

michaelfeil/infinity#250 · 1 comment · Opened Jun 8, 2024
Tensor-parallelism for multi-gpu support Open

wontfix

Difficulty 5/5 Over a week Newbie friendliness 25/100

michaelfeil/infinity#213 · 3 comments · 1 reaction · Opened Apr 29, 2024
Add a TextSplitter in LangChain to share the model of the embedding model Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

michaelfeil/infinity#193 · 1 reaction · Opened Apr 4, 2024
float16 and other optimizations help? Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

michaelfeil/infinity#159 · 6 comments · Opened Mar 18, 2024

michaelfeil/infinity

Get new issues in your inbox