michaelfeil / michaelfeil/infinity

Late interaction models dimensions

Open
#537 3 comments 2 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
2.9k
Forks
206
PR merge metrics
No merged PRs in 30d

Description

Hello,

first of all, nice work!

I have been trying to understand the shapes of the colbert models.
as far as I have seen the _**colbert-ir/colbertv2.0**_ has a dimension of **(n_of_tokens - 1) x 768**
the jina model: **_jinaai/jina-colbert-v2_** has a dimension of **(n_of_tokens - 1) x 1024**

compared to the fastembed library the dimensions for both models is **n_of_tokes x 128**

the [example in the provided notebook](https://colab.research.google.com/drive/14FqLc0N_z92_VgL_zygWV5pJZkaskyk7?usp=sharing) is about colipali and there to the embedding dimension is 128.
Is there a compression step missing? or am I missing something else?

thanks!

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the provided ColPali notebook and trace how the ColBERT models are loaded and their embeddings are shaped. Compare those results with the dimensions reported by fastembed, then document whether a compression or projection step exists and what the expected dimensions are.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.