[DISCUSS] Could we have a different ANN algorithm for Learned Sparse Vectors?

### Description

Learned Sparse Vectors claim to combine the benefits of sparse (i.e. lexical) and dense (i.e. vector) representations

From https://en.wikipedia.org/wiki/Learned_sparse_retrieval:
> Learned sparse retrieval or sparse neural search is an approach to [text search](https://en.wikipedia.org/wiki/Full-text_search) which uses a sparse vector representation of queries and documents.[[1]](https://en.wikipedia.org/wiki/Learned_sparse_retrieval#cite_note-1) It borrows techniques both from lexical [bag-of-words](https://en.wikipedia.org/wiki/Bag-of-words_model) and [vector embedding](https://en.wikipedia.org/wiki/Vector_embedding) algorithms, and is claimed to perform better than either alone. The best-known sparse neural search systems are SPLADE[[2]](https://en.wikipedia.org/wiki/Learned_sparse_retrieval#cite_note-2) and its successor SPLADE v2

From https://zilliz.com/learn/enhancing-information-retrieval-learned-sparse-embeddings:
> Learn sparse embeddings denote sparse vector representations of data crafted through sophisticated machine learning models such as [SPLADE](https://zilliz.com/learn/discover-splade-revolutionize-sparse-data-processing) and BGE-M3. Unlike traditional sparse vectors, which rely solely on statistical methods like BM25, learned sparse embeddings enrich the sparse representation with contextual information while retaining keyword search capabilities. They can discern the significance of adjacent or correlated tokens, even if not explicitly present in the text, resulting in a "learned" sparse representation adept at capturing relevant keywords and classes.

A famous model for such sparse representations of documents is SPLADE (https://github.com/naver/splade)

### Paper

Came across _Efficient Inverted Indexes for Approximate Retrieval over Learned Sparse Representations (https://arxiv.org/pdf/2404.18812)_ which shows promising benchmarks for KNN search over learned sparse vectors:

> Experimental results show average per-query latency in microsecond territory on various sparse embeddings of Ms Marco. Impressively, Seismic outperforms the graph-based winning solutions of the BigANN Challenge by a factor of at least 3.4 at 95% accuracy on Splade and 12 on Efficient Splade, with the margin widening substantially as accuracy increases. Other baselines, including state-of-the-art inverted index-based algorithms, are consistently one to two orders of magnitude slower than Seismic

**Figure 3** in the linked paper summarizes the design of the algorithm:

> The design of Seismic. Inverted lists are independently partitioned into geometrically-cohesive blocks. Each block is a set of document identifiers with a summary vector. The inner product of a query with the summary approximates the inner product attainable with the documents in that block. The forward index stores the complete vectors (including values).

Learned Sparse Vectors seem to be naturally compatible with inverted indexes, and many aspects of the algorithm are already implemented in Lucene
Could we use this for faster KNN search when sparse vectors are used?