apache / apache/lucene

Learned sorting algorithm for Lucene

Open
#12,463 5 comments 2 reactions 0 assignees View on GitHub
type:enhancement
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

### Description

I found this [article about an interesting SIGMOD paper](https://blog.acolyer.org/2020/10/19/the-case-for-a-learned-sorting-algorithm/) about a learned sorting algorithm which outperforms Radixsort by the factor 1.49. While the implementation doesn't look trivial, it might be an opportunity to significantly speed up the sorting in Lucene.

edit: There is an updated version ([LearnedSort 2.0](https://learnedsystems.mit.edu/defeating-dups-learned-sort/)) which can deal better with duplicates.

cc: @jpountz @bruno-roustant

Contributor guide

Open the contributing guide

Research direction

The issue names no Lucene file, test, or entry point. Start by reading the linked SIGMOD article and LearnedSort 2.0 material, then locate Lucene’s current Radixsort path and relevant benchmarks; completion would require a scoped implementation proposal and evidence that it improves sorting without regressions.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.