Learned sorting algorithm for Lucene
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
### Description
I found this [article about an interesting SIGMOD paper](https://blog.acolyer.org/2020/10/19/the-case-for-a-learned-sorting-algorithm/) about a learned sorting algorithm which outperforms Radixsort by the factor 1.49. While the implementation doesn't look trivial, it might be an opportunity to significantly speed up the sorting in Lucene.
edit: There is an updated version ([LearnedSort 2.0](https://learnedsystems.mit.edu/defeating-dups-learned-sort/)) which can deal better with duplicates.
cc: @jpountz @bruno-roustant
Contributor guide
Research direction
The issue names no Lucene file, test, or entry point. Start by reading the linked SIGMOD article and LearnedSort 2.0 material, then locate Lucene’s current Radixsort path and relevant benchmarks; completion would require a scoped implementation proposal and evidence that it improves sorting without regressions.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100