apache / apache/lucene

Investigate points disk usage reduction on NYC Taxis between 2020-11-01 and 2021-09-24 [LUCENE-10124]

Open
#11,161 1 comment 0 reactions 0 assignees View on GitHub
legacy-jira-priority:Minor type:enhancement
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

There's been a 35% ⚠ reduction of disk usage for points on the NYC Taxis benchmark some time between 2020-11-01 and 2021-09-24 when I make disk usage for points work again. I can't think of a change that would have decreased disk usage by so much.

https://home.apache.org/\~mikemccand/lucenebench/sparseResults.html#index_size_parts

---
Migrated from [LUCENE-10124](https://issues.apache.org/jira/browse/LUCENE-10124) by Adrien Grand (@jpountz), updated Sep 29 2021

Contributor guide

Open the contributing guide

Research direction

Start with the NYC Taxis benchmark results at sparseResults.html#index_size_parts and compare the points disk-usage data between 2020-11-01 and 2021-09-24. Trace the Lucene changes in that interval that could affect points storage, then document the cause of the reported 35% reduction and confirm it against the benchmark.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.