Explore delta of delta encoding [LUCENE-7806]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
From http://search-lucene.com/m/Lucene/l6pAi1YEfXhuGGIl1?subj=Re+Delta+of+delta+encoding
> delta of delta encoding is one of the Facebook Gorilla tricks that allows it to compress 16 bytes into 1\.37 bytes on average – see section 4\.1 that describes it – http://www\.vldb\.org/pvldb/vol8/p1816-teller\.pdf
>
> This seems to be aimed at both time fields and numerical values\.
>
> https://github\.com/burmanm/gorilla-tsc is a recent Java impl
CC @jpountz
---
Migrated from [LUCENE-7806](https://issues.apache.org/jira/browse/LUCENE-7806) by Otis Gospodnetic (@otisg), updated Jul 22 2021
Linked issues:
- #11072
Contributor guide
Research direction
Start by reading the linked Gorilla paper, especially section 4.1, and the referenced Java implementation at github.com/burmanm/gorilla-tsc. The issue proposes exploring delta-of-delta encoding for time fields and numerical values, but names no Lucene files or tests and does not define a completion criterion.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100