Payloads should be compressed [LUCENE-6764]
Open
legacy-jira-priority:Minor
type:enhancement
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
I think we should at least try to do something simple, eg. deduplicate or apply simple LZ77 compression. For instance if you use enclosing html tags to give different weights to individual terms, there might be lots of repetitions as there are not that many unique html tags.
---
Migrated from [LUCENE-6764](https://issues.apache.org/jira/browse/LUCENE-6764) by Adrien Grand (@jpountz), updated Sep 02 2015
Contributor guide
Research direction
The issue names no files, tests, or entry points. Start by locating Lucene's payload handling, then evaluate the proposed deduplication or simple LZ77 compression approaches. Done means a selected compression approach is implemented and its effect on payload storage and use is validated.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100