apache / apache/lucene

Would `DirectMonotonicWriter` give a wee bit better compression if we rounded instead of truncated the `expected` value?

Open
#12,379 0 comments 0 reactions 0 assignees View on GitHub
type:enhancement
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

### Description

Spinoff from https://github.com/apache/lucene/pull/12377.

I noticed this loop in `DirectMonotonicWriter` that is computing the error signal of a simple "linear fit" model, for better compression of monotonically increasing `long` values:

```
final float avgInc =
(float) ((double) (buffer[bufferSize - 1] - buffer[0]) / Math.max(1, bufferSize - 1));
for (int i = 0; i < bufferSize; ++i) {
final long expected = (long) (avgInc * (long) i);
buffer[i] -= expected;
}
```

We are truncating the predicted value to the nearest `long`, but wouldn't it maybe be a bit better compression if we `Math.round`'d instead? The abs(error) might sometimes be lower by 1? I'm not sure, it's hard to think about!

The problem / challenge is backwards compatibility ...

Contributor guide

Open the contributing guide

Research direction

Start by reading the DirectMonotonicWriter loop described in the issue and the backwards-compatibility context from the linked pull request. Determine whether rounding the expected value improves compression without breaking compatibility; the issue does not name a test or define a finalized implementation target.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.