LatLonPoint quantization should use same rounding mode as GeoPointField [LUCENE-7160]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
Both of these fields quantize doubles in a lossy way, but GeoPoint rounds towards negative infinity (because all values get shifted to positive space first), whereas LatLonPoint currently rounds towards zero.
At the same time they share the same basic logic for e.g. tree traversal and same basic tests: so it creates confusion that they round differently.
Especially since e.g. LatLonPoint does some operations in binary/integer space (some of these optimizations such as its box query are currently buggy for edge cases, and tests are lenient as TODO, another issue, may or may not impact GeoPoint, it at least tries harder).
I think LatLonPoint should round down to be more like GeoPoint here? By the way, doing this also gives more consistency, it reduces the difference between the two implementations in e.g. openstreetmaps searching (distance query benchmark now differs by only 4 hits...)
This quantization makes things complicated to reason about. I understand the advantages it brings but I think along with that comes the responsibility of us handling edge cases better. I don't think we do well at this today...
---
Migrated from [LUCENE-7160](https://issues.apache.org/jira/browse/LUCENE-7160) by Robert Muir (@rmuir)
Attachments: [LUCENE-7160.patch](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-7160/LUCENE-7160.patch)
Contributor guide
Research direction
Start by reading the quantization logic in LatLonPoint and comparing it with GeoPointField, especially how each rounds shifted coordinate values. Review the existing shared traversal and quantization tests, then use the LUCENE-7160.patch as historical context. Done means LatLonPoint uses the intended downward rounding consistently and its edge cases are covered by tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100