Sorting on FloatFiled or FloatPoint producing erroneous results [LUCENE-10515]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
When we use FloatField for sorting, decimal values are getting rounded off and results is erroneous. Even after upgrading version to 6.0.0 FloatPoint results are also erroneous. We are getting correct results only when we use field as FloatDocValues.
I have attached a sample project which can show case the bug which i encountered [LuenceSortFloatIssue.zip](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-10515/LuenceSortFloatIssue.zip)
I debugged to see what happened internally, below are few things which i encountered.
I used the following values to sort 12.5,15,0,0,12. We can clearly see 12.5 and 12 are saved as 1.7E-44 in queue.

when we use `FloatDocValuesField` to store the value then the sorting is working as expected. we can see values are stored without rounding off in below image

Only difference i see in the file Lucene50DocValuesProducer . In case of floatPoint numeric entry format is CONST_COMPRESSED and for rest of field types like int point, long point or FloatDocValuesField the entry format is TABLE_COMPRESSED.


---
Migrated from [LUCENE-10515](https://issues.apache.org/jira/browse/LUCENE-10515) by SAILENDRA PAVAN
Attachments: [FloatDocValuesStored.png](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-10515/FloatDocValuesStored.png), [FloatFieldStored.png](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-10515/FloatFieldStored.png), [Lucene50DocValuesProducer.png](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-10515/Lucene50DocValuesProducer.png), [LuenceSortFloatIssue.zip](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-10515/LuenceSortFloatIssue.zip), [screenshot-1.png](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-10515/screenshot-1.png)
Contributor guide
Research direction
Start by reproducing the sorting behavior with the attached sample project and the values 12.5, 15, 0, 0, and 12. Then inspect Lucene50DocValuesProducer and the FloatField, FloatPoint, and FloatDocValuesField paths, comparing their numeric entry formats. Done means decimal sorting for FloatField and FloatPoint preserves the expected ordering instead of rounding values.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- search
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100