apache / apache/lucene

Field class should not let you analyze int values? [LUCENE-8108]

Open
#9,156 2 comments 0 reactions 0 assignees View on GitHub
legacy-jira-priority:Major type:bug
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

I stumbled on this by accident, by creating a `Field` instance with a `Integer` value for its `fieldsData` and then setting `tokenized = true` in its `FieldType`.

If you do this then Lucene silently converts the int to a string and then tokenizes it, e.g. applying synonyms, etc., if that's what your analysis chain does.

---
Migrated from [LUCENE-8108](https://issues.apache.org/jira/browse/LUCENE-8108) by Michael McCandless (@mikemccand), updated Dec 28 2017
Attachments: [LUCENE-8108.patch](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-8108/LUCENE-8108.patch)

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the issue with a Field whose fieldsData is an Integer and a FieldType with tokenized=true. Trace the Field analysis path to determine where the integer becomes a string, then verify that integer values are no longer silently analyzed, including behavior involving synonyms.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
search
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.