Updating JSON index config
- Dominant language
- Java
- Stars
- 6.1k
- Forks
- 1.5k
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 195
Description
Let’s say I create a segment with the JSON index with default params and I decide afterwards that I want to exclude some fields.
At the moment there isn't a way to reload the segment because that code path checks if a JSON index exists on that column (which it does) and then skips over it.
(https://github.com/apache/pinot/blob/master/pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/loader/invertedindex/JsonIndexHandler.java#L89)
I am able to replace the JSON index if I first completely remove it from the table config, reload segments, then add it back to the table config, and reload all segments.
But it would be cool if we could do that in one step rather than two.
cc @Jackie-Jiang
Contributor guide
Research direction
Start in pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/loader/invertedindex/JsonIndexHandler.java at the referenced line, then trace the segment reload path and its table-config checks. Confirm how an existing JSON index is handled when its excluded fields change. Done means updated JSON index configuration can replace the existing index during one reload operation, without removing and re-adding it in the table config.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- databases
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100