NumShards will cause broker can't load segments
- Dominant language
- Java
- Stars
- 14.1k
- Forks
- 3.8k
- Avg merge
- 2d 58m
- Merged PRs (30d)
- 233
Description
NumShards will cause broker can't load segments
### Affected Version
0.9.2/0.12.3
### Description
- We load batch data which set `NumShards` to 100
- The number of batch data is 10, and their cardinality is 10 too.
- So although the reduce number is 100, but finally the shards will only 10 ,and their shards number is not continuous.
- Broker will add them in `incompletePartitionsTimeline` when creating versionedIntervalTimeline.
- The query about this datasource will return `null`
Contributor guide
Research direction
Reproduce the case using batch data with NumShards set to 100, 10 batches, and cardinality 10. Trace how the broker adds the resulting non-contiguous shards to incompletePartitionsTimeline while creating versionedIntervalTimeline. Done means queries for the datasource return results instead of null.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100