Close WorkerManager routing gaps for optional/unavailable segments and empty partitions
- Dominant language
- Java
- Stars
- 6.1k
- Forks
- 1.5k
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 195
Description
### Motivation
`pinot-query-planner/src/main/java/org/apache/pinot/query/routing/WorkerManager.java` still has TODOs for several unsupported routing cases in the multi-stage engine:
- optional segments
- unavailable segments
- partitions with no segments
- workers whose assigned partitions all have no segments
These are correctness and resiliency gaps in stage assignment, especially for hybrid or partially unavailable tables.
### Scope
- Define expected routing behavior for each unsupported case.
- Add planner and runtime tests for replicated and partitioned leaf stages.
- Implement the missing routing and metadata handling, or fail with clearer behavior if a case must stay unsupported.
### Notes
Observed on `upstream/master` on April 15, 2026.
Contributor guide
Research direction
Start with pinot-query-planner/src/main/java/org/apache/pinot/query/routing/WorkerManager.java and inspect the TODOs for optional and unavailable segments, empty partitions, and workers with no assigned segments. Define the expected behavior for each case before adding planner and runtime tests for replicated and partitioned leaf stages. Done means the routing and metadata behavior is implemented or unsupported cases fail clearly.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- distributed-systems
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100