apache / apache/pinot

Close WorkerManager routing gaps for optional/unavailable segments and empty partitions

Open
#18,223 0 comments 0 reactions 0 assignees View on GitHub
bug java multi-stage priority: medium query
Dominant language
Java
Stars
6.1k
Forks
1.5k
Avg merge
2d 3h
Merged PRs (30d)
195

Description

### Motivation
`pinot-query-planner/src/main/java/org/apache/pinot/query/routing/WorkerManager.java` still has TODOs for several unsupported routing cases in the multi-stage engine:
- optional segments
- unavailable segments
- partitions with no segments
- workers whose assigned partitions all have no segments

These are correctness and resiliency gaps in stage assignment, especially for hybrid or partially unavailable tables.

### Scope
- Define expected routing behavior for each unsupported case.
- Add planner and runtime tests for replicated and partitioned leaf stages.
- Implement the missing routing and metadata handling, or fail with clearer behavior if a case must stay unsupported.

### Notes
Observed on `upstream/master` on April 15, 2026.

Contributor guide

Open the contributing guide

Research direction

Start with pinot-query-planner/src/main/java/org/apache/pinot/query/routing/WorkerManager.java and inspect the TODOs for optional and unavailable segments, empty partitions, and workers with no assigned segments. Define the expected behavior for each case before adding planner and runtime tests for replicated and partitioned leaf stages. Done means the routing and metadata behavior is implemented or unsupported cases fail clearly.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
distributed-systems
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.