apache / apache/pinot

[multistage] [bug] Sort operator issue when # of pinot servers is greater 1

Open
#9,717 1 comment 0 reactions 0 assignees View on GitHub
multi-stage
Dominant language
Java
Stars
6.1k
Forks
1.5k
Avg merge
2d 55m
Merged PRs (30d)
182

Description

StagePlan is sent to all available intermediate stage servers. When # of intermediate stage servers is greater than 1, only first server will receive leaf server data.

The other servers will spin operator waiting on mailbox.

Correctness: if any of first server returns first, we will just return error.

Wasted CPU: All servers except 1, will spin CPU on waiting something they don’t need. With non-blocking fix(https://github.com/apache/pinot/issues/9615#issuecomment-1282911839), this will become a memory “leak” problem.

HotSpot: First server gets way more load than the other server

Should be related to https://github.com/apache/pinot/issues/9611

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.