apache / apache/pinot

Cluster Manager slow or unresponsible with high segment count tables

Open
#6,654 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
6.1k
Forks
1.5k
Avg merge
1d 21h
Merged PRs (30d)
189

Description

I am experiencing an issue where the Cluster Manager with a single table cluster hosting 27 000 segments.

UI is not broken but very slow and sometime even unresponsive. With enough patience, most administration task works.

A suggestion would be to lazy load segment and use a paging iterator rather than loading all information in the browser.

Contributor guide

Open the contributing guide

Research direction

Start at the Cluster Manager UI with a reproduction using a table containing 27,000 segments, then measure the slow administration views in the browser. Done means the UI remains responsive and administration tasks work without loading all segment information at once.

Written by the indexing model from the issue text.

Assessment

Domain
frontend, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.