apache / apache/pinot

Heavy loaded server not serving queries after start

Open
#7,421 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
6.1k
Forks
1.5k
Avg merge
1d 21h
Merged PRs (30d)
189

Description

For a server with a lot of segments for Upsert tables, the server start process may result in high GC activity and its zk client may be disconnected temporarily. In such case, the server may not register itself to serve query traffic because the following zk config writes fails -- note that the other server start process still works fine. So a solution here is to at least retry the config write instead of giving it up after one attempt.

https://github.com/apache/pinot/blob/7e9ca6a5a4afe0d4e283ac1307c45430e474cbf2/pinot-server/src/main/java/org/apache/pinot/server/starter/helix/BaseServerStarter.java#L468

Contributor guide

Open the contributing guide

Research direction

Start in pinot-server/src/main/java/org/apache/pinot/server/starter/helix/BaseServerStarter.java around line 468 and trace the ZooKeeper configuration write during server startup. Confirm the failure path when the client is temporarily disconnected; done means the configuration write is retried so the server can register to serve query traffic.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
distributed-systems
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.