the bookie heap used not balance after run a while and cause one bookie OOM.
- Dominant language
- Java
- Stars
- 15.3k
- Forks
- 3.8k
- Avg merge
- 1d 14h
- Merged PRs (30d)
- 160
Description
**Describe the bug**
We have four bookie node, all the parameter setting was the same, but the bookie heap used not balance after run a while, one bookie head memory used high and the gc time enlarge and frequencily , this can risk bookie OOM.
**version**: pulsar-all:2.9.1
PULSAR_GC: |
-XX:+UseG1GC -XX:MaxGCPauseMillis=20 -XX:+ParallelRefProcEnabled -XX:+UnlockExperimentalVMOptions -XX:+AggressiveOpts -XX:+DoEscapeAnalysis -XX:ParallelGCThreads=4 -XX:ConcGCThreads=4 -XX:G1NewSizePercent=30 -XX:G1MaxNewSizePercent=60 -XX:+DisableExplicitGC -XX:-ResizePLAB -XX:+ExitOnOutOfMemoryError -XX:+PerfDisableSharedMem -verbosegc
PULSAR_MEM: |
-Xms10g -Xmx10g -XX:MaxDirectMemorySize=7g
besides,we have 700,00 topics, and 40000 connections.
**To Reproduce**
out request is stable, pulsar run about two weeks the blance issue comes. and the issue comes again after two weeks when restart the bookie node.
**Expected behavior**
the bookie heap used balanced in all node.
**Screenshots**

**Desktop (please complete the following information):**
- OS: [e.g. iOS]
Ubuntu 20.04.4 LTS ( kubernetes pod)
**Additional context**
Add any other context about the problem here.
Contributor guide
Research direction
The report names no source files, tests, or entry points. Start by reviewing the two-week reproduction details, Bookie heap and GC settings, topic and connection counts, and the attached screenshot; done means explaining the imbalance and preventing the affected bookie from reaching OOM.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, kubernetes, ubuntu
- Domain
- distributed-systems, performance
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100