Bookie Journal Queue Length is increasing gradually during performance test executions
- Dominant language
- Java
- Stars
- 15.3k
- Forks
- 3.8k
- Avg merge
- 1d 22h
- Merged PRs (30d)
- 142
Description
**Describe the bug**
Bookie Journal Queue Length is increasing gradually during performance executions
**To Reproduce**
Steps to reproduce the behavior:
1. Install Pulsar 2.5.0-08 on kubernetes cluster environment ( 3 Bookies, 3 brokers,3 proxy).
2. Create a topic with three partitions.
2. Run longevity test (48 hours) by passing 1 KB message size as input to send data to created topic in step 2
3. After completion of the few hours execution, Check size of "Journal Queue Length" bookie metric.
**Expected behavior**
Should not increase queue length of journal and bookies should be able to handle properly.
**Screenshots**
Attached screenshot of "Journal Queue Size Length" metric.
**Desktop (please complete the following information):**
- OS: CentOS 7.7
- Master Node: 1
- Worker Nodes : 3
**H/w Details**:
- Master node: 12x16x200 (Not running any Pulsar pods on master node)
- Worker nodes : 16x32x200 ( each worker node has additional 4 virtual disks for Pulsar pods)
Contributor guide
Research direction
No source files or tests are named. Start by reproducing the Pulsar 2.5.0-08 Kubernetes setup with three bookies, three brokers, and three proxies, then run the 48-hour longevity test while monitoring the Bookie Journal Queue Length metric. Done means the journal queue does not increase gradually and the bookies continue handling the workload.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, kubernetes
- Domain
- distributed-systems, observability-sre
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100