apache / apache/pulsar

msgBacklog, storageSize, backlogSize metrics keep increasing after stopping all consumers

Open
#16,606 7 comments 1 reaction 0 assignees View on GitHub
Stale type/bug
Dominant language
Java
Stars
15.3k
Forks
3.8k
Avg merge
1d 14h
Merged PRs (30d)
160

Description

**Describe the bug**
Anytime after I start a consumer for a topic and stop it when the backlog is >= 0, the msgBacklog, storageSize, backlogSize metrics continuously rise. Expected these metrics to remain constant when the consumers stop.

**To Reproduce**
Steps to reproduce the behavior:
1. Create topic to test
2. Check topic stats via `pulsar-admin topics partitioned-stats my_topic`
3. Run a consumer for `my_topic`, track the msgBacklog, storageSize, backlogSize metrics and then stop the consumer and publisher.
4. Check topic stats again. The msgBacklog, storageSize, backlogSize metrics should be increasing.

**Expected behavior**
Expecting metrics to stay constant when producer and consumer have been stopped. Even when only the consumer has existed in for that topic, the same unexpected behavior occurs

**Desktop (please complete the following information):**
- OS: MacOS 12.4

**Additional context**
Using Python 3.7.13, and Pulsar version 2.10.1(stable) standalone. Got the same behavior on previous versions on Pulsar

Contributor guide

Open the contributing guide

Research direction

Reproduce the issue with a standalone Pulsar 2.10.1 topic using `pulsar-admin topics partitioned-stats my_topic`, stopping the consumer and publisher, then observe the three metrics. Trace the metric updates after consumers stop; done means msgBacklog, storageSize, and backlogSize remain constant when no producer or consumer is active.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, python
Domain
distributed-systems, observability
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.