A newly created topic metrics do not survive after broker restarts
- Dominant language
- Java
- Stars
- 15.3k
- Forks
- 3.8k
- Avg merge
- 1d 22h
- Merged PRs (30d)
- 142
Description
**Describe the bug**
A newly created topic's Prometheus metrics does not survive broker restarts. The topic is being created by pulsar_admin with no traffic nor subscription. `pulsar_topics_count` is not accurate after the broker restarts.
**To Reproduce**
1. Create a persistent topic with pulsar-admin
2. Verify the topic's prometheus metrics exist under pulsar_in_bytes_total, pulsar_consumers_count, and a few others.
3. `pulsar_topics_count` is also incremented by 1
4. Restart the broker
5. After the broker has restarted, all metrics can be queried successfully.
6. However, the newly created topic's metrics are missing from the prometheus query. `pulsar_topics_count` is also reset to the number before the topic was created.
**Expected behavior**
The new created topic metrics survive the broker restarts.
**Screenshots**
n/a
**Desktop (please complete the following information):**
Tested on 2.7.2
**Additional context**
Add any other context about the problem here.
Contributor guide
Research direction
Start by reproducing the issue with pulsar-admin: create a persistent topic, query pulsar_in_bytes_total, pulsar_consumers_count, and pulsar_topics_count, then restart the broker and repeat the queries. Compare the metrics before and after the restart; done means the newly created topic remains represented and pulsar_topics_count stays incremented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- prometheus
- Domain
- distributed-systems, observability
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100