apache / apache/pulsar

Inactive topic cleaning up does not work properly

Open
#15,881 7 comments 0 reactions 0 assignees View on GitHub
Stale type/bug
Dominant language
Java
Stars
15.3k
Forks
3.8k
Avg merge
1d 14h
Merged PRs (30d)
160

Description

**Describe the bug**
I'm using Pulsar 2.10.0 with `brokerDeleteInactivePartitionedTopicMetadataEnabled=true` and `allowAutoTopicCreationType=partitioned`. We disabled the `brokerDeleteInactiveTopicsEnabled` at broker level and enabled it in namespace level with this inactive topic policy: `{"inactiveTopicDeleteMode": "delete_when_subscriptions_caught_up", "maxInactiveDurationSeconds": 3600, "deleteWhileInactive": true}`.

We also have some functions keep running all the time with regex subscriptions.

Full settings:

```
allowAutoTopicCreationType=partitioned
brokerDeleteInactivePartitionedTopicMetadataEnabled=true
brokerDeleteInactiveTopicsEnabled=false
clusterName=mycluster
configurationMetadataStoreUrl=zookeeper:2181
forceDeleteNamespaceAllowed=true
forceDeleteTenantAllowed=true
functionsWorkerEnabled=false
managedLedgerDefaultAckQuorum=2
managedLedgerDefaultEnsembleSize=3
managedLedgerDefaultWriteQuorum=3
metadataStoreUrl=zookeeper:2181
numHttpServerThreads=32
numIOThreads=8
systemTopicEnabled=true
topicLevelPoliciesEnabled=true
```

**To Reproduce**
Steps to reproduce the behavior:
1. Deploy a pulsar 2.10.0 with these settings above:
2. Produce some messages to random topic (pulsar will auto create these new partitioned topics)
3. Wait for few hours later
4. These inactive topics are still there

**Expected behavior**
The inactive topics are deleted properly

**Desktop (please complete the following information):**
- OS: Linux

**Additional question**
Is it possible to force the inactive topic cleaner to run immediately via REST api or admin cli? This feature is risky because it can cause data lost so I think it is better if we can provide enough tools to verify.

Contributor guide

Open the contributing guide

Research direction

Use the listed broker and namespace settings to reproduce the inactive-topic cleanup failure with auto-created partitioned topics. Start by checking the inactive-topic cleaner and the REST API/admin CLI entry points mentioned, including whether an immediate run can be verified. Done means inactive topics are deleted according to the policy and the behavior is testable without risking unintended data loss.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.