apache / apache/pulsar

[Tiered Storage] Create offloader for every namespace

Open
#11,177 1 comment 0 reactions 1 assignee Claimed by @gaoran10 View on GitHub
lifecycle/stale type/bug
Dominant language
Java
Stars
15.3k
Forks
3.8k
Avg merge
1d 22h
Merged PRs (30d)
142

Description

**Describe the bug**

Currently, Pulsar will create a new ledger offloader for every namespace even if not setting offload policies for the namespace, create a ledger offloader is an expensive operation, if there are many namespaces, this will cause create topic operation timeout.

**error logs**
```
Caused by: java.util.concurrent.TimeoutException: Future didn't finish within deadline
at org.apache.pulsar.broker.service.BrokerService.futureWithDeadline(BrokerService.java:938) ~[io.streamnative-pulsar-broker-2.7.2.4.jar:2.7.2.4]
at org.apache.pulsar.broker.service.BrokerService.getManagedLedgerConfig(BrokerService.java:1165) ~[io.streamnative-pulsar-broker-2.7.2.4.jar:2.7.2.4]
at org.apache.pulsar.broker.service.BrokerService.createPersistentTopic(BrokerService.java:1099) ~[io.streamnative-pulsar-broker-2.7.2.4.jar:2.7.2.4]
```

**Expected behavior**

We could reuse the ledger offloader that has the same offload policies.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.