apache / apache/pulsar

broker-client-shared-internal-executor doing blocking task

Open
#26,037 1 comment 1 reaction 1 assignee Claimed by @lhotari View on GitHub
type/enhancement
Dominant language
Java
Stars
15.3k
Forks
3.8k
Avg merge
1d 14h
Merged PRs (30d)
160

Description

### Search before reporting

- [x] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.

### Motivation

I see that broker-client-shared-internal-executor thread is used to load system topic policies in SystemTopicBasedTopicPoliciesService.java while doing initPolicesCache. After loading the topic policies, same thread is used to call listener.onUpdate ( to update the topic policy). But internally listener.onUpdate operation is a blocking operation. onUpdate(PersistentTopic) ->
applyUpdatedTopicPolicies ->
checkMessageExpiry->
sub.expireMessages->
getNumberOfEntriesInBacklog->
topic.isOldestMessageExpired->
cursor.getNthEntry->
counter.await(ledger.getConfig().getMetadataOperationsTimeoutSeconds(), TimeUnit.SECONDS);
Here the default operation timeout is 1minute.
That means, if there is a single topic for which the oldest ledger is not accessible for the time being, it will block loading all other topic for a significant time, depending on the number of subscription present on the topic, eg. if there are 20 subscription on the topic , then policies loading of other system-topics can be delayed by 20mins(This happens due to recursive loading of policies event, eg. an ns with 20 topics, has to complete 20 callbacks one after other, to load the topic policy)

### Solution

_No response_

### Alternatives

_No response_

### Anything else?

_No response_

### Are you willing to submit a PR?

- [ ] I'm willing to submit a PR!

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.