broker-client-shared-internal-executor doing blocking task
- Dominant language
- Java
- Stars
- 15.3k
- Forks
- 3.8k
- Avg merge
- 1d 14h
- Merged PRs (30d)
- 160
Description
### Search before reporting
- [x] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.
### Motivation
I see that broker-client-shared-internal-executor thread is used to load system topic policies in SystemTopicBasedTopicPoliciesService.java while doing initPolicesCache. After loading the topic policies, same thread is used to call listener.onUpdate ( to update the topic policy). But internally listener.onUpdate operation is a blocking operation. onUpdate(PersistentTopic) ->
applyUpdatedTopicPolicies ->
checkMessageExpiry->
sub.expireMessages->
getNumberOfEntriesInBacklog->
topic.isOldestMessageExpired->
cursor.getNthEntry->
counter.await(ledger.getConfig().getMetadataOperationsTimeoutSeconds(), TimeUnit.SECONDS);
Here the default operation timeout is 1minute.
That means, if there is a single topic for which the oldest ledger is not accessible for the time being, it will block loading all other topic for a significant time, depending on the number of subscription present on the topic, eg. if there are 20 subscription on the topic , then policies loading of other system-topics can be delayed by 20mins(This happens due to recursive loading of policies event, eg. an ns with 20 topics, has to complete 20 callbacks one after other, to load the topic policy)
### Solution
_No response_
### Alternatives
_No response_
### Anything else?
_No response_
### Are you willing to submit a PR?
- [ ] I'm willing to submit a PR!
Contributor guide
Assessment
This issue has not been assessed yet.