[Bug] Topic policy initialization applies each policy layer separately, causing a transient inconsistent state and side effects from overridden policies
- Dominant language
- Java
- Stars
- 15.3k
- Forks
- 3.8k
- Avg merge
- 1d 14h
- Merged PRs (30d)
- 160
Description
### Search before reporting
- [x] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.
### Read release policy
- [x] I understand that unsupported versions don't get bug fixes. I will attempt to reproduce the issue on a supported version of Pulsar client and Pulsar broker.
### Version
master (and all currently supported versions).
### Issue Description
During topic initialization, the topic's effective configuration is built from several layers — broker
defaults, namespace policy, topic global policy, and topic local policy — where higher-priority layers override
lower ones. These layers are applied in **separate steps**, and each step both (a) updates the stored
hierarchical config values and (b) immediately triggers the corresponding actions / side effects based on the
current, still-partial effective state:
- the namespace policy is applied in `initialize()` (`updateTopicPolicyByNamespacePolicy(...)`);
- the topic global and local policies are applied afterwards, via the policy listener's `onUpdate(...)` (from
`AbstractTopic#initTopicPolicy()` → `TopicPolicyListenerWrapper#completeInitialization()`), which calls
`updateTopicPolicy(...)`.
Because these run one after another, the topic briefly runs with an intermediate, not-yet-final configuration
— for example the namespace policy is effective for a short window until the topic policy overrides it — and
the actions are triggered multiple times. This can:
- produce surprising transient behavior while a lower-priority layer is effective, and
- create **persistent, incorrect side effects** from a lower-priority policy before a higher-priority policy
overrides it. For example, if the namespace (or topic global) policy enables compaction and the topic-local
policy disables it, applying the lower-priority layer first can create a compaction subscription that the
local policy would have prevented; applying the local policy afterwards does not undo the subscription that
was already created.
(#26134 reordered the topic-local vs topic-global application — local before global — to reduce this hazard
between those two layers, but the broader multi-step application, including applying the namespace layer before
the topic layers, remains.)
### Error messages
None — there is no error. This is a behavioral/consistency issue during initialization.
### Reproducing the issue
The general pattern: pick a policy whose application has an action/side effect, set a lower-priority layer
(namespace or topic-global) to a value that triggers the action and a higher-priority layer (topic-local) to a
value that would suppress it, then load the topic and observe that the action is taken from the lower-priority
layer during initialization even though the final effective (higher-priority) value would have suppressed it.
Compaction is one concrete example: set the namespace/topic-global compaction threshold so compaction is
enabled and the topic-local policy so compaction is disabled; on load, a compaction subscription can be created
from the lower-priority layer even though the effective (local) policy disables compaction.
### Additional information
Suggested direction: separate the two responsibilities during initialization —
- **Loading phase:** apply all layers' values to the hierarchical config **without** triggering actions.
- **After all policies are loaded:** trigger the actions **once**, based on the final merged effective
configuration.
This makes the topic apply a single, consistent configuration at the end of initialization instead of a
sequence of partial ones, avoiding both transient inconsistency and side effects from overridden values.
Related:
- #26134 — hardens topic-policy initialization; reordered topic-local vs topic-global application to reduce the
ordering hazard between those two layers.
- #16144 — introduced `initTopicPolicy` for persistent topics.
### Are you willing to submit a PR?
- [x] I'm willing to submit a PR!
Contributor guide
Assessment
This issue has not been assessed yet.