[Bug] Retry topic without topic ({subscription_name}-RETRY) is getting created [broker >=3.0.6, and clients’ version is <3.0.6], related to PIP-344 compatibility
- Dominant language
- Java
- Stars
- 15.3k
- Forks
- 3.8k
- Avg merge
- 1d 14h
- Merged PRs (30d)
- 160
Description
### Search before reporting
- [x] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.
### Read release policy
- [x] I understand that [unsupported versions](https://pulsar.apache.org/contribute/release-policy/#supported-versions) don't get bug fixes. I will attempt to reproduce the issue on a supported version of Pulsar client and Pulsar broker.
### User environment
Broker version - 2.11.0 (or <3.0.6)]
Client version - 4.0.4 (or >=3.0.6)
### Issue Description
- Context - We are migrating from 2.11.0 to 4. We connected a 4.0.4 client with a 2.11.0 server
- Expected - RETRY topics with topic name should've been created - {topic_name}-{sub_name}-RETRY (just like it was working with 2.11.0 server)
- What happened - RETRY topics without topic name got created - {sub_name}-RETRY, we use the service name as the subscription name. This makes all my unrelated consumer subscribers of the same RETRY topic under the same subscription.
- Bug - Backward compatibility for for non PIP-344 brokers is incomplete, merged [PR](https://github.com/apache/pulsar/pull/23136/files#diff-8761c864a30fab92d9116a3c965a9c383e8c527520116f605101a93f7830e083) that was meant to solve for compatibility
As per this PR, the implementation of the non-PIP-344 brokers will always create a new metadata in case the topic does not exist. The broker passes a value instead of the exception. Value contains `partition = 0`.
There is one more merged [PR](https://github.com/apache/pulsar/pull/22838/files#diff-1357b66ed34aa22cd5e680559dfcb645dce2876bdc0bda33c07f0170e7313441R171) which updates the logic of topic existence from `if partition > 0` to `if partition metadata exist`, when checking for the RETRY topic (without topic existence for compatibility). Non-PIP-344 broker (<3.0.6) returns metadata for the topic that does not exist. And the client (>=3.0.6) considers it as if the topic exists, and the topic without a topic name is used as the RETRY topic.
### Error messages
```
NA
```
### Reproducing the issue
Use client <3.0.6
### Additional information
Broker version >=3.0.6
Client version <3.0.6
Have multiple consumer-topic relationships with the same subscription name.
### Are you willing to submit a PR?
- [x] I'm willing to submit a PR!
Contributor guide
Research direction
Start by reviewing the compatibility logic in the two linked PRs, especially the retry-topic existence check and the metadata returned for non-PIP-344 brokers. Reproduce the case with a broker older than 3.0.6 and a client at or above 3.0.6, then verify that retry topics retain both the topic and subscription name and do not collide across unrelated consumer-topic relationships.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100