apache / apache/pulsar

[Bug] Retry topic without topic ({subscription_name}-RETRY) is getting created [broker >=3.0.6, and clients’ version is <3.0.6], related to PIP-344 compatibility

Open
#24,369 1 comment 0 reactions 0 assignees View on GitHub
type/bug
Dominant language
Java
Stars
15.3k
Forks
3.8k
Avg merge
1d 14h
Merged PRs (30d)
160

Description

### Search before reporting

- [x] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.

### Read release policy

- [x] I understand that [unsupported versions](https://pulsar.apache.org/contribute/release-policy/#supported-versions) don't get bug fixes. I will attempt to reproduce the issue on a supported version of Pulsar client and Pulsar broker.

### User environment

Broker version - 2.11.0 (or <3.0.6)]
Client version - 4.0.4 (or >=3.0.6)

### Issue Description

- Context - We are migrating from 2.11.0 to 4. We connected a 4.0.4 client with a 2.11.0 server
- Expected - RETRY topics with topic name should've been created - {topic_name}-{sub_name}-RETRY (just like it was working with 2.11.0 server)
- What happened - RETRY topics without topic name got created - {sub_name}-RETRY, we use the service name as the subscription name. This makes all my unrelated consumer subscribers of the same RETRY topic under the same subscription.
- Bug - Backward compatibility for for non PIP-344 brokers is incomplete, merged [PR](https://github.com/apache/pulsar/pull/23136/files#diff-8761c864a30fab92d9116a3c965a9c383e8c527520116f605101a93f7830e083) that was meant to solve for compatibility
As per this PR, the implementation of the non-PIP-344 brokers will always create a new metadata in case the topic does not exist. The broker passes a value instead of the exception. Value contains `partition = 0`.
There is one more merged [PR](https://github.com/apache/pulsar/pull/22838/files#diff-1357b66ed34aa22cd5e680559dfcb645dce2876bdc0bda33c07f0170e7313441R171) which updates the logic of topic existence from `if partition > 0` to `if partition metadata exist`, when checking for the RETRY topic (without topic existence for compatibility). Non-PIP-344 broker (<3.0.6) returns metadata for the topic that does not exist. And the client (>=3.0.6) considers it as if the topic exists, and the topic without a topic name is used as the RETRY topic.

### Error messages

```
NA
```

### Reproducing the issue

Use client <3.0.6

### Additional information

Broker version >=3.0.6
Client version <3.0.6
Have multiple consumer-topic relationships with the same subscription name.

### Are you willing to submit a PR?

- [x] I'm willing to submit a PR!

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the compatibility logic in the two linked PRs, especially the retry-topic existence check and the metadata returned for non-PIP-344 brokers. Reproduce the case with a broker older than 3.0.6 and a client at or above 3.0.6, then verify that retry topics retain both the topic and subscription name and do not collide across unrelated consumer-topic relationships.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.