apache / apache/pulsar

[Bug] update partition process is not reasonable since branch-3.0

Open
#25,041 6 comments 0 reactions 1 assignee Claimed by @mattisonchao View on GitHub
type/bug
Dominant language
Java
Stars
15.3k
Forks
3.8k
Avg merge
1d 14h
Merged PRs (30d)
160

Description

### Search before reporting

- [x] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.

### Read release policy

- [x] I understand that [unsupported versions](https://pulsar.apache.org/contribute/release-policy/#supported-versions) don't get bug fixes. I will attempt to reproduce the issue on a supported version of Pulsar client and Pulsar broker.

### User environment

pulsar version 3.0.x

### Issue Description

step 1: admin update topic partition from 50 to 150, throw error
step 2: see topic partition metadata has been updated to 150, and producer try to produce msg to new partition
step 3: there are 4 partition keep throw exception "topic does not exist", and other new partition can succeed.

The problem is in this pr, https://github.com/apache/pulsar/pull/19166
before pr, update partition process is:
1. create missing partitions
2. create subscription for new partition
3. update topic metadata

after pr, it become:
1. update topic metadata
2. create missing partitions
3. create subscription for new partition

That is not reasonable, pulsar producer and consumer rely on topic partition metadata to know whether the topic partition has been updated, if metadata updated, client can add new producer or new consumer to the new partition.

However, with the pr's change, although update partition is not complete, client regard it as success and try to construct new connection to the new topic partition. That is wrong.

There is another high risk situation, if update topic metadata and create missing partitions succeed, but fail on create subscription. In this moment, producer can send new msg to new partition, but after a while consumer may auto create subscription on new partition, and the read position is latest. That would result in consumer lost some message. It is very dangerous.

I don't understand why previous pr need to change the process, while it only illustrate clean the duplicate code

### Error messages

```text

```

### Reproducing the issue

update partition, fail on step 2 or 3

1. update topic metadata
2. create missing partitions
3. create subscription for new partition

### Additional information

_No response_

### Are you willing to submit a PR?

- [x] I'm willing to submit a PR!

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.