subscribe to partitioned topic(s) error while non-partitioned topics ok through re.Pattern topics
- Dominant language
- Java
- Stars
- 15.3k
- Forks
- 3.8k
- Avg merge
- 1d 14h
- Merged PRs (30d)
- 160
Description
**Describe the bug**
I tested on subscribing to multiple topics ( partitioned topics error, but non-partitioned topics ok)
**To Reproduce**
Steps to reproduce the behavior:
1. Using pulsar-admin to create one partitioned topic with one partition
2. Using one python consumer to subscribe.
e.g. TOPICS = re.compile(f"persistent://{MQ_TENANT}/{MQ_NAMESPACE}/.*")
3. Error with occur
4. In step 2, if you set your `TOPICS` to subscribe multiple non-partitioned topics seems OK.
**Expected behavior**
By using re.Pattern topics to subscribe should also be OK, just like the non-partitioned topics behavior.
**Screenshots**
```
│ 2021-09-09 23:24:59.043 INFO [140028596160320] ConnectionPool:84 | Created connection for pulsar+ssl://pulsar-proxy.mq:6651
│ 2021-09-09 23:24:59.046 INFO [140028147660544] ClientConnection:372 | [10.0.80.42:47898 -> 172.20.146.245:6651] Connected to broker
│ 2021-09-09 23:24:59.105 INFO [140028147660544] HandlerBase:55 | [persistent://dev/spapi/product_listing_offers-partition-0, spapi.consumer.to_datalake, 0] Getting connection from pool
│r2021-09-09 23:24:59.107 INFO [140028147660544] ConnectionPool:84 | Created connection for pulsar+ssl://pulsar-broker-0.pulsar-broker.mq.svc.cluster.local:6651
│ 2021-09-09 23:24:59.109 INFO [140028147660544] ClientConnection:374 | [10.0.80.42:47902 -> 172.20.146.245:6651] Connected to broker through proxy. Logical broker: pulsar+ssl://pulsar-broker-0.pulsar-broker.mq.svc.cluster.local:6651
│ 2021-09-09 23:24:59.161 WARN [140028147660544] ClientConnection:980 | [10.0.80.42:47902 -> 172.20.146.245:6651] Received error response from server: BrokerPersistenceError (org.apache.bookkeeper.mledger.ManagedLedgerException: Not enough non-faulty bookies available) -- req_id: 0
│ 2021-09-09 23:24:59.161 ERROR [140028147660544] ConsumerImpl:263 | [persistent://dev/spapi/product_listing_offers-partition-0, spapi.consumer.to_datalake, 0] Failed to create consumer: BrokerPersistenceError
│ 2021-09-09 23:24:59.161 ERROR [140028147660544] MultiTopicsConsumerImpl:91 | Failed when subscribed to topic persistent://dev/spapi/product_listing_offers in TopicsConsumer. Error - BrokerPersistenceError
│ 2021-09-09 23:24:59.161 ERROR [140028147660544] MultiTopicsConsumerImpl:101 | Unable to create Consumer - [Muti Topics Consumer: TopicName - persistent://dev/spapi/.* - Subscription - spapi.consumer.to_datalake] Error - BrokerPersistenceError
2021-09-09 23:24:59.161 ERROR [140028147660544] MultiTopicsConsumerImpl:412 | Closing the consumer failed for partition - persistent://dev/spapi/product_listing_offers-partition-0 with error - AlreadyClosed
```
**Desktop (please complete the following information):**
- Base Docker Image: python:3.8.12-slim-buster (Alpine 9.2.0)
**Additional context**
Add any other context about the problem here.
Contributor guide
Research direction
Reproduce the report by creating a one-partition topic with pulsar-admin and subscribing from a Python consumer using the shown re.Pattern expression. Compare the partitioned-topic failure with subscriptions to multiple non-partitioned topics, and inspect the subscription entry point behind this pattern-based consumer flow. Done means partitioned and non-partitioned topics both subscribe successfully without the reported BrokerPersistenceError.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100