apache / apache/pulsar-client-python

pattern_auto_discovery_period in pulsar.Client is not used.

未關閉
#171 1 則留言 2 個 reaction 已指派 0 人 在 GitHub 檢視
主要語言
Python
星號
75
分支
53
PR 合併指標
30 天內沒有已合併 PR

描述

This took me a little while to track down but im pretty sure its an actual bug. This seems simple enough to fix and I'd be willing to take on the PR to fix it. I've prototyped that change locally and its 1 line in the code and 1-2 lines in the unit test.

The `Client` class has a `subscribe` function. It has a parameter called `pattern_auto_discovery_period`

This value is unused and the value will always default to 60 seconds (because that is what is in the cpp files).

Here is a POC that shows the problem

```
import pulsar
import multiprocessing as M
import time
import re
import _pulsar

mode = "non-persistent" # ?? nothing received

TOPIC = f"{mode}://public/default/my-topic/whatever"
TOPIC2 = f"{mode}://public/default/my-topic/something"
TOPIC_WILD = f"{mode}://public/default/my-topic/.*"

def callback( consumer , msg ):
print(f"Got topic {msg.topic_name()} {msg.data()}\n", flush=True)
def main():

sub_mode = _pulsar.RegexSubscriptionMode.NonPersistentOnly

time.sleep( 2 )
client = pulsar.Client('pulsar://localhost:6650')
# This topic will be found right away becuase i created it first!
producer = client.create_producer(TOPIC)

client.subscribe( re.compile(TOPIC_WILD) , subscription_name="subname111",message_listener=callback , regex_subscription_mode=sub_mode , pattern_auto_discovery_period=100 )
# This topic will not be found until 60 seconds into the run regardless of pattern_auto_discovery_period
producer2 = client.create_producer( TOPIC2 )

for k in range(0,120):
time.sleep(1 )
o = f"some data {k}"
if k % 2 == 0:
producer.send( o.encode() )
else:
producer2.send( o.encode() )
client.close()
if __name__ == "__main__":
main( )
```
Here is a screenshot of the pattern discovery happening at 60 seconds.

![image](https://github.com/apache/pulsar-client-python/assets/28128048/81bb2260-7c04-40fe-8b7e-d9e995b6cd96)

There is a unit test that is supposed to be testing for this: [test_topics_pattern_consumer](https://github.com/apache/pulsar-client-python/blob/main/tests/pulsar_test.py#L1153)

However, this test has a bug as well. Since all the producers are made before the subscribe call the topics are all found right away.

```

producer1 = client.create_producer(topic1)
producer2 = client.create_producer(topic2)
producer3 = client.create_producer(topic3) ##### MOVE THIS after the subscribe to demonstrate auto-recovery!

consumer = client.subscribe(
re.compile(topics_pattern),
"my-pattern-consumer-sub",
consumer_type=ConsumerType.Shared,
receiver_queue_size=10,
pattern_auto_discovery_period=1,
)

# wait enough time to trigger auto discovery
time.sleep(2)

```

貢獻指南

開啟貢獻指南

研究方向

從 Python Client.subscribe 入口開始,閱讀 tests/pulsar_test.py 中約第 1153 行的 test_topics_pattern_consumer。在 subscribe 後建立第三個 producer 後執行測試,並驗證 pattern consumer 是根據 pattern_auto_discovery_period,而不是固定的 60 秒間隔來發現它。

由索引模型根據 Issue 內容生成。

評估

技術堆疊
python
領域
api
Issue 類型
缺陷
難度
2/5
預估耗時
1-3 小時
活躍度
停滯
描述清晰度
描述清楚
新手友好度
48/100

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。