apache / apache/pulsar

python pulsar function delaying and missing messages when input is given from topic pattern

Open
#12,327 1 comment 0 reactions 0 assignees View on GitHub
lifecycle/stale type/bug
Dominant language
Java
Stars
15.3k
Forks
3.8k
Avg merge
1d 14h
Merged PRs (30d)
160

Description

**Describe the bug**
I am using topicpattern to invoke pulsar function. When a topic is created by creating producer and it immediately start publishing than some of the messages missed by pulsar function. It seems like pulsar function does not know that this new topic patterned topic is there and when It gets to know it starts publishing without missing further messages from this topic.

**To Reproduce**
Steps to reproduce the behavior:
1. create a python pulsar function which receive input from topic pattern
2. create producer of this topic pattern without explicitly creating topic
3. stars publishing to this producer
4. One would see that some messages would be missed by pulsar function and after some time it starts receiving message from this topic. After this pulsar function does not miss messages.

**Expected behavior**
Messages should not be missed by pulsar function.

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the reported sequence with a Python Pulsar function using a topic pattern, then create a producer for a new matching topic and publish immediately. Compare messages published with messages received after the function discovers the topic; done means no messages are missed during discovery.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.