deduplication is not working as expected
- Dominant language
- Java
- Stars
- 15.3k
- Forks
- 3.8k
- Avg merge
- 1d 14h
- Merged PRs (30d)
- 160
Description
**Describe the bug**
A clear and concise description of what the bug is.
**To Reproduce**
Steps to reproduce the behavior:
1. Create pulsar cluster with version 2.7.1
2. enabled topic level policy at broker config. https://pulsar.apache.org/docs/en/pulsar-admin/#topics-1
3. created a topic and set the deduplicationEnabled to true
4. Created a pulsar producer with a static name and set the timeout to 0 second https://pulsar.apache.org/docs/en/cookbooks-deduplication/
5. produce message with same sequenceId and consume
6. But still I am able to produce and consume multiple message with same sequence id.
**Expected behavior**
Broker must not persist two message with same service id
**Screenshots**
NA
**Desktop (please complete the following information):**
Deployed pulsar on AKS
**Additional context**
How can we handle producer timeout if producer always must have the timeout set to 0(means effective no timeout)
Contributor guide
Research direction
Start with the reproduction steps in the issue and the linked Pulsar topic-policy and deduplication documentation, using Pulsar 2.7.1 and the described producer settings. Investigate why messages with the same sequence ID are both persisted and consumed, and clarify the expected behavior when the producer timeout is zero. Done means duplicate messages are not persisted or consumed under the stated configuration.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- kubernetes
- Domain
- distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100