apache / apache/pulsar

Seek fails with partitioned topics and SubscriptionInitialPosition.Earliest

Open
#10,630 2 comments 0 reactions 0 assignees View on GitHub
lifecycle/stale type/bug
Dominant language
Java
Stars
15.3k
Forks
3.8k
Avg merge
1d 14h
Merged PRs (30d)
160

Description

**Describe the bug**
When using `seek(timestamp)` on `Consumer` with partitioned topics and `SubscriptionInitialPosition.Earliest` consumer fails to received expected messages. All messages from topic are received plus expected ones (repeated).

This behaviour does not happen with `SubscriptionInitialPosition.Latest`.

Tested with Pulsar 2.7.1.

**To Reproduce**
I have created following project with a test suite to reproduce the problem: https://github.com/fmiguelez/pulsar-issue-10630-test

Output from tests:
```
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR] PartitionedTopicSeekIT.testEarliest:57->testSeek:98 expected: <[A1, B1, C1, D1]> but was: <[A1, A0, B0, B1, C0, C1, D0, D1, A1, B1, C1, D1]>
[INFO]
[ERROR] Tests run: 2, Failures: 1, Errors: 0, Skipped: 0
```

**Expected behavior**
All tests in test suite must pass

**Screenshots**

**Desktop (please complete the following information):**
- OS: Tested locally with Docker Desktop for Windows

**Additional context**
I would like to add that we have tests for other variants using `seek(timestamp)` and this is the only case where it fails. Tested combinations:

| interface | start position | partitioned | result |
|--------------|----------------|-------------|--------|
|`Reader` | `MessageId.latest`| no | ✔ |
|`Reader` | `MessageId.earliest`| no | ✔ |
|`Reader` | `MessageId.latest`| yes | ✔ |
|`Reader` | `MessageId.earliest`| yes | ✔ |
|`Consumer` | `SubscriptionInitialPosition.Latest`| no | ✔ |
|`Consumer` | `SubscriptionInitialPosition.Earliest`| no | ✔ |
|`Consumer` | `SubscriptionInitialPosition.Latest`| yes | ✔ |
|`Consumer` | `SubscriptionInitialPosition.Earliest`| yes | ❌ |

Contributor guide

Open the contributing guide

Research direction

Start by running the linked reproduction project and inspect PartitionedTopicSeekIT.testEarliest and testSeek, which demonstrate the duplicate messages with Consumer, partitioned topics, SubscriptionInitialPosition.Earliest, and seek(timestamp). Trace the Consumer seek path for partitioned topics and compare it with the passing Latest case. Done means the failing test passes and the expected messages are received without the earlier partition messages repeating.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.