apache / apache/pulsar-client-python
has_message_available() with timestamp seek returns True on first call when reaching end of the topic
- Lenguaje dominante
- Python
- Estrellas
- 75
- Forks
- 53
- Métricas de merge de PR
- Sin PR fusionados en 30 d
Descripción
Ran into this while trying to upgrade to the v3.8.0 Python client. Tested on a Pulsar v4.0.4 cluster, which is after [this issue](https://github.com/apache/pulsar/pull/23502) was fixed in Pulsar v4.0.1.
Possibly related to [apache/pulsar #23917](https://github.com/apache/pulsar/issues/23917)?
---
It seems that if you:
- Have a reader
- Seek by timestamp to a point where there's messages after it
- `read_next()` until there's no more messages
Then the next `has_message_available()` call returns `True`, even though there's no more messages afterwards. Additionally, calling `has_message_available()` again correctly returns `False`, meaning that this bug only happens the first time.
This is an issue because a loop like:
```py
while reader.has_message_available():
msg = reader.read_next()
```
will end up hanging when it reaches the end of the topic, due to the incorrectly returned `True` value from `has_message_available()`.
---
Tested by having a producer that publishes one message per second:
```py
import json
import time
import pulsar
client = pulsar.Client("pulsar://localhost:6650")
producer = client.create_producer(
"persistent://foo/bar/baz",
)
while True:
data = json.dumps({"ts": time.time()}).encode("utf-8")
print("Sending data:", data, flush=True)
producer.send(data)
time.sleep(1)
```
Then start a reader like so:
```py
import time
import pulsar
client = pulsar.Client("pulsar://localhost:6650")
reader = client.create_reader(
"persistent://foo/bar/baz",
start_message_id=pulsar.MessageId.latest,
)
reader.seek(int(time.time() - 10) * 1000) # Seek to 10s ago.
while True:
print(reader.has_message_available(), flush=True)
print(reader.has_message_available(), flush=True)
print(reader.has_message_available(), flush=True)
print("="*20)
msg = reader.read_next()
```
Observed output:
- Initially there's a lot of `True True True`, for messages in the past while the reader is catching up
- Once the reader catches up, the outputs become `True False False`, indicating that the first `has_message_available()` call each time is incorrectly giving `True`
Expected output:
- Once the reader has caught up, the outputs should be `False False False`, since there's no message available and `read_next()` needs to block until a new message comes in.
Notes:
- The bug doesn't seem to happen if there's no seek by timestamp done at the start.
- The bug doesn't seem to happen if there's a small delay after the `read_next()` (e.g. `time.sleep(0.1)`) -- for this it's easier to observe if you make the producer publish slower.
- The bug was probably introduced in v3.5.0, since it does not occur with the v3.4.0 client but it does with the v3.5.0 client.
Guía de contribución
Evaluación
Este issue todavía no se ha evaluado.