aio-libs / aio-libs/aiokafka

Kafka producer lost headers for message

Abierto
#605 2 comentarios 0 reacciones 0 asignados Ver en GitHub
Lenguaje dominante
Python
Estrellas
1.4k
Forks
269
Merge medio
1 d 1 h
PR fusionados (30 d)
6

Descripción

Sometimes AIOKafkaProducer writes messages without headers.

Environment:
- python version `3.7.4`
- aiokafka version `0.5.2`
- kafka-python version `1.4.6`
- docker image `confluentinc/cp-kafka:5.2.1`

In new environment I created a new topic `test` and launched code in Jupyter:
```python
import aiokafka
consumer = aiokafka.AIOKafkaConsumer(
'test',
loop=asyncio.get_event_loop(),
bootstrap_servers='localhost:9092',
)
await consumer.start()
producer = aiokafka.AIOKafkaProducer(
loop=asyncio.get_event_loop(),
bootstrap_servers='localhost:9092',
)
await producer.start()

msg = ('a'*100).encode()
for _ in range(500):
await producer.send('test', value=msg, headers=[('type', b'Normal')])

for _ in range(500):
doc = await consumer.getone()
assert doc.headers == (('type', b'Normal'),)
```
I got AssertionError at ~135th message. Kafka Tool also shows message without headers.
Interesting that producing smaller amount of messages does not raise an error.
Tested also with other Kafka entities and got same result.

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.