confluentinc / confluentinc/confluent-kafka-python

flush leaves messages remaining

Open
#1,817 2 comments 1 reaction 0 assignees View on GitHub
component:producer enhancement status:waiting-for-interest
Dominant language
Python
Stars
509
Forks
964
Avg merge
2d 2h
Merged PRs (30d)
14

Description

Description
===========

I call flush in my setup and expect the message to be sent, and no warning produced, but a simple test of having one message produced, then a flush, leaves the following output

```
>>> print(KafkaProducer().flush())
%4|1726771494.331|TERMINATE|rdkafka#producer-8| [thrd:app]: Producer terminating with 1 message (3328 bytes) still in queue or transit: use flush() to wait for outstanding message delivery
None
```

You see that `None` is returned indicating the messages are sent, but I still get the Producer terminating message. Not sure how best to proceed.

How to reproduce
================

Produce a message with a payload, don't poll, but call flush

```
>>> print(KafkaProducer().flush())
%4|1726771494.331|TERMINATE|rdkafka#producer-8| [thrd:app]: Producer terminating with 1 message (3328 bytes) still in queue or transit: use flush() to wait for outstanding message delivery
None
```

Checklist
=========
Please provide the following information:

- [X] confluent-kafka-python and librdkafka version (`confluent_kafka.version()` and `confluent_kafka.libversion()`): `('2.5.0', 33882112)`, `('2.5.0', 33882367)`
- [X] Apache Kafka broker version: `3.5.1` through AWS MSK
- [X] Client configuration: `{
"api.version.request": False,
"bootstrap.servers": 'msk_server_1,msk_server_2',
"security.protocol": "SSL",
"ssl.ca.location": 'path/to/local/ssl/cert/,
}`
- [X] Operating system: `Debian`
- [ ] Provide client logs (with `'debug': '..'` as necessary)
- [ ] Provide broker log excerpts
- [ ] Critical issue

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.