Improve buffer flushing for JDBC sink connector [DBZ-7122]
Open
component/jdbc-connector
component/mongodb-connector
migrated-from-jira
- Dominant language
- HTML
- Stars
- 6
- Forks
- 8
- Avg merge
- 2d 19h
- Merged PRs (30d)
- 1
Description
Migrated from [DBZ-7122](https://issues.redhat.com/browse/DBZ-7122)
Current implementation will flush the buffer in any case every *put* call received by Kafka Connect.
The consequence is that *batch.size* will not be honored when *consumer.max.poll.records* < {*}batch.size{*}.
We want to eliminate this limitation adding a _heartbeat_ thread that will send messages to a dedicated topic so that Kafka is forced to call the connector *put* method. When the connector receive a message from _heartbeat_ the buffer will be flushed.
See DatabaseHeartbeatImpl for reference.
Contributor guide
Assessment
This issue has not been assessed yet.