Aiven-Open / Aiven-Open/jdbc-connector-for-apache-kafka

Using primary or unique keys to construct kafka keys

Ouverte
#88 0 commentaires 0 réactions 0 personnes assignées Voir sur GitHub
Langage dominant
Java
Étoiles
113
Forks
62
Merge moyen
4 h 12 min
PR mergées (30 j)
1

Description

The jdbc-source-connector does not create kafka keys by default. I found some question on stackoverlow that mentions using a kafka transformer. Instead why not use primary keys as keys in Kafka? What fields are primary keys is already retrieved via GenericDatabaseDialect amd stored in ColumnDefinition. I'd think it would be not too hard to extend the jdbc-source-connecor to use the analoguous property PK_MODE. When set to RECORD_KEY the source connector could create keys as well using primary keys.
Advantage would be that compacting could be enabled for topics created using the jdbc-source-connector.

At first glance I'd think this feature wouldn't be too hard to add. But before I'd start working an a pull-request, Are there any reasons why this feature is not added? Is there additional complexity that I am missing?

Guide de contribution

Ouvrir le guide de contribution

Évaluation

Cette issue n'a pas encore été évaluée.

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.