Generify ExtractNewRecordState to support pluggable Messaging systems [DBZ-1608]
- Dominant language
- HTML
- Stars
- 6
- Forks
- 8
- Avg merge
- 2d 19h
- Merged PRs (30d)
- 1
Description
Migrated from [DBZ-1608](https://issues.redhat.com/browse/DBZ-1608)
Right now [ExtractNewRecordState.java|https://github.com/debezium/debezium/blob/master/debezium-core/src/main/java/io/debezium/transforms/ExtractNewRecordState.java] is tightly coupled and implemented based on Kafka. We are using Debezium with Apache Pulsar for CDC but one missing feature is the Record Transformation in producer side before publishing to the topic.
As per the Apache Pulsar [Transformation support in Github issue|https://github.com/apache/pulsar/issues/4672#issuecomment-509877837], tight coupling nature of ExtractNewRecordState with Kafka is a blocker. Instead of writing our implementation of ExtractNewRecordState for Pulsar, it would be best to Generify the ExtractNewRecordState class in debezium to support any messaging system (which should be configurable at connector setup).
PS: The comment in the provided Github issue link is by official maintainer / contributor of Apache Pulsar. I am just an end user of Apache Pulsar.
Contributor guide
Research direction
Start by reading debezium-core/src/main/java/io/debezium/transforms/ExtractNewRecordState.java and the linked Apache Pulsar transformation discussion. Identify where the transform is coupled to Kafka and how connector setup would need to select messaging-system behavior. The issue is done when the transform supports the requested pluggable messaging systems without requiring a separate Pulsar implementation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, kafka
- Domain
- distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100