Add support for storing Debezium offsets and schema history in Apache Pulsar [DBZ-9715]
- Dominant language
- HTML
- Stars
- 6
- Forks
- 8
- Avg merge
- 2d 19h
- Merged PRs (30d)
- 1
Description
Migrated from [DBZ-9715](https://issues.redhat.com/browse/DBZ-9715)
Debezium currently supports a set of offset and schema history storage backends (Kafka, File, JDBC, Redis, etc.), but Pulsar is not supported, which limits native integration when Debezium is deployed inside a Pulsar ecosystem.
*Feature request*
Add a Pulsar-based implementation of:
* OffsetBackingStore
* SchemaHistory
so that offsets and schema history can be persisted directly in Pulsar topics, without requiring Kafka or external storage.
*Implementation idea*
Pulsar already provides a working implementation in its Kafka-connect adapter module, specifically in [PulsarSchemaHistory|https://github.com/apache/pulsar/blob/master/pulsar-io/debezium/core/src/main/java/org/apache/pulsar/io/debezium/PulsarSchemaHistory.java] and [PulsarOffsetBackingStore|https://github.com/apache/pulsar/blob/master/pulsar-io/kafka-connect-adaptor/src/main/java/org/apache/pulsar/io/kafka/connect/PulsarOffsetBackingStore.java]
Perhaps this code may be integrated into debezium ?
Contributor guide
Assessment
This issue has not been assessed yet.