debezium / debezium/dbz

Add support for storing Debezium offsets and schema history in Apache Pulsar [DBZ-9715]

Open
#1,150 0 comments 0 reactions 0 assignees View on GitHub
component/core-library migrated-from-jira type/enhancement
Dominant language
HTML
Stars
6
Forks
8
Avg merge
2d 19h
Merged PRs (30d)
1

Description

Migrated from [DBZ-9715](https://issues.redhat.com/browse/DBZ-9715)

Debezium currently supports a set of offset and schema history storage backends (Kafka, File, JDBC, Redis, etc.), but Pulsar is not supported, which limits native integration when Debezium is deployed inside a Pulsar ecosystem.

 

*Feature request*

Add a Pulsar-based implementation of:
* OffsetBackingStore
* SchemaHistory

so that offsets and schema history can be persisted directly in Pulsar topics, without requiring Kafka or external storage.

*Implementation idea*

Pulsar already provides a working implementation in its Kafka-connect adapter module, specifically in [PulsarSchemaHistory|https://github.com/apache/pulsar/blob/master/pulsar-io/debezium/core/src/main/java/org/apache/pulsar/io/debezium/PulsarSchemaHistory.java] and [PulsarOffsetBackingStore|https://github.com/apache/pulsar/blob/master/pulsar-io/kafka-connect-adaptor/src/main/java/org/apache/pulsar/io/kafka/connect/PulsarOffsetBackingStore.java]

Perhaps this code may be integrated into debezium ?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.