apache / apache/pulsar-connectors

[feat][io] Add Debezium CockroachDB source connector

Open
#68 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
26
Forks
25
Avg merge
17h 7m
Merged PRs (30d)
1

Description

## Motivation

Debezium ships a CockroachDB source connector (`io.debezium:debezium-connector-cockroachdb`, present in the `debezium-bom` at our pinned version), but this repository does not wrap it. We currently wrap 5 of Debezium's 12 source connectors: MySQL, PostgreSQL, MongoDB, Oracle, and SQL Server.

Unlike the other gaps, this connector does **not** use the logical-decoding or binlog machinery — it consumes CockroachDB's native **changefeeds**.

## Proposal

Add a `debezium/cockroachdb` module (`pulsar-io-debezium-cockroachdb`) following the structure of the existing Debezium modules:

- `DebeziumCockroachDbSource` extending `DebeziumSource`
- `include("debezium:pulsar-io-debezium-cockroachdb")` in `settings.gradle.kts`
- `debezium-connector-cockroachdb = { module = "io.debezium:debezium-connector-cockroachdb" }` in the version catalog
- NAR packaging + distribution wiring
- A Testcontainers integration test using the `cockroachdb/cockroach` image (`start-single-node --insecure`), enabling rangefeeds (`SET CLUSTER SETTING kv.rangefeed.enabled = true`) and creating a changefeed in setup

## Notes

- **This is the newest and least battle-tested connector of the set** — it arrived in the 3.4 line. Treat it as lower priority than #65 (MariaDB) and Db2 unless there is concrete user demand, and expect the connector's own behavior to shift across Debezium minors (see the migration tracked in #63).
- Because it uses native changefeeds rather than the standard relational CDC path, verify the emitted record shape and offset semantics match what `DebeziumSource` and `PulsarSchemaHistory` expect — it may not need schema history at all.
- Use the `GenericContainer` pattern (see the Testcontainers version conflict noted in #62).

_Identified by a scan of the Debezium connector catalog against our connector inventory._

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by comparing the existing Debezium modules and their NAR and distribution wiring, then inspect settings.gradle.kts and the version catalog. Build the module around DebeziumCockroachDbSource and use a GenericContainer integration test with the cockroachdb/cockroach image, rangefeeds, and a changefeed. Done means the module is packaged and wired into the distribution, and the test verifies emitted records and offset semantics.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, java
Domain
databases, testing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.