apache / apache/pulsar-connectors
[feat][io] Add Debezium CockroachDB source connector
- Dominant language
- Java
- Stars
- 26
- Forks
- 25
- Avg merge
- 17h 7m
- Merged PRs (30d)
- 1
Description
## Motivation
Debezium ships a CockroachDB source connector (`io.debezium:debezium-connector-cockroachdb`, present in the `debezium-bom` at our pinned version), but this repository does not wrap it. We currently wrap 5 of Debezium's 12 source connectors: MySQL, PostgreSQL, MongoDB, Oracle, and SQL Server.
Unlike the other gaps, this connector does **not** use the logical-decoding or binlog machinery — it consumes CockroachDB's native **changefeeds**.
## Proposal
Add a `debezium/cockroachdb` module (`pulsar-io-debezium-cockroachdb`) following the structure of the existing Debezium modules:
- `DebeziumCockroachDbSource` extending `DebeziumSource`
- `include("debezium:pulsar-io-debezium-cockroachdb")` in `settings.gradle.kts`
- `debezium-connector-cockroachdb = { module = "io.debezium:debezium-connector-cockroachdb" }` in the version catalog
- NAR packaging + distribution wiring
- A Testcontainers integration test using the `cockroachdb/cockroach` image (`start-single-node --insecure`), enabling rangefeeds (`SET CLUSTER SETTING kv.rangefeed.enabled = true`) and creating a changefeed in setup
## Notes
- **This is the newest and least battle-tested connector of the set** — it arrived in the 3.4 line. Treat it as lower priority than #65 (MariaDB) and Db2 unless there is concrete user demand, and expect the connector's own behavior to shift across Debezium minors (see the migration tracked in #63).
- Because it uses native changefeeds rather than the standard relational CDC path, verify the emitted record shape and offset semantics match what `DebeziumSource` and `PulsarSchemaHistory` expect — it may not need schema history at all.
- Use the `GenericContainer` pattern (see the Testcontainers version conflict noted in #62).
_Identified by a scan of the Debezium connector catalog against our connector inventory._
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by comparing the existing Debezium modules and their NAR and distribution wiring, then inspect settings.gradle.kts and the version catalog. Build the module around DebeziumCockroachDbSource and use a GenericContainer integration test with the cockroachdb/cockroach image, rangefeeds, and a changefeed. Done means the module is packaged and wired into the distribution, and the test verifies emitted records and offset semantics.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, java
- Domain
- databases, testing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100