apache / apache/beam

Upgrade DebeziumIO to Debezium 3.5.2.Final

Open
#39,568 0 comments 0 reactions 0 assignees View on GitHub
io java P2
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
1d 20h
Merged PRs (30d)
196

Description

### What needs to happen?

`DebeziumIO` currently pins Debezium `3.1.3.Final` (bumped in #37571). Upstream has moved on considerably and the connector should be brought up to the current stable patch line, `3.5.2.Final`.

What the upgrade absorbs:

* **Kafka Connect 4.x alignment.** Debezium 3.5 builds against Kafka Connect 4.1.2, so the `connect-api` pin has to move from 3.9.0. Connect 4.1 also adds an abstract `pluginMetrics()` to `SourceTaskContext`, which `BeamSourceTaskContext` must implement.
* **`debezium-core` module split.** Debezium 3.5 split `debezium-core` into smaller modules (`debezium-config`, `debezium-connector-common`, and friends). The Gradle dependencies and the expansion-service shadow jar need auditing so the split classes are still bundled.
* **Config and behaviour changes.** The `schema_only` snapshot mode was removed in 3.3, and connection failures now surface wrapped in a `RetriableException`.
* **Postgres JDBC driver.** The 3.5 Postgres connector requires driver 42.7.x, while Beam globally pins 42.6.2 for CVE reasons (24f562d84f6).

### Notes

`3.6.0` is intentionally skipped for now: it shipped a rewritten MySQL DDL parser with several open regressions and there is no `3.6.1` yet. A follow-up bump to the 3.6 line should be trivial once those settle.

### Issue Priority

Priority: 2 (default / most bugs should be filed as P2)

### Issue Components

- [x] Component: Java SDK
- [x] Component: IO connector

Contributor guide

Open the contributing guide

Research direction

Start by locating DebeziumIO, its Gradle dependencies, the expansion-service shadow jar, and BeamSourceTaskContext. Compare the current Debezium 3.1.3.Final and Kafka Connect pins with the 3.5.2.Final requirements, then inspect snapshot and connection-failure handling. Done means the split modules are bundled, pluginMetrics() is supported, the required Postgres driver is compatible, and connector tests pass.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, postgresql
Domain
backend, database
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.