Upgrade DebeziumIO to Debezium 3.5.2.Final
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 196
Description
### What needs to happen?
`DebeziumIO` currently pins Debezium `3.1.3.Final` (bumped in #37571). Upstream has moved on considerably and the connector should be brought up to the current stable patch line, `3.5.2.Final`.
What the upgrade absorbs:
* **Kafka Connect 4.x alignment.** Debezium 3.5 builds against Kafka Connect 4.1.2, so the `connect-api` pin has to move from 3.9.0. Connect 4.1 also adds an abstract `pluginMetrics()` to `SourceTaskContext`, which `BeamSourceTaskContext` must implement.
* **`debezium-core` module split.** Debezium 3.5 split `debezium-core` into smaller modules (`debezium-config`, `debezium-connector-common`, and friends). The Gradle dependencies and the expansion-service shadow jar need auditing so the split classes are still bundled.
* **Config and behaviour changes.** The `schema_only` snapshot mode was removed in 3.3, and connection failures now surface wrapped in a `RetriableException`.
* **Postgres JDBC driver.** The 3.5 Postgres connector requires driver 42.7.x, while Beam globally pins 42.6.2 for CVE reasons (24f562d84f6).
### Notes
`3.6.0` is intentionally skipped for now: it shipped a rewritten MySQL DDL parser with several open regressions and there is no `3.6.1` yet. A follow-up bump to the 3.6 line should be trivial once those settle.
### Issue Priority
Priority: 2 (default / most bugs should be filed as P2)
### Issue Components
- [x] Component: Java SDK
- [x] Component: IO connector
Contributor guide
Research direction
Start by locating DebeziumIO, its Gradle dependencies, the expansion-service shadow jar, and BeamSourceTaskContext. Compare the current Debezium 3.1.3.Final and Kafka Connect pins with the 3.5.2.Final requirements, then inspect snapshot and connection-failure handling. Done means the split modules are bundled, pluginMetrics() is supported, the required Postgres driver is compatible, and connector tests pass.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, postgresql
- Domain
- backend, database
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100