apache / apache/pulsar-connectors

[Bug] Canal source client is incompatible with modern protobuf-java (NoSuchMethodError on connect)

Open
#99 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
26
Forks
25
Avg merge
17h 7m
Merged PRs (30d)
1

Description

## Description

The Canal source connector's client library (`com.alibaba.otter:canal.client:1.1.7` / `canal.protocol:1.1.7`) is incompatible with the protobuf-java version the build resolves. Its generated `CanalPacket` code calls `com.google.protobuf.GeneratedMessageV3.makeExtensionsImmutable()`, a method **removed** in modern protobuf-java (the 3.21+/4.x line). So `CanalConnector.connect()` throws `NoSuchMethodError` and the source can never complete the canal-server handshake.

Found while adding the first integration test for the module (#48 / #98) — the module had zero tests, so this was never exercised.

## Evidence

On the canal module's **test** classpath, protobuf-java resolves to 4.31.1 (pulled by `pulsar-client`/`pulsar-functions-instance`), and `GeneratedMessageV3` in that version has no `makeExtensionsImmutable`. The integration test reproduces the `NoSuchMethodError` on `connect()` without a version pin.

## Nuance worth a maintainer's eye — is production affected?

- The module's **runtimeClasspath** (what the NAR bundles) resolves **no** external `protobuf-java` at all, and `canal.protocol-1.1.7.jar` does not bundle protobuf classes. So the NAR appears to rely on whatever protobuf the Pulsar runtime provides — which is modern — meaning the connector would very likely hit the same `NoSuchMethodError` **in production**, not just in the test. This should be confirmed against a real deployment.

## The fix in #98, and why it needs review

#98 makes the integration test pass by forcing `protobuf-java` to **3.6.1** (canal's declared version) via `configurations.all { resolutionStrategy { force(...) } }`. That works, but:

1. **3.6.1 is from 2018** and carries known CVEs (e.g. CVE-2021-22569, CVE-2022-3171). Pinning the whole module — including the shipped NAR — to it is heavy.
2. The demonstrated breakage is on the **test** classpath; whether the NAR needs the pin depends on the production question above. If production is unaffected, the force should be scoped to test configurations only.
3. **`canal.client` 1.1.8 exists** (latest). If it is built against a newer protobuf, upgrading is a cleaner fix than downgrading protobuf. Worth checking before committing to the 3.6.1 pin.

## Suggested resolution

Decide among: (a) upgrade to `canal.client`/`canal.protocol` 1.1.8 if protobuf-4-compatible; (b) shade/relocate protobuf for the canal NAR; or (c) keep the 3.6.1 pin but scope it correctly (test-only if production is unaffected) and accept the CVE tradeoff. #98 currently takes (c) module-wide; flagging for a decision.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start in the Canal source module's dependency configuration and inspect integration test #98, which reproduces the failure at CanalConnector.connect(). Compare the test and runtime classpaths, check whether canal.client/canal.protocol 1.1.8 is compatible, and confirm done when the handshake succeeds without an unsafe module-wide dependency choice.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.