apache / apache/pulsar-connectors

[Bug] Debezium SQL Server source fails to start: multi-partition connector never receives task.id

Open
#61 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
26
Forks
25
Avg merge
17h 7m
Merged PRs (30d)
1

Description

## Description

The Debezium SQL Server source (`DebeziumMsSqlSource`) fails during `open()` with a `NullPointerException` inside Debezium's JMX metric-name construction:

```
java.lang.NullPointerException
at io.debezium.metrics.Metrics ... Sanitizer.jmxSanitize(null)
via SchemaHistoryMetrics
```

This was found while adding integration test coverage for the module (#43 / #60), and it reproduces against a real SQL Server container with the documented configuration. It is not a test-only problem.

## Root cause

`AbstractKafkaConnectSource.open()` only calls `connector.taskConfigs(1)` when the config contains the adaptor's own `kafkaConnectorSourceClass` key ([AbstractKafkaConnectSource.java#L144-L170](https://github.com/apache/pulsar-connectors/blob/master/kafka-connect-adaptor/src/main/java/org/apache/pulsar/io/kafka/connect/AbstractKafkaConnectSource.java#L144)):

```java
if (config.get(CONNECTOR_CLASS) != null) { // CONNECTOR_CLASS = "kafkaConnectorSourceClass"
...
List> configs = connector.taskConfigs(1);
taskConfig = configs.get(0);
} else {
// backward-compat path: instantiate the task directly from task.class
sourceTask = Class.forName(stringConfig.get(TaskConfig.TASK_CLASS_CONFIG))...;
taskConfig = stringConfig; // <-- raw user config, nothing injected
}
```

The Debezium sources set Debezium's own `connector.class` and `task.class` keys (see `DebeziumMsSqlSource.setDbConnectorClass/setDbConnectorTask`), **not** `kafkaConnectorSourceClass`. So every Debezium source takes the `else` branch and `SqlServerConnector.taskConfigs()` is never invoked.

That matters for SQL Server specifically because it is a **multi-partition** connector: `taskConfigs()` is what assigns each task its `task.id`, and `SqlServerConnectorConfig` requires it. Starting the task directly leaves `task.id` unset, and the null propagates into the metrics layer.

Single-partition connectors (MySQL, Postgres) don't hit this, which is why the existing tests pass.

## Workaround

Setting `task.id=0` explicitly in the connector config lets the source start. #60 does this in its integration test with an explanatory comment so the test can land, but the underlying gap should be fixed so users don't have to.

## Suggested fix

Route Debezium sources through `taskConfigs()` (i.e. have `DebeziumSource` populate `kafkaConnectorSourceClass`, or invoke `taskConfigs()` in the backward-compat branch when a connector class is resolvable), so multi-partition connectors are configured the way Kafka Connect configures them. A regression test for SQL Server exists in #60 once merged; removing the explicit `task.id` there would validate the fix.

Contributor guide

No contributing guide indexed for this repository

Research direction

Read AbstractKafkaConnectSource.open() at the linked lines, then inspect DebeziumSource and DebeziumMsSqlSource.setDbConnectorClass/setDbConnectorTask. Run the SQL Server integration coverage from #60 and verify the source starts without an explicit task.id; done means multi-partition task configuration reaches the Debezium task and the regression passes.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, sql
Domain
backend, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
62/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.