apache / apache/pulsar-connectors
[Bug] Debezium SQL Server source fails to start: multi-partition connector never receives task.id
- Dominant language
- Java
- Stars
- 26
- Forks
- 25
- Avg merge
- 17h 7m
- Merged PRs (30d)
- 1
Description
## Description
The Debezium SQL Server source (`DebeziumMsSqlSource`) fails during `open()` with a `NullPointerException` inside Debezium's JMX metric-name construction:
```
java.lang.NullPointerException
at io.debezium.metrics.Metrics ... Sanitizer.jmxSanitize(null)
via SchemaHistoryMetrics
```
This was found while adding integration test coverage for the module (#43 / #60), and it reproduces against a real SQL Server container with the documented configuration. It is not a test-only problem.
## Root cause
`AbstractKafkaConnectSource.open()` only calls `connector.taskConfigs(1)` when the config contains the adaptor's own `kafkaConnectorSourceClass` key ([AbstractKafkaConnectSource.java#L144-L170](https://github.com/apache/pulsar-connectors/blob/master/kafka-connect-adaptor/src/main/java/org/apache/pulsar/io/kafka/connect/AbstractKafkaConnectSource.java#L144)):
```java
if (config.get(CONNECTOR_CLASS) != null) { // CONNECTOR_CLASS = "kafkaConnectorSourceClass"
...
List> configs = connector.taskConfigs(1);
taskConfig = configs.get(0);
} else {
// backward-compat path: instantiate the task directly from task.class
sourceTask = Class.forName(stringConfig.get(TaskConfig.TASK_CLASS_CONFIG))...;
taskConfig = stringConfig; // <-- raw user config, nothing injected
}
```
The Debezium sources set Debezium's own `connector.class` and `task.class` keys (see `DebeziumMsSqlSource.setDbConnectorClass/setDbConnectorTask`), **not** `kafkaConnectorSourceClass`. So every Debezium source takes the `else` branch and `SqlServerConnector.taskConfigs()` is never invoked.
That matters for SQL Server specifically because it is a **multi-partition** connector: `taskConfigs()` is what assigns each task its `task.id`, and `SqlServerConnectorConfig` requires it. Starting the task directly leaves `task.id` unset, and the null propagates into the metrics layer.
Single-partition connectors (MySQL, Postgres) don't hit this, which is why the existing tests pass.
## Workaround
Setting `task.id=0` explicitly in the connector config lets the source start. #60 does this in its integration test with an explanatory comment so the test can land, but the underlying gap should be fixed so users don't have to.
## Suggested fix
Route Debezium sources through `taskConfigs()` (i.e. have `DebeziumSource` populate `kafkaConnectorSourceClass`, or invoke `taskConfigs()` in the backward-compat branch when a connector class is resolvable), so multi-partition connectors are configured the way Kafka Connect configures them. A regression test for SQL Server exists in #60 once merged; removing the explicit `task.id` there would validate the fix.
Contributor guide
No contributing guide indexed for this repository
Research direction
Read AbstractKafkaConnectSource.open() at the linked lines, then inspect DebeziumSource and DebeziumMsSqlSource.setDbConnectorClass/setDbConnectorTask. Run the SQL Server integration coverage from #60 and verify the source starts without an explicit task.id; done means multi-partition task configuration reaches the Debezium task and the regression passes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, sql
- Domain
- backend, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 62/100