apache / apache/rocketmq-externals

[rocketmq-connect-jdbc: JdbcSinkConnector fetch topic list error]

Open
#558 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
4.6k
Forks
3k
Avg merge
2h 33m
Merged PRs (30d)
1

Description

The issue tracker is **ONLY** used for bug report and feature request.

Any question or RocketMQ proposal please use our [mailing lists](http://rocketmq.apache.org/about/contact/).

**BUG REPORT**

1. Please describe the issue you observed:
According to ISSUE #549 and thanks to @gripson findings, we found that `fetch topic list error` appears at a constant speed even after the connector "semantically" stopped.

The root cause for this is that, runtime doesn't force connectors to stop. It all relies on the connectors `stop()` method to make sure resources are released properly. However because JdbcSinkConnector introduced a route topic listener to monitor route changes, the scheduled task is not cancelled at `stop()`

```
public void startListener() {
listenerHandle = executor.scheduleAtFixedRate(new Runnable() {
boolean first = true;
HashMap> origin = null;

@Override
public void run() {
buildRoute();
if (first) {
origin = CloneUtils.clone(topicRouteMap);
first = false;
}
if (!compare(origin, topicRouteMap)) {
context.requestTaskReconfiguration();
origin = CloneUtils.clone(topicRouteMap);
}
}
}, ((SinkDbConnectorConfig) dbConnectorConfig).getRefreshInterval(), ((SinkDbConnectorConfig) dbConnectorConfig).getRefreshInterval(), TimeUnit.SECONDS);
}
```

A direct solution (but very likely not the best) would be cancel the schedule at `stop()`, something like
```
listenerHandle.cancel(true);
```
where listenerHandle is a `ScheduledFuture`.

Other solutions are more than welcomed (as I really think mine is so ugly LOL)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with JdbcSinkConnector's startListener entry point and its stop lifecycle, focusing on the ScheduledFuture used for the route listener. Verify that stopping the connector prevents further topic-list fetches and that the scheduled task is properly cancelled or otherwise shut down.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.