apache / apache/rocketmq-externals
[rocketmq-connect-runtime] Connector not recomputing task configs after connector config changes
- Dominant language
- Java
- Stars
- 4.6k
- Forks
- 3k
- Avg merge
- 2h 33m
- Merged PRs (30d)
- 1
Description
The issue tracker is **ONLY** used for bug report and feature request.
Any question or RocketMQ proposal please use our [mailing lists](http://rocketmq.apache.org/about/contact/).
**BUG REPORT**
1. Please describe the issue you observed:
```
Set stoppedConnector = new HashSet<>();
for (WorkerConnector workerConnector : workingConnectors) {
String connectorName = workerConnector.getConnectorName();
ConnectKeyValue keyValue = connectorConfigs.get(connectorName);
if (null == keyValue || 0 != keyValue.getInt(RuntimeConfigDefine.CONFIG_DELETED)) {
workerConnector.stop();
log.info("Connector {} stop", workerConnector.getConnectorName());
stoppedConnector.add(workerConnector);
} else if (!keyValue.equals(workerConnector.getKeyValue())) {
// I think we should trigger a task recompute, like if task parallelism changed, we should spawn more tasks
workerConnector.reconfigure(keyValue);
}
}
workingConnectors.removeAll(stoppedConnector);
```
- What did you do (The steps to reproduce)?
- What did you expect to see?
- What did you see instead?
2. Please tell us about your environment:
3. Other information (e.g. detailed explanation, logs, related issues, suggestions how to fix, etc):
**FEATURE REQUEST**
1. Please describe the feature you are requesting.
2. Provide any additional detail on your proposed use case for this feature.
2. Indicate the importance of this issue to you (blocker, must-have, should-have, nice-to-have). Are you currently using any workarounds to address this issue?
4. If there are some sub-tasks using -[] for each subtask and create a corresponding issue to map to the sub task:
- [sub-task1-issue-number](example_sub_issue1_link_here): sub-task1 description here,
- [sub-task2-issue-number](example_sub_issue2_link_here): sub-task2 description here,
- ...
Contributor guide
No contributing guide indexed for this repository
Research direction
Locate the connector-management loop in rocketmq-connect-runtime and inspect WorkerConnector.reconfigure and the handling of workingConnectors. Reproduce a connector configuration change, especially a task-parallelism change, and verify that the resulting task configurations are recomputed and the running tasks reflect the update.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100