apache / apache/rocketmq-externals

[rocketmq-connect] For replicator design ideas, seek advice.

Open
#686 5 comments 1 reaction 0 assignees View on GitHub
discuss enhancement
Dominant language
Java
Stars
4.6k
Forks
3k
Avg merge
2h 33m
Merged PRs (30d)
1

Description

After reading the code of rocketmq connect module, I have some questions to ask for advice.

1.The current usage of rocketmq-connect is that transferred to the middle tier by sourceTask, and then transferred to the target cluster by sinkTask. For rocketmq type data sources, can they be directly synchronized to the target cluster?

2.As for RocketMQConverter, I understand whether it is a special processing for rocketmq, which should not be implemented in rocketmq-connect or implemented in RocketMQConverter. Do you have any opinions on this part of the community?
![image](https://user-images.githubusercontent.com/30995057/109784581-2bd70980-7c46-11eb-8ed2-842cdc192a43.png)

3.I plan to implement the WorkerDirctTask of direct forwarding by encapsulating WokerSourceTask and WokerSinkTask. Is this method in line with the community development plan?

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading the rocketmq-connect module and tracing the sourceTask and sinkTask flow. Then examine RocketMQConverter, WorkerDirctTask, WokerSourceTask, and WokerSinkTask to compare the proposed direct-forwarding design. The issue is done only when the community provides a decision on the design direction.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.