[Feature] Cdc support new synchronization mode
- Dominant language
- Java
- Stars
- 3.4k
- Forks
- 1.4k
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 396
Description
### Search before asking
- [X] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar.
### Motivation
paimon cdc 支持divided和 combined 同步模式,对大量表同步场景,两种模式都有一定局限性。一般业务数据库大表很少,可能top5内,小表比较多,两种模式都不合适。
建议一种新模式,支持为大表设置独立sink,剩余表在一个sink中,可以为每个sink 设置并发度。
### Solution
_No response_
### Anything else?
_No response_
### Are you willing to submit a PR?
- [ ] I'm willing to submit a PR!
Contributor guide
No contributing guide indexed for this repository
Research direction
No files, tests, or entry points are named in the issue, so first clarify the proposed CDC synchronization mode and how it differs from divided and combined modes. Done should include a defined way to assign large tables to independent sinks, group remaining tables into one sink, and configure concurrency per sink.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering, stream-processing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100