apache / apache/paimon

[Bug] In cdc database synchronization, data changes to new tables using the combined mode are not monitored, and the job needs to be restarted

Open
#3,511 4 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Java
Stars
3.4k
Forks
1.4k
Avg merge
1d 11h
Merged PRs (30d)
396

Description

### Search before asking

- [X] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar.

### Paimon version

paimon-flink-1.18-0.9-20240610.002222-13
paimon-flink-action-0.9-20240610.002222-13

### Compute Engine

flink1.18

### Minimal reproduce step

In the cdc database synchronization, the combined mode synchronizes new tables, but does not monitor data changes to the new tables, so the job needs to restart

### What doesn't meet your expectations?

I want to dynamically listen to the binlog of new table data without restarting the job

### Anything else?

_No response_

### Are you willing to submit a PR?

- [ ] I'm willing to submit a PR!

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the CDC database synchronization path for combined mode and reproduce the issue on Flink 1.18 using the versions listed. Trace how newly synchronized tables are registered for binlog monitoring; done means data changes in new tables are observed without restarting the job.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
data-engineering, databases, stream-processing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.