[Bug] Incorrectly including tables matching excludingTablePattern in combined mode cdc
- Dominant language
- Java
- Stars
- 3.4k
- Forks
- 1.4k
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 396
Description
### Search before asking
- [X] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar.
### Paimon version
master
### Compute Engine
flink
### Minimal reproduce step
1. Create a database named cdc_test in mysql, then create primary table named pk_1, pk_2 ... pk_100 and non-primary table named non_pk_1, non_pk_2, ... non_pk_100.
2. Start a combined mode mysql database cdc job, and set excludingTablePattern to 'non_pk_.+'.
3. Concurently create non-primary table named non_pk_101 when starting cdc job.
4. Finally, the Jobmanager log will show "com.ververica.cdc.connectors.mysql.source.utils.TableDiscoveryUtils [] - including ‘cdc_test.non_pk_101’ for further processing".
### What doesn't meet your expectations?
non_pk_101 obviously matches the excludingTablePattern and needs to be excluded. Since non_pk_101 is a non-primary key table, MySqlChunkSplitter will report an error:
Caused by: org.apache.flink.table.api.ValidationException: Incremental snapshot for tables requires primary key, but table cdc_test.non_pk_101 doesn't have primary key.

combinedModeTableList func uses excluding pattern ?!(^db\\.tbl$)|(^...$),and this will miss the newly table that matches excludingTablePattern and created when starting cdc job.
### Anything else?
_No response_
### Are you willing to submit a PR?
- [X] I'm willing to submit a PR!
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by tracing the combinedModeTableList function and the TableDiscoveryUtils log path involved in MySQL CDC table discovery. Reproduce the concurrent creation of non_pk_101 with excludingTablePattern set, then verify that the matching table is excluded and MySqlChunkSplitter no longer reports the missing-primary-key error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, mysql
- Domain
- data-engineering, databases, stream-processing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100