apache / apache/doris-flink-connector
[Bug] Doris include table pattern null
- Dominant language
- Java
- Stars
- 385
- Forks
- 279
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 6
Description
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues.
### Version
Doris flink connector: flink-doris-connector-1.17-1.5.0
### What's Wrong?
When I use `--single-sink` flag to start my flink job, the task will report error:
```text
Caused by: java.lang.IllegalArgumentException: Can't find any matched tables, please check your configured database-name: [svc_dw] and table-name: [((svc_dw)\.(null))(?!(svc_dw\.(upload_histories|supported_schemas))$)]
```
I try to figure out the question and I found the [code](https://github.com/apache/doris-flink-connector/blob/d0c1d7c61db56c891071de6cb4d5bcea26835e15/flink-doris-connector/src/main/java/org/apache/doris/flink/tools/cdc/DatabaseSync.java#L308-L325) that causes the question:
```java
protected String getSyncTableList(List syncTables) {
if (!singleSink) {
return syncTables.stream()
.map(v -> getTableListPrefix() + "\\." + v)
.collect(Collectors.joining("|"));
}
// includingTablePattern and ^excludingPattern
String includingPattern =
String.format("(%s)\\.(%s)", getTableListPrefix(), includingTables);
if (StringUtils.isNullOrWhitespaceOnly(excludingTables)) {
return includingPattern;
} else {
String excludingPattern =
String.format("?!(%s\\.(%s))$", getTableListPrefix(), excludingTables);
return String.format("(%s)(%s)", includingPattern, excludingPattern);
}
}
```
If I use `--excluding-tables` flag but not use `--including-tables` flag with `--single-sink` flag,the mysql flink cdc source will get the `--table-name` with value: `((svc_dw)\.(null))(?!(svc_dw\.(upload_histories|supported_schemas))$)`
### What You Expected?
Start the flink job successfully with `--single-sink` and `--excluding-tables` flags.
### How to Reproduce?
Start a flink job with `--single-sink` and `--excluding-tables` flags and without `--including-tables` flag.
### Anything Else?
_No response_
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
Contributor guide
Research direction
Start in flink-doris-connector/src/main/java/org/apache/doris/flink/tools/cdc/DatabaseSync.java at getSyncTableList, referenced in the issue. Reproduce with --single-sink and --excluding-tables without --including-tables, then verify the generated table pattern no longer contains null and the Flink job starts successfully.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, mysql
- Domain
- backend, data-engineering
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 35/100