[Question] How to write a configuration file for merging feature levels of different data sources and tables?
- Dominant language
- Java
- Stars
- 9.7k
- Forks
- 2.4k
- Avg merge
- 3d 9h
- Merged PRs (30d)
- 204
Description
### Search before asking
- [x] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues.
### What happened
How to write a configuration file for merging feature levels of different data sources and tables? Can it be achieved at present? If it's not possible, can you help me explain the reason? If possible, please help me write a related configuration file. For example, the tables in the two data sources have the same field ID number, but they have different attribute fields. So I want to combine the different attribute fields of the data with the same ID number into one, and then write them into the table of the target data source.
### SeaTunnel Version
2.312
### SeaTunnel Config
```conf
This is the configuration of different tables from the same source:
"{\"source\":[{\"plugin_name\":\"JDBC\",\"plugin_output\":\"fake\",\"url\":\"jdbc:mysql://192.168.8.25:3306/test\",\"driver\":\"com.mysql.cj.jdbc.Driver\",\"user\":\"root\",\"password\":\"123456\",\"query\":\"select UUID() as id,table1.name,table1.age,table1.address,table1.id_card_number,table2.phone from table1 LEFT JOIN table2 on table1.id_card_number = table2.id_card_number\"}],\"sink\":[{\"plugin_name\":\"JDBC\",\"plugin_input\":\"fake\",\"url\":\"jdbc:postgresql://192.168.8.127:54321/test\",\"driver\":\"org.postgresql.Driver\",\"user\":\"yid_blc_mds\",\"password\":\"Yuyao123456\",\"query\":\"INSERT INTO table_result (id, name, age,address, id_card_number, phone) VALUES (?, ?, ?, ?, ?, ?)\"}]}","env": "{\"parallelism\":2,\"job.mode\":\"BATCH\",\"checkpoint.interval\":10000}"
```
### Running Command
```shell
This is the configuration of different tables from the same source:
"{\"source\":[{\"plugin_name\":\"JDBC\",\"plugin_output\":\"fake\",\"url\":\"jdbc:mysql://192.168.8.25:3306/test\",\"driver\":\"com.mysql.cj.jdbc.Driver\",\"user\":\"root\",\"password\":\"123456\",\"query\":\"select UUID() as id,table1.name,table1.age,table1.address,table1.id_card_number,table2.phone from table1 LEFT JOIN table2 on table1.id_card_number = table2.id_card_number\"}],\"sink\":[{\"plugin_name\":\"JDBC\",\"plugin_input\":\"fake\",\"url\":\"jdbc:postgresql://192.168.8.127:54321/test\",\"driver\":\"org.postgresql.Driver\",\"user\":\"yid_blc_mds\",\"password\":\"Yuyao123456\",\"query\":\"INSERT INTO table_result (id, name, age,address, id_card_number, phone) VALUES (?, ?, ?, ?, ?, ?)\"}]}","env": "{\"parallelism\":2,\"job.mode\":\"BATCH\",\"checkpoint.interval\":10000}"
```
### Error Exception
```log
no
```
### Zeta or Flink or Spark Version
_No response_
### Java or Scala Version
_No response_
### Screenshots
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the JDBC source query and PostgreSQL sink INSERT configuration shown in the issue, then determine whether SeaTunnel supports merging rows from different tables or data sources by a shared ID. Done would require a confirmed supported configuration or a documented explanation that this workflow is not currently possible.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- mysql, postgresql
- Domain
- data-engineering, databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100