从mysql 同步数据到 elasticsearch 时,只同步更新 elasticsearch 部分字段,不更新的字段不覆盖,可以吗
- Dominant language
- Java
- Stars
- 17.4k
- Forks
- 5.7k
- PR merge metrics
- No merged PRs in 30d
Description
{
"job": {
"setting": {
"speed": {
"channel": 2,
"record": -1,
"byte": -1,
"batchSize": 2048
}
},
"content": [
{
"reader": {
"name": "mysqlreader",
"parameter": {
"username": "xxx",
"password": "xxx",
"connection": [
{
"querySql": [
"SELECT t.id as _id, t.id as id, td.delegation as delegation FROM uat_partner.t_house_info t left join uat_partner.t_house_supporting_info ts on t.id = ts.house_id left join uat_partner.t_house_delegation_info td on t.id = td.house_id left join uat_hb_manager.sys_office ks on t.know_office_id = ks.id where t.id = 2111010924279766254 order by t.update_date desc;"
],
"jdbcUrl": [
"jdbc:mysql://xxx.xx.xxx.xxx:3306"
]
}
]
}
},
"writer": {
"name": "elasticsearchwriter",
"parameter": {
"endpoint": "http://xxx.xx.xx.xx:9200",
"accessId": "xxx",
"accessKey": "xxx",
"index": "in_index",
"indexType": "default",
"type": "_doc",
"cleanup": false,
"dynamic": false,
"batchSize": 1000,
"trySize": 30,
"multiThread": true,
"column": [
{
"name": "_id",
"type": "id"
},
{
"name": "id",
"type": "long"
},
{
"name": "delegation",
"type": "long"
}
]
}
}
}
]
}
}
我只想同步更新 ES 中的 delegation 字段,其它字段的值不更新,有什么方式吗
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the supplied DataX job configuration, especially the mysqlreader and elasticsearchwriter entries and the writer's column mapping. Investigate how elasticsearchwriter handles records when only delegation is selected, then define and verify behavior where unspecified Elasticsearch fields remain unchanged while delegation is updated.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- elasticsearch, java, mysql
- Domain
- data, databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100