我下载过了,并没有什么用,没有文档没有源码。出问题也无法定位到问题。建议别用。> es动态column 有实现案例吗
- Dominant language
- Java
- Stars
- 17.4k
- Forks
- 5.7k
- PR merge metrics
- No merged PRs in 30d
Description
> es动态column 有实现案例吗
> > > > > > 使用datax elasticsearchwriter往 es 7.9.2 版本写入数据,column配置没有生效
> > > > > > elasticsearch版本7.9.2
> > > > > > datax配置:
> > > > > > "writer": {
> > > > > > "name": "elasticsearchwriter",
> > > > > > "parameter": {
> > > > > > "endpoint": "http://XXX:9200",
> > > > > > "accessId": "xxxx",
> > > > > > "accessKey": "xxxx",
> > > > > > "index": "XXXX",
> > > > > > "type": "_doc",
> > > > > > "cleanup": true,
> > > > > > "settings": {"index": {"number_of_shards": 1, "number_of_replicas": 0}},
> > > > > > "column": [
> > > > > > {"name": "company_id", "type": "integer"},
> > > > > > { "name": "company_nm","type": "text" },
> > > > > > { "name": "is_list","type": "integer" },
> > > > > > { "name": "list_type","type": "integer" },
> > > > > > { "name": "is_new_otc","type": "integer" },
> > > > > > { "name": "is_bond","type": "integer" },
> > > > > > { "name": "is_finance","type": "integer" },
> > > > > > { "name": "fin_type","type": "integer" },
> > > > > > { "name": "fin_type_supp","type": "text" },
> > > > > > { "name": "is_itfin","type": "integer" },
> > > > > > { "name": "is_del","type": "integer" },
> > > > > > { "name": "create_dt","type": "date", "format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis"},
> > > > > > { "name": "updt_dt","type": "date", "format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis"}
> > > > > > ],
> > > > > > "discovery": false,
> > > > > > "dynamic": false,
> > > > > > "batchSize": 1000
> > > > > > }
> > > > > > }
> > > > > > }
> > > > > > 最终生成的映射中column配置都没有生效,全量使用的是es默认mapping,特别是日期格式
> > > > >
> > > > >
> > > > > es客户端过时了,创建不了索引,得用es7的客户端改源码
> > > >
> > > >
> > > > 我同意你说的,目前提供的elasticsearchwriter还是旧版的,准备尝试使用elasticsearch7.x的客户端来改造writer
> > > > 链接:https://pan.baidu.com/s/1Wt5M10GlamNyI8MaqYZ3nQ
> > > > 提取码:137e,官方版本比较老,我这个已经升级,可以直接使用
> >
> >
> > 谢谢~
>
> 链接:https://pan.baidu.com/s/1PNIpPBCyKYTYpofFiwq3Wg 提取码:uk9z,之前的还是老版本,新版本地址是这个,请使用,有问题反馈
您好,您改造了elasticsearchwriter 是吗? 支持elasticsearch 7.x ? 方便提供下源码吗?
_Originally posted by @yunf17 in https://github.com/alibaba/DataX/issues/867#issuecomment-1120129113_
Contributor guide
No contributing guide indexed for this repository
Research direction
Start at the elasticsearchwriter implementation and its writer configuration, focusing on index creation, column mappings, dynamic settings, and the Elasticsearch client version. Reproduce the configuration against Elasticsearch 7.9.2; done means the writer creates the index with the requested mappings, including date formats, rather than relying on the default mapping.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- elasticsearch, java
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100