csv抽取至hive,字段与数据对应关系不正确
- Dominant language
- Java
- Stars
- 17.4k
- Forks
- 5.7k
- PR merge metrics
- No merged PRs in 30d
Description
csv数据:

实际导入数据:

datax的json配置文件:
{
"job": {
"setting": {
"speed": {
"channel": 32
},
"errorLimit": {
"record": 0,
"percentage": 0.02
}
},
"content": [{
"reader": {
"name": "txtfilereader",
"parameter": {
"path": ["/home/haotian//measurements.csv"],
"encoding": "UTF-8",
"column": [{"index":0,"type":"string"},{"index":6,"type":"string"},{"index":8,"type":"string"},{"index":1,"type":"string"},{"index":12,"type":"string"},{"index":5,"type":"string"},{"index":2,"type":"string"},{"index":7,"type":"string"},{"index":10,"type":"string"},{"index":9,"type":"string"},{"index":4,"type":"string"},{"index":11,"type":"string"},{"index":3,"type":"string"}],
"fieldDelimiter":","
}
},
"writer":
{
"name": "hdfswriter",
"parameter": {
"defaultFS": "hdfs://192.168.2.112:8020",
"fileType": "text",
"path": "/apps/hive/warehouse/odsdb.db",
"fileName": "xwj3",
"column":[{"name":"CapturedTime","type":"string"},{"name":"DeviceID","type":"string"},{"name":"Height","type":"string"},{"name":"Latitude","type":"string"},{"name":"LoaderID","type":"string"},{"name":"LocationName","type":"string"},{"name":"Longitude","type":"string"},{"name":"MD5Sum","type":"string"},{"name":"Radiation","type":"string"},{"name":"Surface","type":"string"},{"name":"Unit","type":"string"},{"name":"UploadedTime","type":"string"},{"name":"Value","type":"string"}],
"writeMode": "append",
"fieldDelimiter": "\t"
}
}
}]
}
}
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reproducing the supplied CSV-to-Hive job with the shown txtfilereader and hdfswriter configuration, then inspect how the reader column indexes are paired with writer columns. Done means the imported fields match the intended CSV values and the mapping is covered by a reproducible verification.
Written by the indexing model from the issue text.
Assessment
- Domain
- data-engineering, databases
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100