alibaba / alibaba/DataX

一下把数据文件全清空了,还好是测试环境

Open
#139 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
17.4k
Forks
5.7k
PR merge metrics
No merged PRs in 30d

Description

本地文件导入hive。。。。
json
{
"job": {
"content": [
{
"reader": {
"name": "txtfilereader",
"parameter": {
"encoding": "UTF-8",
"fieldDelimiter": "\t",
"path": ["D:/dataxPath"]
}
},
"writer": {
"name": "hdfswriter",
"parameter": {
"column": [{
"name":"a",
"type":"String"
},{
"name":"b",
"type":"String"
}],
"defaultFS": "hdfs://xxxx:xxxx",
"fieldDelimiter": "\t",
"fileName": "xxxx",
"fileType": "text",
"path": "/apps/hive/warehouse/orc_table",
"writeMode": "append"
}
}
}
],
"setting": {
"speed": {
"channel": "2"
}
}
}
}

最后hdfs上数据文件都没了
日志输出:
![default](https://user-images.githubusercontent.com/35592438/42742500-a10043b0-88ed-11e8-9867-40cb84de6dd5.png)

Contributor guide

No contributing guide indexed for this repository

Research direction

Review the provided txtfilereader and hdfswriter configuration and attached log first. Reproduce the import against a test HDFS/Hive path, investigate why files under /apps/hive/warehouse/orc_table were removed despite writeMode append, and define a regression test or reproduction that demonstrates the corrected behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
data-engineering, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.