读取orc文件错误
- Dominant language
- Java
- Stars
- 17.4k
- Forks
- 5.7k
- PR merge metrics
- No merged PRs in 30d
Description
1、环境:
HDFS 3.1.1.3.1
HIVE 3.0.0.3.1
hive表是ORC格式,
2、JOB配置:
{
"job": {
"setting": {
"speed": {
"channel": 3
}
},
"content": [
{
"reader": {
"name": "hdfsreader",
"parameter": {
"path": "/warehouse/tablespace/managed/hive/st.db/detail_article_book/date_no=2023/base_0000001/bucket_00000",
"defaultFS": "hdfs://10.80.1.100:8020/",
"column": ["*"],
"fileType": "orc",
"encoding": "UTF-8",
"fieldDelimiter": "\u0001",
"compress":"SNAPPY"
}
},
"writer": {
"name": "streamwriter",
"parameter": {
"print": true
}
}
}
]
}
}
3、问题:
直接从HDFS读数据后打印出来象下面这样,标红的那段不是表中的数据,不知从何而来,然后表中的数据被全部读到了一个字段中,就是标红后面的花括号括起来的那部分。

各位有遇到过吗?
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the hdfsreader entry point and reproduce the supplied job against the ORC path and configuration. Inspect how the reader interprets ORC records and maps columns, then verify the output contains only table values with each value in its own field.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100