alibaba / alibaba/DataX

hdfs2oracle时,如果hdfs目录存在空文件,且没有后缀名会报文件格式错误

Open
#896 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
17.4k
Forks
5.7k
PR merge metrics
No merged PRs in 30d

Description

报错内容:
经DataX智能分析,该任务最可能的错误原因是:
com.alibaba.datax.common.exception.DataXException: Code:[HdfsReader-10], Description:[读取文件出错]. - 检查文件[hdfs://172.25.88.117:9000/user/hive/warehouse/cmcc_new.db/dw_vod_view/pyear=2020/pmonth=11/pday=26/000000_0]类型失败,目前支持ORC,SEQUENCE,RCFile,TEXT,CSV五种格式的文件,请检查您文件类型和文件是否正确。 - java.lang.IndexOutOfBoundsException

Contributor guide

No contributing guide indexed for this repository

Research direction

The report identifies the HdfsReader-10 format-detection failure but names no source file or test. Locate the HdfsReader entry point, reproduce the case with an empty file without an extension, and trace how its format is classified. Done means the intended handling is defined and covered by a regression test.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
data-engineering
Issue type
Bug
Difficulty
3/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.