当数据中有几串连续的空的，数据在解析的时候将空的给过滤了，比如[1,2,,,,6]解析得到的是[1,2,6],出现列越界的问题
Open
- Dominant language
- Java
- Stars
- 17.4k
- Forks
- 5.7k
- PR merge metrics
- No merged PRs in 30d
Description
使用Datax进行两个集群间的数据同步，在读取HDFS数据时，当数据中有几串连续的空的，数据在解析的时候将空的给过滤了，比如[1,2,,,,6]解析得到的是[1,2,6],出现列越界的问题。
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reproducing the HDFS-reading case with a row such as [1,2,,,,6] and trace DataX's parsing path to find where consecutive empty fields are removed. Confirm the fix by verifying that empty fields remain in their original positions and that the resulting column count no longer causes an out-of-bounds error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- hadoop, java
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100