HDFS Writer String 类型 中存在 '\n'
Open
- Dominant language
- Java
- Stars
- 17.4k
- Forks
- 5.7k
- PR merge metrics
- No merged PRs in 30d
Description
当数据中存在 \n 时写入的数据会换行、、由于hive、impala等默认是 换行符 分割 每条数据,所以会导致 这部分数据 错乱,可不可以考虑在写入的时候 replace 成 空格呢,我遇到的情况是 直接存 html ,其中的 \n 导致的
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by locating the HDFS Writer path that serializes String values, then reproduce the issue with HTML containing newline characters and check how Hive or Impala reads the resulting records. Confirm the intended newline handling and add or update coverage so the output no longer splits the value into separate records.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- hadoop, java
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 42/100