SnowflakeIO fails to recognize new-line char in a column
Open
bug
io
java
P2
snowflake
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 196
Description
When a snowflake string column contains new-line, the snowflakeIO fails to read it as a record. Instead two invalid records gets created.
I see the issue due to the usage of TextIO; as the exported extract from the snowflake contains the column data enclosed with quoted string, making it a valid CSV.
Possibly be solved if FileIO(with custom splitable dofn to handle big file) is used instead of TextIO.
Imported from Jira [BEAM-14526](https://issues.apache.org/jira/browse/BEAM-14526). Original Jira may contain additional context.
Reported by: Varun Rauthan.
Contributor guide
Assessment
This issue has not been assessed yet.