[FEA] Add support for escape characters in CSV
Open
0 - Backlog
cuIO
feature request
libcudf
Python
Spark
- Dominant language
- C++
- Stars
- 9.8k
- Forks
- 1.1k
- Avg merge
- 3d 6m
- Merged PRs (30d)
- 278
Description
**Is your feature request related to a problem? Please describe.**
By default Spark uses an escape character `\` to escape things like double quotes within other double quotes i.e. `"\""`. Pandas by default uses two double quotes right next to each other. Spark does not support this method of escaping.
**Describe the solution you'd like**
We would like an option in the csv reader setting to allow setting an escape character. There should be an option to either disable the double double quote escaping, or have it automatically disabled if an escape character is provided.
**Describe alternatives you've considered**
write our own CSV parser.
Contributor guide
Assessment
This issue has not been assessed yet.