NVIDIA / NVIDIA/cudf

[FEA] Add support for escape characters in CSV

Open
#11,984 0 comments 0 reactions 0 assignees View on GitHub
0 - Backlog cuIO feature request libcudf Python Spark
Dominant language
C++
Stars
9.8k
Forks
1.1k
Avg merge
3d 6m
Merged PRs (30d)
278

Description

**Is your feature request related to a problem? Please describe.**
By default Spark uses an escape character `\` to escape things like double quotes within other double quotes i.e. `"\""`. Pandas by default uses two double quotes right next to each other. Spark does not support this method of escaping.

**Describe the solution you'd like**
We would like an option in the csv reader setting to allow setting an escape character. There should be an option to either disable the double double quote escaping, or have it automatically disabled if an escape character is provided.

**Describe alternatives you've considered**
write our own CSV parser.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.