NVIDIA / NVIDIA/cudf

[FEA] Support nanValue Spark CSV parse option in cudf CSV reader

Open
#4,001 0 comments 0 reactions 0 assignees View on GitHub
cuIO feature request libcudf Spark
Dominant language
C++
Stars
9.8k
Forks
1.1k
Avg merge
3d 6m
Merged PRs (30d)
278

Description

_Description of the request:_
Apache Spark CSV reader options include specifying values that should be interpreted as NaNs via `nanValue`.
Refer to [https://spark.apache.org/docs/latest/api/java/index.html?org/apache/spark/sql/DataFrameReader.html](url)
cuDF CSV reader seems to lack an equivalent

_Description of a possible solution_:
It would helpful to have a translation/support for that option in the cudf CSV reader options. (analogous to its na_values)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.