NVIDIA / NVIDIA/cudf

[FEA] `cudf.read_text` does not use the GDS-enabled datasource

Open
#13,880 2 comments 0 reactions 1 assignee Claimed by @adi-kmt View on GitHub
cuIO feature request good first issue libcudf Performance Python
Dominant language
C++
Stars
9.8k
Forks
1.1k
Avg merge
3d 6m
Merged PRs (30d)
278

Description

libcudf's `multibyte_split` has multiple ways to specify input data source when reading from a file:
1. `auto const source = cudf::io::text::make_source_from_file(temp_file_name); // no GDS`

2. `auto const datasource = cudf::io::datasource::create(temp_file_name);`
`auto const source = cudf::io::text::make_source(*datasource); // supports GDS`

`cudf.read_text` uses the first option to create the `data_chunk_source` object, so it does not benefit from potential direct reads with GDS.

`cudf.read_text` should create a `cudf::io::datasource` object to improve performance on GDS-enabled systems.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.