NVIDIA / NVIDIA/cudf

[FEA] Separate datasource from reader parameters in cuIO

Open
#17,159 1 comment 0 reactions 0 assignees View on GitHub
cuIO feature request
Dominant language
C++
Stars
9.8k
Forks
1.1k
Avg merge
3d 6m
Merged PRs (30d)
278

Description

Currently, for reading various file formats, we do this:
```
auto opts = cudf::io::xxx_reader_options::builder(cudf::io::source_info{input_data_source_info})
.some_options(...)
...
.build();

auto output = read_xxx(opts);
```

When we have a large number of options to specify and have different datasources, it would be very burdensome to set the parameters repeatedly for every datasource.

We can avoid doing so by separating datasource from reading options. By doing so, we can just set the reading parameters once, then reuse the options instance multiple times:
```
auto opts = cudf::io::xxx_reader_options::builder(cudf::io::source_info{input_data_source_info})
.some_options(...)
...
.build();

auto output1 = read_xxx(source1, opts);
auto output2 = read_xxx(source2, opts);
....
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.