NVIDIA / NVIDIA/cudf

[BUG] `pl.scan_csv(...).slice(...).collect(engine="gpu")` past the length of the csv raises RuntimeError

Open
#20,518 0 comments 0 reactions 0 assignees View on GitHub
bug cudf-polars cuIO libcudf
Dominant language
C++
Stars
9.8k
Forks
1.1k
Avg merge
3d 6m
Merged PRs (30d)
278

Description

xref https://github.com/pola-rs/polars/issues/25142

**Steps/Code to reproduce bug**
t1.csv
```
c0
true
false
```

```python
In [9]: import polars as pl

In [10]: pl.scan_csv("t1.csv").slice(10, 10).collect(engine=pl.GPUEngine(raise_on_fail=False))
...
File ~/cudf/python/cudf_polars/cudf_polars/dsl/ir.py:730, in Scan.do_evaluate(cls, schema, typ, reader_options, paths, with_columns, skip_rows, n_rows, row_index, include_file_paths, predicate, parquet_options, context)
728 if comment is not None:
729 options.set_comment(comment)
--> 730 tbl_w_meta = plc.io.csv.read_csv(options, stream=stream)
731 pieces.append(tbl_w_meta)
732 if include_file_paths is not None:

File ~/cudf/python/pylibcudf/pylibcudf/io/csv.pyx:673, in pylibcudf.io.csv.read_csv()

File ~/cudf/python/pylibcudf/pylibcudf/io/csv.pyx:699, in pylibcudf.io.csv.read_csv()

RuntimeError: CUDF failure at: /cudf/cpp/src/io/csv/reader_impl.cu:84: New size must be smaller
```

**Additional context**
We're getting the `slice` bounds translated to `n_rows` and `skip_rows` (10, 10) for the example.

https://github.com/rapidsai/cudf/blob/4c798ae35b908e5596c2c0ea06eb9be2d4f2717e/python/cudf_polars/cudf_polars/dsl/translate.py#L275

Which when passed to `plc.io.csv.CsvReaderOptionsBuilder.nrows/skiprows` seem to be hitting this validation error instead of returning "empty"

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.