[BUG] hybrid scan build_row_mask_with_page_index_stats should not fail if page index isn't found
- Dominant language
- C++
- Stars
- 9.8k
- Forks
- 1.1k
- Avg merge
- 3d 6m
- Merged PRs (30d)
- 278
Description
**Describe the bug**
I am trying to load a file with the hybrid reader, and when I call `build_row_mask_with_page_index_stats` it fails with:
```
cudf/cpp/src/io/parquet/experimental/page_index_filter.cu:853: Page pruning requires the Parquet page index for all output columns
```
I believe, instead we should ignore this and return an "all rows" mask like: https://github.com/rapidsai/cudf/issues/20832. Alternatively, if there is a world where if one column has a page index, that all must, we could make that the check "column A had the page index, but columns B, C, D... didn't". I do not know if this is the case.
**Steps/Code to reproduce bug**
Try to read a parquet file without page indices with the hybrid reader.
**Expected behavior**
An "all rows" mask is what I would expect. In spark, the filtering done in parquet is opportunistic (it's done to reduce IO). Spark always adds a filter after the scan, to actually filter the rows it wants, so if we get all rows from this step, we would be OK.
Contributor guide
Assessment
This issue has not been assessed yet.