[BUG] PQ PageIndex filter columns mis-resolve
- Dominant language
- C++
- Stars
- 9.8k
- Forks
- 1.1k
- Avg merge
- 3d 6m
- Merged PRs (30d)
- 278
Description
**Describe the bug**
Follow-up to #22802, which was fixed for row-group statistics filtering in #22803. This issue tracks the same defect in the page-level statistics filter used by the experimental hybrid-scan reader.
`page_stats_caster` in `cpp/src/io/parquet/experimental/page_index_filter.cu` resolves a column’s `schema_idx` using only source 0 and reuses that index for all sources:
- `page_stats_caster::compute_host_data()` matches `col.schema_idx == schema_idx` using the unmapped index for every source, and reads timestamp scale from `per_file_metadata[0].schema[schema_idx]`.
- The dispatch sites pass a single source-0 schema index, e.g. `output_column_schemas[col_idx]` / `output_column_schemas.front()`, into the caster.
**Why this cannot be reproduced today**
1. No public API currently feeds multiple sources into the page filter.
2. The internal implementation is not linkable from tests.
**Expected behavior**
Page-stats filtering should map the source-0 schema index into each source’s schema using `aggregate_reader_metadata::map_schema_index(schema_idx, src_idx)`, and use the per-source mapped index for both the column-chunk match and per-source schema lookup, including `ts_scale`. This should mirror the `row_group_stats_caster` fix in #22803.
**Proposed fix**
In `cpp/src/io/parquet/experimental/page_index_filter.cu`, precompute `per_source_schema_indices` using `map_schema_index(schema_idx, src_idx)` for each source, and pass them into `page_stats_caster::compute_host_data` and `page_stats_to_row_mask_converter`. Use the mapped index instead of the raw `schema_idx` in the `find_if` match and the schema lookup, similar to the fix in `cpp/src/io/parquet/predicate_pushdown.cpp`.
**Environment overview**
cudf container
**Additional context**
> I think we also need this change in `page_index_filter.cu` where we filter based on page stats. Subclasses that derive from the same `stats_caster_base` base class. Perhaps a follow up PR.
_Originally posted by @mhaseeb123 in https://github.com/rapidsai/cudf/pull/22803#discussion_r3416827230_
When the fix lands, also add a regression test against the public multi-source page-filter API.
Contributor guide
Assessment
This issue has not been assessed yet.