NVIDIA / NVIDIA/cudf

[BUG] Parquet column selection by name with schemas including list<struct<X, Y>> does not work.

Open
#14,539 5 comments 0 reactions 0 assignees View on GitHub
0 - Backlog bug cuIO libcudf Python
Dominant language
C++
Stars
9.8k
Forks
1.1k
Avg merge
3d 6m
Merged PRs (30d)
278

Description

If you have a schema that contains a list-of-struct, selecting a subset of the inner columns doesn't work. Example

`list>`
If the schema for this column was

```
A (list)
B (struct)
C (int)
D (float)
```
Attempting to select "A.B.C" would not work. I believe this is being caused by some schema preprocessing that we are doing that is injecting fake schema elements to ease schema interpretation. Essentially we see a schema that looks like this:

```
A (list)
list (the fake element
B (struct)
C (int)
D (float)
```
So "A.B.C" doesn't actually exist, only "A.list.B.C" and the code returns 0 columns.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.