apache / apache/arrow

[C++][Dataset] Filtering dataset that has column with StringView type gives empty table as result

Open
#39,637 0 comments 0 reactions 0 assignees View on GitHub
Component: C++ Critical Fix Type: bug
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 13h
Merged PRs (30d)
88

Description

### Describe the bug, including details regarding any error messages, version, and platform.

Not a C++ reproducer, but with very preliminary (non-merged) Python bindings to illustrate it:

```
import pyarrow as pa
import pyarrow.dataset as ds

# create IPC file with StringView column
builder = pa.lib.StringViewBuilder()
builder.append("test")
builder.append("some long string that is not inlined")
builder.append(None)
arr = builder.finish()

table = pa.table({"a": [1, 2, 3], "b": arr})
ds.write_dataset(table, "test_string_view", format="ipc")

# read back as a dataset
dataset = ds.dataset("test_string_view/", format="ipc")
```

The dataset looks good, and can be read as a whole:

```
>>> dataset.schema
a: int64
b: string_view

>>> dataset.to_table()
pyarrow.Table
a: int64
b: string_view
----
a: [[1,2,3]]
b: [["test","some long string that is not inlined",null]]
```

But when reading it with a filter on the other integer column, it silently gives an empty table (I would have expected it to raise an error because filter/take is not yet implemented for string_view):

```
>>> dataset.to_table(filter=ds.field('a') > 1)
pyarrow.Table
a: int64
b: string_view
----
a: [[]]
b: [[]]
```

This doesn't seem to happen if the string array only has inlined strings (then I get the expected error).

### Component(s)

C++

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.