apache / apache/arrow

pa.Table has different length than its column

Open
#47,555 0 comments 0 reactions 0 assignees View on GitHub
Component: Python Type: bug
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 13h
Merged PRs (30d)
88

Description

### Describe the bug, including details regarding any error messages, version, and platform.

If I run the following script

```python
import pyarrow as pa
import numpy as np

tables = [
pa.table({"x": [1, 2, 3]}),
pa.table({"x": [4]}),
pa.table({"x": [5, 6]}),
]

sa = pa.concat_tables(tables).to_struct_array().combine_chunks()

offsets = [0, 0, 0, 0]
offsets[1:] = np.cumsum([len(t) for t in tables])

la = pa.ListArray.from_arrays(offsets=offsets, values=sa)

for i in range(3):
value = la[i].values
table = pa.Table.from_struct_array(value)
print(len(value), len(table), len(table["x"]))
```

I get the following output:

```
3 3 6
1 1 1
2 2 2
```

This means that for some reason, table has a different length than one of its constituent columns, which should be impossible!

I am running this using Python 3.12 + PyArrow 21 (installed via conda-forge on Fedora Linux).
### Component(s)

Python

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.