apache / apache/arrow

[Python] RecordBatch.from_struct_array(array_of_lists_of_structs[i].values) needs a test

Open
#48,182 8 comments 0 reactions 1 assignee Claimed by @Gnananjali View on GitHub
Component: Python good-first-issue Type: bug
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 13h
Merged PRs (30d)
88

Description

### Describe the bug, including details regarding any error messages, version, and platform.

RecordBatch.from_struct_array(array_of_lists_of_structs[i].values) returns a RecordBatch that is somewhat corrupt and seems to contain all structs from the array (as if the lists had been flattened), not just the rows from the ith sub-list.

Example code:

```python
import pyarrow

struct_type = pyarrow.struct([("x", pyarrow.int64()), ("y", pyarrow.int64())])

list0 = [{"x": 1, "y": 2}, {"x": 3, "y": 4}]
list1 = [{"x": 5, "y": 6}]
list2 = [{"x": 7, "y": 8}, {"x": 9, "y": 10}]

list_of_structs = pyarrow.array([list0, list1, list2], type=pyarrow.list_(struct_type))

# Get the first list - should be [{x:1,y:2}, {x:3,y:4}]
elem0 = list_of_structs[0]
batch_for_elem0 = pyarrow.RecordBatch.from_struct_array(elem0.values)

print("ver", pyarrow.__version__)
print("RIGHT:", len(elem0.values), elem0.values.field("x"))

print("WRONG?:", len(batch_for_elem0), batch_for_elem0["x"])
```

Output:

```
ver 22.0.0
RIGHT: 2 [
1,
3
]
WRONG?: 2 [
1,
3,
5,
7,
9
]
```

### Component(s)

Python

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.