apache / apache/texera

Arrow tuple reading fails across different column chunk boundaries

Open
#8,209 1 comment 0 reactions 1 assignee Claimed by @carloea2 View on GitHub
Dominant language
Scala
Stars
314
Forks
187
Avg merge
1d 21h
Merged PRs (30d)
214

Description

### Feature Summary

ArrowTableTupleProvider tracks row positions using the first column's chunk layout and applies those chunk coordinates to every column. Valid Arrow tables can have different chunk boundaries per column, causing later fields to throw IndexError or read the wrong position.

Before: different column chunk boundaries -> second row throws IndexError

After: different column chunk boundaries -> rows are read by global index

Reproduction evidence:

Create a three-row Arrow table whose first column has chunk lengths 2 and 1 while its second column has chunk lengths 1 and 2. Read the second row through ArrowTableTupleProvider.

Version and commit evidence:

1.3.0-incubating-SNAPSHOT (main)

**Commit Hash**

70c21145887920528d7d5540e3fb790b43e8b759

**What browsers are you seeing the problem on?**

Not browser-specific.

**Relevant log output**

```text
chunk_layouts= [2, 2]
row1= 1 10
row2_a= 2
IndexError: index out of bounds
```

### Proposed Solution or Design

Read each Arrow column by global row index instead of reusing the first column's chunk coordinates.

### Affected Area

Workflow Engine (Amber)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.