apache / apache/arrow

[C++][Engine] Substrait consumer silently drops modern FetchRel and AggregateRel fields

Open
#50,634 0 comments 0 reactions 1 assignee Claimed by @fangchenli View on GitHub
Component: C++ Type: bug
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 13h
Merged PRs (30d)
88

Description

### Describe the bug, including details regarding any error messages, version, and platform.

Arrow's Substrait consumer reads relation fields through deprecated accessors from a vendored Substrait proto pinned to v0.44.0, so values that current producers such as DataFusion send through the newer forms introduced around v0.63.0 are silently dropped and defaulted, producing wrong results with no error.

FetchRel limits arrive via count_expr/offset_expr while Arrow reads the unset scalar count/offset, turning every LIMIT n into LIMIT 0 (empty result) and ignoring OFFSET.

AggregateRel grouping keys arrive via expression_references while Arrow reads the deprecated inline field, collapsing every row into a single group.

We should bump the vendored proto to v0.63.0 (matching DataFusion's pinned substrait-rs, where the new forms coexist with the still-readable deprecated arms) and updates the FetchRel and AggregateRel consumers to read the modern forms with a fallback for older producers.

### Component(s)

C++

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.