apache / apache/arrow

[C++] Add ordering information to exec batches

Open
#32,991 2 comments 0 reactions 1 assignee Claimed by @westonpace View on GitHub
Component: C++ Type: enhancement
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 13h
Merged PRs (30d)
88

Description

I proposed this ages ago in https://lists.apache.org/thread/xpn9gyrs6kqc3g9t8k4ts8dmy7yyxskq and am finally getting around to implementing it.

I propose to add ordering information to exec nodes (mostly for node validation) and indices to exec batches. This is a fundamental step to allow nodes to consume this ordering information to achieve features such as ARROW-10883 and ARROW-16628. It also can replace the complicated batch enumeration in the current scanner to support in-order table reassembly.

**Reporter**: [Weston Pace](https://issues.apache.org/jira/browse/ARROW-17762) / @westonpace
**Assignee**: [Weston Pace](https://issues.apache.org/jira/browse/ARROW-17762) / @westonpace
**Watchers**: [Rok Mihevc](https://issues.apache.org/jira/browse/ARROW-17762) / @rok
#### Related issues:
- [[C++][Dataset] Preserve order when writing dataset](https://github.com/apache/arrow/issues/26818) (is required by)
#### PRs and other links:
- [GitHub Pull Request #14158](https://github.com/apache/arrow/pull/14158)

**Note**: *This issue was originally created as [ARROW-17762](https://issues.apache.org/jira/browse/ARROW-17762). Please see the [migration documentation](https://github.com/apache/arrow/issues/14542) for further details.*

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.