[Java] Support reading a subset of fields from an IPC file or stream
- 主要语言
- Java
- 星标
- 94
- 派生
- 152
- 平均合并
- 3 天 16 小时
- 30 天内合并 PR
- 11
描述
C++ supports `IpcReadOptions.included_fields` which lets you load a subset of (top-level) fields from an IPC file or stream, potentially saving on I/O costs. It would be useful to support this in Java as well. Some refactoring would be required since MessageSerializer currently reads record batch messages in as a whole, and it would be good to quantify how much of a benefit this provides in different scenarios.
**Reporter**: [David Li](https://issues.apache.org/jira/browse/ARROW-17914) / @lidavidm
**Note**: *This issue was originally created as [ARROW-17914](https://issues.apache.org/jira/browse/ARROW-17914). Please see the [migration documentation](https://github.com/apache/arrow/issues/14542) for further details.*
贡献指南
调研方向
从 MessageSerializer 开始,跟踪当前如何从 IPC 文件和流中读取 record batch 消息。将期望的行为与 C++ IpcReadOptions.included_fields 进行比较,然后定义顶层字段选择应如何工作,并量化不同场景下的 I/O 收益。
由索引模型根据 Issue 内容生成。
评估
- 技术栈
- java
- 领域
- data
- Issue 类型
- 功能
- 难度
- 5/5
- 预计耗时
- 一周以上
- 活跃度
- 停滞
- 描述清晰度
- 基本清楚
- 新手友好度
- 30/100