apache / apache/arrow-java

[Java] Inconsistent logic for type IDs in Union vectors

未关闭
#255 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
Type: bug
主要语言
Java
星标
94
派生
152
平均合并
3 天 16 小时
30 天内合并 PR
11

描述

The current logic for calculating the type IDs in UnionVector#getField and DenseUnionVector#getField is:
1. DenseUnionVector uses an increasing counter 
1. UnionVector uses the ordinal of the type enum
1. Both completely ignore the type IDs provided at construction as part of fieldType (if provided)

We encountered this inconsistency while testing a direct roundtrip of a union vector between pyarrow and Java with the C Data Interface ('direct' here means without using VectorSchemaRoot/RecordBatch). The identifiers for the type IDs differ after completing a roundtrip. 

 

**Reporter**: [Roee Shlomo](https://issues.apache.org/jira/browse/ARROW-14271) / @roee88

**Note**: *This issue was originally created as [ARROW-14271](https://issues.apache.org/jira/browse/ARROW-14271). Please see the [migration documentation](https://github.com/apache/arrow/issues/14542) for further details.*

贡献指南

打开贡献指南

调研方向

从 UnionVector#getField 和 DenseUnionVector#getField 开始,然后跟踪构造过程中如何处理 fieldType 及其提供的类型 ID。复现直接的 pyarrow 到 Java C Data Interface roundtrip,并验证两种向量类型在 roundtrip 后都保留一致的类型 ID。

由索引模型根据 Issue 内容生成。

评估

技术栈
java
领域
data-engineering
Issue 类型
缺陷
难度
4/5
预计耗时
3-5 天
活跃度
停滞
描述清晰度
基本清楚
新手友好度
35/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。