apache / apache/arrow-java

[Java] Avro to Arrow converter doesn't appear to generate valid arrow data

未关闭
#383 1 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
Type: bug
主要语言
Java
星标
94
派生
152
平均合并
3 天 16 小时
30 天内合并 PR
11

描述

I think this is related to how Unions are handled (I had thought unions of with a null and one other type would get created to the nullable type, but that is a separate issue).

 

I haven't had time to fully diagnose, but remnants of the code I tried to use are at

 

And the CSV file from https://issues.apache.org/jira/browse/ARROW-11629?jql=text%20~%20%22arrow%20drill%20parquet%20dictionary%22

 

produce data that isn't readable by the C++ implementation.

**Reporter**: [Micah Kornfield](https://issues.apache.org/jira/browse/ARROW-12340) / @emkornfield

**Note**: *This issue was originally created as [ARROW-12340](https://issues.apache.org/jira/browse/ARROW-12340). Please see the [migration documentation](https://github.com/apache/arrow/issues/14542) for further details.*

贡献指南

打开贡献指南

调研方向

从 Java Avro-to-Arrow 转换器开始,使用 ARROW-11629 中引用的 CSV 文件重现该问题。检查链接 gist 中的残留内容,重点关注 union 处理,然后将生成的数据与 C++ 实现所接受的数据进行比较。当转换器生成可由 C++ 实现读取的 Arrow 数据,并为失败的输入添加回归测试时,即视为完成。

由索引模型根据 Issue 内容生成。

评估

技术栈
java
领域
data-engineering
Issue 类型
缺陷
难度
4/5
预计耗时
3-5 天
活跃度
停滞
描述清晰度
需要澄清
新手友好度
35/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。