parquet-pig should not throw an exception reading a timestamp from impala file
- 主要语言
- Java
- 星标
- 3.1k
- 派生
- 1.6k
- 平均合并
- 3 天 12 小时
- 30 天内合并 PR
- 33
描述
Hi, I need to load impala table with timestamp columns with pig. I think loading timestamp cols is still a NYI feature. Waiting for [#218](https://github.com/Parquet/parquet-mr/issues/218) TBD, pig-parquet should not throw an exception but at least should bind Int96 into a bytearray (users can write their own UDF to convert it into pig datetime type).
Is there anyone working on that?
**Reporter**: [gpolaert](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=gpolaert)
**Note**: *This issue was originally created as [PARQUET-195](https://issues.apache.org/jira/browse/PARQUET-195). Please see the [migration documentation](https://issues.apache.org/jira/browse/PARQUET-2502) for further details.*
贡献指南
这个仓库没有索引到贡献指南
调研方向
首先,使用包含 Int96 列的 Impala 文件跟踪 parquet-pig 的时间戳读取器,并查看 issue #218,了解待处理的时间戳上下文。完成的标准是:读取该文件时不再抛出异常,并将 Int96 以 bytearray 的形式暴露给下游 UDF 处理。
由索引模型根据 Issue 内容生成。
评估
- 技术栈
- java
- 领域
- data-engineering
- Issue 类型
- 缺陷
- 难度
- 4/5
- 预计耗时
- 3-5 天
- 活跃度
- 停滞
- 描述清晰度
- 基本清楚
- 新手友好度
- 35/100