apache / apache/parquet-java

parquet-pig should not throw an exception reading a timestamp from impala file

Open
#1,741 4 comments 0 reactions 0 assignees View on GitHub
Component: Parquet Priority: Blocker Type: bug
Dominant language
Java
Stars
3.1k
Forks
1.6k
Avg merge
3d 12h
Merged PRs (30d)
33

Description

Hi, I need to load impala table with timestamp columns with pig. I think loading timestamp cols is still a NYI feature. Waiting for [#218](https://github.com/Parquet/parquet-mr/issues/218) TBD, pig-parquet should not throw an exception but at least should bind Int96 into a bytearray (users can write their own UDF to convert it into pig datetime type).
Is there anyone working on that?

**Reporter**: [gpolaert](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=gpolaert)

**Note**: *This issue was originally created as [PARQUET-195](https://issues.apache.org/jira/browse/PARQUET-195). Please see the [migration documentation](https://issues.apache.org/jira/browse/PARQUET-2502) for further details.*

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by tracing parquet-pig's timestamp reader with an Impala file containing Int96 columns, and review issue #218 for the pending timestamp context. Done means reading the file no longer throws an exception and exposes Int96 as a bytearray for downstream UDF handling.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
data-engineering
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.