parquet-hive (and therefore Hive) depends on ParquetInputSplit constructor
- Dominant language
- Java
- Stars
- 3.1k
- Forks
- 1.6k
- Avg merge
- 3d 12h
- Merged PRs (30d)
- 33
Description
The issue is not really parquet-hive which we can modify in sync but rather Hive itself. As we want to be able to change the split implementation without breaking Hive. (Users might want to use the latest Parquet with their version of Hive)
[ParquetRecordReaderWrapper in parquet-hive](https://github.com/apache/incubator-parquet-mr/blob/647b8a70f9b7c94cabf9a7ec7bce2e7cbbb4c05b/parquet-hive/parquet-hive-storage-handler/src/main/java/org/apache/hadoop/hive/ql/io/parquet/read/ParquetRecordReaderWrapper.java#L223)
and in [Hive](https://github.com/apache/hive/blob/e58b9d273cb78bda2947148bc54f4befb2514241/ql/src/java/org/apache/hadoop/hive/ql/io/parquet/read/ParquetRecordReaderWrapper.java#L221)
**Reporter**: [Julien Le Dem](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=julienledem) / @julienledem
**Assignee**: [Julien Le Dem](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=julienledem) / @julienledem
**Note**: *This issue was originally created as [PARQUET-86](https://issues.apache.org/jira/browse/PARQUET-86). Please see the [migration documentation](https://issues.apache.org/jira/browse/PARQUET-2502) for further details.*
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the cited ParquetRecordReaderWrapper locations in parquet-hive and Hive, especially the constructor use around line 221/223. Trace how the ParquetInputSplit constructor is used in both projects and define completion as allowing the split implementation to change without breaking Hive.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 20/100