apache / apache/parquet-java

make it easy to read and write parquet files in java without depending on hadoop

Open
#1,497 6 comments 3 reactions 0 assignees View on GitHub
Component: Parquet Priority: Major Type: enhancement
Dominant language
Java
Stars
3.1k
Forks
1.6k
Avg merge
3d 12h
Merged PRs (30d)
33

Description

I am happy to help with this but I'd love some guidance on:

1) likelihood of being accepted as a patch.
2) how critical it is to maintain backwards compatibility in APIs.

For instance, we probably want to introduce a new artifact that lives under the existing hadoop depending artifact, and move as much code as possible to that, keeping the hadoop apis in the old artifact.

Welcome comments on solving this issue.

**Reporter**: [Oscar Boykin](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=posco)
#### Related issues:
- [Make ParquetIO Read splittable](https://issues.apache.org/jira/browse/BEAM-4379) (blocks)
- [hadoop-common is not an optional dependency](https://github.com/apache/parquet-java/issues/2556) (is duplicated by)
- [Avoid leaking Hadoop API to downstream libraries](https://github.com/apache/parquet-java/issues/2097) (incorporates)
- [Add Java NIO Avro OutputFile InputFile](https://github.com/apache/parquet-java/issues/2447) (is related to)
#### PRs and other links:
- [GitHub Pull Request #1376](https://github.com/apache/parquet-java/pull/1376)

**Note**: *This issue was originally created as [PARQUET-1126](https://issues.apache.org/jira/browse/PARQUET-1126). Please see the [migration documentation](https://issues.apache.org/jira/browse/PARQUET-2502) for further details.*

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reviewing pull request #1376 and the related issues on Hadoop dependencies, API leakage, and Java NIO support. The issue proposes a Hadoop-independent artifact while preserving compatibility in the existing artifact, but it does not define a settled design or acceptance criteria; confirm the current status before beginning.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
data-engineering
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.