apache / apache/parquet-java

[CPP] Unable to read files written by parquet-cpp from parquet-tools

Open
#2,033 12 comments 0 reactions 0 assignees View on GitHub
Component: Parquet Priority: Major Type: bug
Dominant language
Java
Stars
3.1k
Forks
1.6k
Avg merge
3d 12h
Merged PRs (30d)
33

Description

I could not read files written by parquet-cpp from parquet-tools and Hive.
Setting field ids in the schema metadata seems to be the problem. We should make setting the field_id optional.

**Reporter**: [Deepak Majeti](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=mdeepak) / @majetideepak
#### Original Issue Attachments:
- [parquet_cpp_example.parquet](https://issues.apache.org/jira/secure/attachment/12849181/parquet_cpp_example.parquet)

**Note**: *This issue was originally created as [PARQUET-838](https://issues.apache.org/jira/browse/PARQUET-838). Please see the [migration documentation](https://issues.apache.org/jira/browse/PARQUET-2502) for further details.*

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reproducing the failure with the attached parquet_cpp_example.parquet file using parquet-tools and Hive, then inspect how field IDs are written in the parquet-java schema metadata. Done means files written by parquet-cpp can be read by parquet-tools and Hive when field IDs are not required.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, java
Domain
data-engineering
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.