apache / apache/parquet-format
Why the logic of ParquetFileWrite is diff from the parquet format
- Dominant language
- Thrift
- Stars
- 2.6k
- Forks
- 508
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 4
Description
Hi parquet-mr teams,
When I reading the parquet writer in [ParquetFileWriter]( I find that there is no `column metadata` behind the each column chunk described in the [Parquet-Format] ( and [thrift definition](
**Reporter**: [Kun Liu](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=liukun) / @liukun4515
**Note**: *This issue was originally created as [PARQUET-2162](https://issues.apache.org/jira/browse/PARQUET-2162). Please see the [migration documentation](https://issues.apache.org/jira/browse/PARQUET-2502) for further details.*
Contributor guide
Research direction
Start at ParquetFileWriter.java around line 916, then compare the writer behavior with the Parquet-Format file-format documentation and the parquet.thrift definition around line 790. Determine whether the missing column metadata is an implementation defect or an intentional format detail, and document the finding or resulting correction.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100