Parquet check-stats command only supports checking BINARY type columns.
- Dominant language
- Java
- Stars
- 3.1k
- Forks
- 1.6k
- Avg merge
- 3d 12h
- Merged PRs (30d)
- 33
Description
Hi, community, could you tell me why parquet check-stats command only supports checking BINARY type columns?
https://github.com/apache/parquet-java/blob/5ac50884e9ee27a1ce3ee18cc5ed4422147e5387/parquet-cli/src/main/java/org/apache/parquet/cli/commands/CheckParquet251Command.java#L104
Could we add some optional parameters so that it can detect all types of columns? I just want to use this tool to know whether there is any data corruption in the parquet file(not just for BINARY type columns).
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reading parquet-cli/src/main/java/org/apache/parquet/cli/commands/CheckParquet251Command.java around line 104 to understand the current BINARY-only check. Determine how optional parameters should select column types, then verify that the command can detect corruption for non-BINARY columns as well as BINARY columns.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- cli
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100