parquet-cli rewrite option
- Dominant language
- Java
- Stars
- 3.1k
- Forks
- 1.6k
- Avg merge
- 3d 12h
- Merged PRs (30d)
- 33
Description
### Describe the usage question you have. Please include as many useful details as possible.
Hi ,
is it possible to read directly from a gcs bucket to prune a column
like rewrite -i gs:/sourcebbucket/part-00549.parquet -o gs://targetbucket/newdata/dd --prune-columns col4
i am getting error
java.lang.RuntimeException: org.apache.hadoop.fs.UnsupportedFileSystemException: No FileSystem for scheme "gs"
### Component(s)
_No response_
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the parquet-cli rewrite command and reproduce the reported gs:// input and output failure. Trace how rewrite resolves filesystem schemes, then check the Hadoop filesystem setup used by the CLI. Done means the requested GCS prune-columns workflow is supported or its limitation is clearly documented, with the error behavior covered by a test if the project provides one.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- google-cloud, java
- Domain
- cli, cloud
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100