[EPIC] [Parquet] Implement new Parquet encodings in arrow-rs
- Dominant language
- Rust
- Stars
- 3.6k
- Forks
- 1.3k
- Avg merge
- 2d 16h
- Merged PRs (30d)
- 168
Description
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
The parquet community is considering adding new encodings to the format. Part of adding new encodings is evaluating them on both real world datasets and ensuring they are efficiently implementable in multiple languages
Clearly, I would like Rust to be one of those languages
This epic tracks the work for adding new encodings to Arrow-rs if/when they are added to the parquet spec
- [x] https://github.com/apache/arrow-rs/issues/8748
- [ ] https://github.com/apache/arrow-rs/issues/8749
Contributor guide
Research direction
This is an epic tracking new Parquet encodings in arrow-rs, with completed work in issue 8748 and remaining work in issue 8749. Start by reading issue 8749 and the Parquet specification context it references. Done means the remaining encoding work is implemented and the associated evaluation is complete.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100