Blog / paper post about parquet push decoder
- Dominant language
- Rust
- Stars
- 3.6k
- Forks
- 1.3k
- Avg merge
- 2d 18h
- Merged PRs (30d)
- 169
Description
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
- Part of https://github.com/apache/arrow-rs/issues/8000
- Related to https://github.com/apache/arrow-rs/issues/7983
**Describe the solution you'd like**
I would like to explain the benefits of a push decoder in a more digestable format than a PR (even if the code is well documented :bowtie: )
I think this will also help reviewers
**Describe alternatives you've considered**
Idea is to write a blog post about the push parquet decoder, and how it can be used to offer more fine grained control over IO and CPU work in the parquet reader.
The blog post would cover:
1. Motivation: why do we need a push decoder?
2. Design: how does the push decoder work?
3. Examples: how to use the push decoder in practice
4. Performance: how does the push decoder perform compared to the existing parquet reader?
5. Future work: what are the next steps for the push decoder?
**Additional context**
Contributor guide
Assessment
This issue has not been assessed yet.