Support Apache ORC File Format, and Use Sparse Index
Open
enhancement
- Dominant language
- Rust
- Stars
- 9.3k
- Forks
- 2.4k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 344
Description
1. Support a new file format: [apache orc](https://orc.apache.org/)
2. Use sparse index(page level, row group level) in orc to speed up.
Contributor guide
Research direction
Start by reading the Apache ORC documentation linked in the issue and inspect DataFusion's existing file-format support. Determine the scope for reading ORC files and applying sparse page-level or row-group indexes, then define completion as ORC support working with the requested index-based speedup.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- data-engineering, databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100