Short-term roadmap for this implementation

Open
#34 5 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
25/100
Issue type
Feature
Clarity
Needs clarification
Activity status
Stale
Tech stack
rust
Domain
databases

Research direction

Start by reviewing the previous discussion in issue #4707 and the unchecked roadmap items linked from this issue, including benchmarking, file metadata and statistics, pruning, and encryption. The issue does not name implementation files or tests; it is complete when the project agrees on priorities and converts the selected work into scoped, actionable issues.

Written by the indexing model from the issue text.

Description

documentation

Previous discussion: https://github.com/apache/arrow-datafusion/issues/4707

Though the ORC format is not as widely used as parquet in arrow-rs and datafusion related projects, there are still some (growing, to my feelings) interesting and requirements on this format. As @Jefffrey said here, a noticeable and viable milestone for this project is it can be merged into arrow-rs. This draft roadmap is raised to help us discuss, arrange and take our efforts toward that milestone.

Given the ORC format is less complex than parquet, there are still many work to do in various aspects. Here is a list of functionalities need to be done if we consider making ORC files queriable from datafusion as the primary use case on this stage. Please feel free to add/remove/set priorities to them. It's likely that we can't finish all of them in a short term, thus marking what are going to be done is also important.

The below are also related but with lower priorities

Long term items:

  • encryption

Then something I'm not sure about. Looking for more information. Also feel free to change previous two lists.

Dominant language
Rust
Stars
30
Forks
26
Avg merge
19m
Merged PRs (30d)
3

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from datafusion-contrib/orc-rust

All issues in datafusion-contrib/orc-rust

Similar issues

More Rust issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.