apache / apache/datafusion-ballista
Design: Implement bubble execution mode.
- Dominant language
- Rust
- Stars
- 2.1k
- Forks
- 320
- Avg merge
- 1d 22h
- Merged PRs (30d)
- 66
Description
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
Currently ballista uses schedules stage by stage, which is great for fault tolerance, but inefficient for adhoc olap execution. Bubble execution mode is to make ballista more efficient for adhoc olap queries which doesn't require fault tolerance.
**Describe the solution you'd like**
The detailed design can be found [here](https://docs.google.com/document/d/1OdAe078axk4qO0ozUxNqBMD4wKoBhzh9keMuLp_jerE/edit?usp=sharing), welcome to comment and share your thoughts.
**Describe alternatives you've considered**
No.
**Additional context**
No.
Contributor guide
Research direction
Start with the linked detailed design document and compare it with Ballista's current stage-by-stage scheduling approach. The issue names no files or tests; completion would mean implementing the proposed bubble execution mode for ad hoc OLAP queries while preserving the stated trade-off of not requiring fault tolerance.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- data-engineering, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100