apache / apache/datafusion-ballista
Ballista cluster sharing by multiple users (with fairness of resource allocation)
- Dominant language
- Rust
- Stars
- 2.1k
- Forks
- 320
- Avg merge
- 1d 22h
- Merged PRs (30d)
- 66
Description
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
Let's say we've an organization that deploys ballista cluster to which multiple users connect to. It'd be great if executor slots can be allocated to the users fairly. If a query has consumed all slots allotted to a user, it can't progress until more slots are generated (say by passage of time).
**Describe the solution you'd like**
Each query execution is associated with some identity, identities are rate limited for the slots and tasks for the identities are queued and consumed at allowed rate (or concurrency)
**Describe alternatives you've considered**
**Additional context**
Add any other context or screenshots about the feature request here.
Contributor guide
Research direction
The issue names no files, tests, or entry points. Start by locating Ballista's query identity, executor-slot, and task-scheduling components, then define how per-identity rate or concurrency limits and queuing should work; done means multiple users receive fair allocation while excess tasks wait.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100