Improve scheduling algorithm for better throughput
Nobody has claimed this yet.
- Dominant language
- OCaml
- Stars
- 42
- Forks
- 20
- PR merge metrics
- No merged PRs in 30d
Description
We often see the current scheduling to result in one worker getting overloaded with a massive queue due to cache hints while other workers are starved for work.
We want the job scheduling algorithm to take cache hints into account, but also to consider worker capacity and availability. There are times where this should be able to really improve our thru put.
Related to, but more general than #168
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No source file, test, or scheduler entry point is identified in the issue. Start by reading the current scheduling implementation and related issue #168, then define how cache hints, worker capacity, and availability should be balanced; done means demonstrating improved throughput without starving or overloading workers.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- ocaml
- Domain
- ci-cd, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100