Add support for multi-pools for workers
Open
celery
enhancement
Prio2
- Dominant language
- Python
- Stars
- 794
- Forks
- 172
- PR merge metrics
- No merged PRs in 30d
Description
We can add different worker pools that are created to match individual job types (e.g. so Plaso can have worker nodes that have more cpu than workers for other job types). Right now it's one global worker pool.
Contributor guide
Research direction
The issue names no files, tests, or entry points. Start by tracing where the single global worker pool is created and how jobs are assigned to workers. Define how job types select separate pools and what configuration and tests are needed to show that pools can use different worker resources.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100