google / google/turbinia

Add support for multi-pools for workers

Open
#17 2 comments 0 reactions 0 assignees View on GitHub
celery enhancement Prio2
Dominant language
Python
Stars
794
Forks
172
PR merge metrics
No merged PRs in 30d

Description

We can add different worker pools that are created to match individual job types (e.g. so Plaso can have worker nodes that have more cpu than workers for other job types). Right now it's one global worker pool.

Contributor guide

Open the contributing guide

Research direction

The issue names no files, tests, or entry points. Start by tracing where the single global worker pool is created and how jobs are assigned to workers. Define how job types select separate pools and what configuration and tests are needed to show that pools can use different worker resources.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.