Per-user job queues instead of global job queue

Request Type

Feature Request

Work Environment

Problem Description

At the moment all jobs are being run through a single queue. If one user, e.g. a machine account of some tool runs a lot of jobs, no other user will be able to run something in parallel.

Steps to Reproduce

1. Have a one user continuously run a constant amount of jobs.
2. Have other users also wanting to run jobs on demand.
3. The other users will have to wait a long time.

Possible Solutions

My proposal is to have one job queue per user account and instead of just draining one queue, the job scheduler could go through all queues in a round-robin fashion so that all users handled equally.

Complementary information

We are using Cortex as a central API gateway for observable enhancement and therefore could really make good use of a better queuing system.

The issue names no files, tests, or entry points, so first locate the job scheduler and current global queue implementation. Done means jobs are separated by user account and scheduled fairly in round-robin fashion so one user's sustained workload does not block other users.