gresrun / gresrun/jesque

Job moved from inflight queue to comma-joined queue on worker shutdown in RESET_TO_HIGHEST_PRIORITY strategy.

Open
#204 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Java
Stars
630
Forks
132
PR merge metrics
No merged PRs in 30d

Description

I have noticed that in certain scenarios (immediate worker shutdown), jobs are removed from the inflight queue and then moved to another queue.

Consider the case where I have the following queues: q1, q2, q3, q4.
My workers are meant to process jobs from these queues.
WorkerPoolImpl::getNextQueue() returns a string of "q1,q2,q3,q4" (for RESET_TO_HIGHEST_PRIORITY) and this is stored in curQueue variable.

During pop, the lua script (fromMultiplePriorityQueues.sha) pops from a single queue and returns the job. But when the worker is shutting down, the jobs are removed from the inflight queue and pushed to a queue "q1,q2,q3,q4". resulting in a new key on redis "namespace:queue:q1,q2,q3,q4". This code is in WorkerPoolImpl::removeInFlight.

What is the purpose of storing the jobs in this comma-joined queue? From what I could see, no worker would poll from this queue as the lua script would always pop from the individual queues.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with WorkerPoolImpl::getNextQueue() and WorkerPoolImpl::removeInFlight(), then inspect fromMultiplePriorityQueues.sha to compare the queue used for polling with the queue used during shutdown requeueing. Reproduce the RESET_TO_HIGHEST_PRIORITY scenario with q1, q2, q3, and q4, and confirm that shutdown does not leave jobs in an unpolled Redis key such as namespace:queue:q1,q2,q3,q4.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, redis
Domain
backend, distributed-systems
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.