Same Job Repeating Hundreds of Times
- Dominant language
- JavaScript
- Stars
- 9.4k
- Forks
- 858
- PR merge metrics
- No merged PRs in 30d
Description
I am seeing sporadic issues where jobs repeat hundreds of times. It will be the same job ID and I have confirmed that I'm only adding it to the queue once. What happens is that jobs will process correctly for some time and then, randomly it seems, a single job will process 400+ times and then things return to normal.
The app is on AWS and there are two instances on which Kue runs. I noticed in the docs (https://github.com/Automattic/kue#parallel-processing-with-cluster) it shows how to ensure only one instance of the web app runs by checking `cluster.isMaster` and then only starting the Kue web app on that process. While we aren't using the cluster module, we do have two separate boxes (AWS instances) and right now they both run the web app. E.g. they both run `kue.app.listen(config.port);`. Is it problematic to run that multiple times? Could that be the cause of this issue?
If not, is there anything else that might cause this?
Thanks!
Contributor guide
No contributing guide indexed for this repository
Research direction
No source file or test is identified. Start by reproducing the repeated-job behavior with Kue running on both AWS instances, then compare that setup with the documented cluster.isMaster pattern and the kue.app.listen(config.port) entry point. Done means identifying a reproducible cause and documenting a concrete fix or configuration change.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, javascript, node.js, redis
- Domain
- backend, cloud, distributed-systems
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 15/100