done callback is not properly completing the job
- Dominant language
- JavaScript
- Stars
- 9.4k
- Forks
- 858
- PR merge metrics
- No merged PRs in 30d
Description
I have several jobs which definitely finish execution, and our accounting through PMX custom metrics assure us that the done callbacks are being called and that only 10 concurrent jobs are being run. Despite all of this, we have numerous cases where jobs will remain stuck in the queue as "active".
We do have cleanup routines for these, but with the time sensitivity of some jobs, it's painful to have a message stuck in the queue as active if it's not actually running.
We've added try catch wrappers, and promise catch wrappers at multiple layers to detect some other fault that would somehow call our custom metric's dec() counter but fail to recognize the done() callback.
Is it possible we're raising an exception inside of KUE with too much result data?
```
job_counter.inc();
process_job().then(function (res) {
console.log(new Date(), 'ran job');
job_counter.dec();
try {
done(null, res);
} catch(e) {
console.error('Failed to call done,',e);
done();
}
}).catch(function (err) {
try {
done(err);
} catch (e) {
console.error('Failed to call done,',e);
done();
}
});
```
concurrency is set to 10 and the job_counter is always at 10, yet we get about 1-5% of the jobs hung up in the active state and no "failed to call done" logs.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the worker's done callback path and the provided promise-based job handler, focusing on how jobs transition out of the active state when concurrency is set to 10. Reproduce the reported 1–5% stuck-job behavior and inspect the callback and result-data handling; done means completed jobs no longer remain active and the failure is covered by a regression test.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript, node.js, redis
- Domain
- backend, distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100