Test times out due to failed lock acquisition (lock TTL == test timeout)
- Dominant language
- JavaScript
- Stars
- 9.4k
- Forks
- 858
- PR merge metrics
- No merged PRs in 30d
Description
I'm running a customized version of node based on node v0.12.7.
The version of node I'm using modifies the timing of events a bit, but is otherwise a faithful implementation of node v0.12.7.
My machine is Ubuntu 14.04.3.
I took the following steps:
- clone repo
- npm install
- npm install --only=dev
I then tried to run the test suite, but it reliably (23/30 attempts) fails when I run "npm test".
It fails with a timeout on the test case 'should honor original delay at fixed backoff'.
I inserted some console.log() to try to find the root cause.
Here's what I could find:
- Logs in lib/queue/job.js report that the job is registered and its state is set to delayed.
- In lib/kue.js Queue.prototype.checkJobPromotion, the warlock lock acquisition fails for the entire duration of the test. I note that the default test timeout is 2000 ms (2 seconds), and this is the same as the lockTtl in checkJobPromotion.
If I change lockTtl in lockTtl to 1000 ms, the lock is eventually acquired after about 1200 ms, but then the assert in the test case about test case time fails ((now - start).should.be.approximately(400,120)).
If I add 'this.timeout 0' to the test case, the lock is eventually acquired after 2300 ms, but the same timing-sensitive assert fails.
The question is, why do we need to wait until the previous holder's lock expires to acquire it?
I added logs in warlock.lock and the logs appear to be being dropped appropriately by the previous test case. However, if I remove test case 'should be re tried after failed attempts', the entire test suite passes on 10/10 tries.
If I use a 'vanilla' version of v0.12.7, the test suite passes without a problem. I suspect a race condition.
Contributor guide
No contributing guide indexed for this repository
Research direction
Run npm test and focus on the test case 'should honor original delay at fixed backoff'. Read lib/queue/job.js and Queue.prototype.checkJobPromotion in lib/kue.js, then inspect the related warlock.lock calls and the preceding 'should be re tried after failed attempts' test. Done means the suite no longer hangs on lock acquisition and the timing assertion passes reliably.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript, node.js
- Domain
- backend, testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100