dragonflydb / dragonflydb/dragonfly
BullMQ test failure: failed assertion on job completion and "Missing lock" due to Lua script key access
- Dominant language
- C++
- Stars
- 31.5k
- Forks
- 1.3k
- Avg merge
- 1d 10h
- Merged PRs (30d)
- 137
Description
The bullmq-tests workflow is failing during the Job .finished test suite. The test expects a job to be removed upon completion (removeOnComplete), but the job remains in the database, causing the promise to resolve with a value instead of rejecting with a "Missing key" error.
```
Workflow File: .github/workflows/bullmq-tests.yml
Node Version: 22.12.0
```
Failure link:
https://github.com/dragonflydb/dragonfly/actions/runs/22031603203/job/63657569445
Observed Failures:
1. Assertion Error in Job .finished
The test "Job .finished when job was added with removeOnComplete rejects with missing key for job message" fails with an assertion error. The job hash was not deleted from the database.
This is where the job is added (see tests/test_job.ts in bullmq repository):
```
const job = await queue.add(
'test',
{ foo: 'bar' },
{ removeOnComplete: true },
);
```
This is where the assertion fail in the same file:
```
await expect(job.waitUntilFinished(queueEvents)).to.be.rejectedWith(
`Missing key for job ${queue.toKey(job.id)}. isFinished`,
);
```
2. Missing Lock Errors
Multiple errors regarding missing locks appear in the logs during execution, specifically referencing moveToFinished.
Error: Missing lock for job repeat:test:1486459440000. moveToFinished
at Scripts.finishedErrors ...
More Technical Context on both issues:
* See: https://docs.bullmq.io/guide/troubleshooting#missing-locks
* Also consider running Dragonfly with `-default_lua_flags=allow-undeclared-keys` due to the fact that BullMQ Lua scripts generate and access keys dynamically that are not explicitly passed in the KEYS array, while Dragonfly enforces strict Lua key declaration by default. In other words, Dragonfly must know exactly which keys a script will access before it runs and it enforces "strict key deceleration" by default". But BULLMQ might violate this rule and therefore we need to enable that flag.
Contributor guide
Research direction
Start with .github/workflows/bullmq-tests.yml and the Job .finished case in tests/test_job.ts; reproduce the failure on Node 22.12.0 and inspect the moveToFinished missing-lock logs. Compare Dragonfly’s default Lua key policy with the suggested -default_lua_flags=allow-undeclared-keys setting; done means the assertion and missing-lock failures no longer occur.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, lua, node.js
- Domain
- databases, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100