dragonflydb / dragonflydb/dragonfly

BullMQ test failure: failed assertion on job completion and "Missing lock" due to Lua script key access

Open
#6,629 7 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
31.5k
Forks
1.3k
Avg merge
1d 10h
Merged PRs (30d)
137

Description

The bullmq-tests workflow is failing during the Job .finished test suite. The test expects a job to be removed upon completion (removeOnComplete), but the job remains in the database, causing the promise to resolve with a value instead of rejecting with a "Missing key" error.

```
Workflow File: .github/workflows/bullmq-tests.yml
Node Version: 22.12.0
```

Failure link:
https://github.com/dragonflydb/dragonfly/actions/runs/22031603203/job/63657569445

Observed Failures:

1. Assertion Error in Job .finished
The test "Job .finished when job was added with removeOnComplete rejects with missing key for job message" fails with an assertion error. The job hash was not deleted from the database.

This is where the job is added (see tests/test_job.ts in bullmq repository):
```
const job = await queue.add(
'test',
{ foo: 'bar' },
{ removeOnComplete: true },
);
```

This is where the assertion fail in the same file:
```
await expect(job.waitUntilFinished(queueEvents)).to.be.rejectedWith(
`Missing key for job ${queue.toKey(job.id)}. isFinished`,
);
```

2. Missing Lock Errors
Multiple errors regarding missing locks appear in the logs during execution, specifically referencing moveToFinished.

Error: Missing lock for job repeat:test:1486459440000. moveToFinished
at Scripts.finishedErrors ...

More Technical Context on both issues:
* See: https://docs.bullmq.io/guide/troubleshooting#missing-locks
* Also consider running Dragonfly with `-default_lua_flags=allow-undeclared-keys` due to the fact that BullMQ Lua scripts generate and access keys dynamically that are not explicitly passed in the KEYS array, while Dragonfly enforces strict Lua key declaration by default. In other words, Dragonfly must know exactly which keys a script will access before it runs and it enforces "strict key deceleration" by default". But BULLMQ might violate this rule and therefore we need to enable that flag.

Contributor guide

Open the contributing guide

Research direction

Start with .github/workflows/bullmq-tests.yml and the Job .finished case in tests/test_job.ts; reproduce the failure on Node 22.12.0 and inspect the moveToFinished missing-lock logs. Compare Dragonfly’s default Lua key policy with the suggested -default_lua_flags=allow-undeclared-keys setting; done means the assertion and missing-lock failures no longer occur.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, lua, node.js
Domain
databases, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.