docling-project / docling-project/docling-jobkit
fix(rq): watchdog cleanup calls abstract StartedJobRegistry.remove
- Dominant language
- Python
- Stars
- 36
- Forks
- 47
- Avg merge
- 4d 22h
- Merged PRs (30d)
- 15
Description
## AI disclosure
I encountered this issue in my deployment. This issue report was prepared with assistance from AI.
## Description
The RQ watchdog cleanup path introduced for #106 calls `StartedJobRegistry.remove(task_id)`. In current RQ 2.x, `StartedJobRegistry.remove()` is intentionally unimplemented and raises `NotImplementedError`.
With `docling-jobkit==3.4.0` and `rq==2.11.0`, an orphaned worker is detected and the task is published as `FAILURE`, but cleanup then fails:
```text
Failed to clean up RQ state for task ...
NotImplementedError
```
The job remains in `rq:wip:{queue}`, so the watchdog can rediscover it and emit repeated warnings.
## Deterministic reproduction
The underlying RQ failure can be reproduced without Redis:
```bash
uv run python -c "from rq.registry import StartedJobRegistry; StartedJobRegistry(name='convert', connection=None).remove('repro-task')"
```
This raises `NotImplementedError` because `StartedJobRegistry.remove()` is abstract in RQ 2.x.
## Watchdog reproduction
1. Run `docling-jobkit 3.4.0` with RQ 2.x.
2. Let a `SimpleWorker` job lose its heartbeat, or reproduce the watchdog branch with a missing heartbeat key and a started-job registry entry.
3. Wait for `_watchdog_task()` to process the orphaned task.
4. Observe the `StartedJobRegistry.remove()` `NotImplementedError` in the cleanup traceback.
## Expected behavior
The watchdog should set the RQ job to `FAILED` and remove its started execution entry without raising. Missing or already-expired jobs should remain a non-fatal cleanup case.
## Proposed fix
Use the RQ execution-aware cleanup API, such as `StartedJobRegistry.remove_executions(job)`, after fetching the job, and add a regression test covering the cleanup path. This is a follow-up to the implementation released for #106.
## Environment
- docling-jobkit 3.4.0
- rq 2.11.0
- Python 3.12 or 3.13
Contributor guide
Research direction
Start at the watchdog entry point `_watchdog_task()` and reproduce the underlying RQ failure with the provided `uv run python` command. Trace the orphan cleanup path and add a regression test for it; done means the job is marked `FAILED`, its started execution entry is removed, and missing or expired jobs do not raise.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 78/100