codingjoe / codingjoe/threadmill

Silently dropped task ids when the task hash is missing in acquire.lua and mover.lua

Offen
#50 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
Vorherrschende Sprache
Python
Sterne
12
Forks
1
Ø Merge
1 T. 1 Std.
Gemergte PRs (30 T.)
10

Beschreibung

Split out of #48 during the fair multi-queue scheduling change (`codingjoe-fair-multi-queue-scheduling`).

`defer: Silent task loss when a ready/deferred set member's task hash is gone (acquire.lua drops the id, mover.lua drops a due id whose score is gone) with no result row written. Separate durability bug; the sets have no TTL. [threadmill/backends/lua/acquire.lua, threadmill/backends/lua/mover.lua]`

## Problem

`ZPOPMIN` removes the member before the data hash is read, so when `HGET` returns nothing (hash gone, or `cjson.decode` fails) the loop moves on and the task id is left nowhere: not in the ready set, not in the running set, and no result row is written.

`mover.lua` has the same shape: it drops a due id from the deferred set when `HGET score` returns nothing, again without a result row.

The ready and deferred sets have no TTL, while the task hash expires after `lease_ttl * 3 + result_ttl` (27 h with the defaults), so a backlog or a worker outage longer than that silently loses tasks instead of failing them. That conflicts with the project's Durability and Consistency design principles.

## Open questions

- Should the expired hash be reported as a FAILED result (with what payload, given the data is gone), or should the ready/deferred sets be bounded or refreshed instead?
- Is a synthetic result row even deserializable by `peek()` and the inspector, which expect a serialized `TaskResult`?

## QED

1. Enqueue a task.
2. Delete its hash key (`{prefix}:task:{id}`) while it stays in the ready set.
3. `acquire()` — the id vanishes with no trace.

Beitragsleitfaden

Beitragsleitfaden öffnen

Rechercherichtung

Start by reading threadmill/backends/lua/acquire.lua and threadmill/backends/lua/mover.lua, then reproduce the QED case by deleting a task hash while its id remains queued. Review how result rows are serialized and consumed by peek() and the inspector. The issue is done only after the failure policy is decided, task ids are no longer silently lost, and regression coverage verifies the chosen behavior.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Bewertung

Tech-Stack
lua, redis
Bereich
backend, distributed-systems
Issue-Typ
Bug
Schwierigkeit
5/5
Geschätzter Aufwand
Über eine Woche
Aktivitätsstatus
Aktiv
Klarheit
Muss geklärt werden
Anfängerfreundlichkeit
35/100

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.