antirez / antirez/disque

[Suggestion] Move jobs between queues, change the job body, custom metadata for a better failure handling

Offen
#174 2 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
Vorherrschende Sprache
C
Sterne
8.1k
Forks
532
PR-Merge-Kennzahlen
Keine gemergten PRs in 30 T.

Beschreibung

Hi,
this suggestion is connected to #170 in that both concern the handling of failed jobs.

I would like to handle failed jobs as follows:
- NACK the failed job with an ever growing delay (#170)
- If the number of retries is higher than X, move the job to a failure queue (dead letter queue) with a new TTL, so that it can be inspected manually and acted upon

Neither of these actions are possible in Disque right now and if using a workaround - adding a new, copied job - we lose both the job ID as well as the NACK and add. delivery counters.

That's why I would like to propose four enhancements (proposals 3. and 4. are different solutions of the same problem):
1. Allow to NACK a job with a delay (#170)
2. Allow to move a job to a different queue with a new TTL
3. Allow callers to change the job body
4. OR even better, if feasible: Implement custom job metadata support, like NACKs and additional-deliveries but user-defined and mutable

Ad 3. We use the job body to store job metadata. We use metadata to work around missing features 1. and 2. - we store the original job ID as well as the total number of retries there. It could also be helpful to eg. save the exact time and reason the job has failed. This requires changing the existing job body.
Supporting custom, mutable job metadata as a first class citizen in Disque would be even better.

The point of all these suggestions is to keep the ID of a job intact throughout its lifetime while allowing for a more complex handling (delayed NACKing, moving between queues, storing extra details).

What do you think? Are the suggestions too complex? Are they useful?

Beitragsleitfaden

Für dieses Repository ist kein Beitragsleitfaden indexiert

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.