antirez / antirez/disque

[Suggestion] Move jobs between queues, change the job body, custom metadata for a better failure handling

Đang mở
#174 2 bình luận 0 reaction 0 người được giao Xem trên GitHub
Ngôn ngữ chính
C
Star
8.1k
Fork
532
Chỉ số merge pull request
Không có pull request nào được merge trong 30 ngày

Mô tả

Hi,
this suggestion is connected to #170 in that both concern the handling of failed jobs.

I would like to handle failed jobs as follows:
- NACK the failed job with an ever growing delay (#170)
- If the number of retries is higher than X, move the job to a failure queue (dead letter queue) with a new TTL, so that it can be inspected manually and acted upon

Neither of these actions are possible in Disque right now and if using a workaround - adding a new, copied job - we lose both the job ID as well as the NACK and add. delivery counters.

That's why I would like to propose four enhancements (proposals 3. and 4. are different solutions of the same problem):
1. Allow to NACK a job with a delay (#170)
2. Allow to move a job to a different queue with a new TTL
3. Allow callers to change the job body
4. OR even better, if feasible: Implement custom job metadata support, like NACKs and additional-deliveries but user-defined and mutable

Ad 3. We use the job body to store job metadata. We use metadata to work around missing features 1. and 2. - we store the original job ID as well as the total number of retries there. It could also be helpful to eg. save the exact time and reason the job has failed. This requires changing the existing job body.
Supporting custom, mutable job metadata as a first class citizen in Disque would be even better.

The point of all these suggestions is to keep the ID of a job intact throughout its lifetime while allowing for a more complex handling (delayed NACKing, moving between queues, storing extra details).

What do you think? Are the suggestions too complex? Are they useful?

Hướng dẫn đóng góp

Chưa lập chỉ mục được hướng dẫn đóng góp cho kho mã nguồn này

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.