clj-commons / clj-commons/durable-queue

support archiving/replaying completed tasks/slabs for longer-term disaster recovery

Open
#31 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Clojure
Stars
405
Forks
30
PR merge metrics
No merged PRs in 30d

Description

`durable-queue` does a great job already of recovering from application restart. however, it cannot handle longer-term disaster recovery very well for a couple of reasons:
1. empty slabs (i.e. slabs with all `:complete`d tasks) [get removed](https://github.com/clj-commons/durable-queue/blob/master/src/clj_commons/durable_queue.clj#L530) fairly frequently; and
2. on startup, slab tasks in the `:complete` state are skipped over even if you provide [a `:complete?` fn](https://github.com/clj-commons/durable-queue/blob/master/src/clj_commons/durable_queue.clj#L580)

as it stands now, when using `durable-queue` a secondary storage solution is necessary to be able to replay events in the case that downstream repairs are required.

it would be nice if `durable-queue` could handle being able to archive/replay slabs at the user's discretion.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.