apalis-dev / apalis-dev/apalis

[Feature]: partitions, virtual queues by some key

Aperta
#718 2 commenti 0 reazioni 0 assegnatari Vedi su GitHub
enhancement
Lingua principale
Rust
Stelle
1.4k
Fork
92
Metriche di merge delle PR
Nessuna PR unita negli ultimi 30g

Descrizione

### Is your feature request related to a problem?

All changes related to one entity are ordered and should be executed in the same order to avoid race conditions

I am looking for something similar to Kafka’s key, meaning the ability to ensure ordering within a partition.

It would be great if Apalis allowed only one active job for a particular key.

For example, all jobs related to `order 123` should be executed sequentially, which would guarantee that jobs do not overwrite each other’s results.

### Describe the solution you'd like

```
WorkerBuilder::new("emails")
.backend(storage)
.parition_by(|job: &SendEmail| job.account_id)
```
Backend picks next job taking partition key into account + skips parition keys which currently are locked.

### Describe alternatives you've considered

_No response_

### Use case

As I mentioned before, all changes related to one entity are ordered and should be executed in the same order to avoid race conditions

### Additional context

_No response_

### Code of Conduct

- [x] I agree to follow this project's Code of Conduct

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.