element-hq / element-hq/synapse

AS scheduler may drop events during a restart

Open
#11,637 0 comments 0 reactions 0 assignees View on GitHub
A-Application-Service T-Enhancement
Dominant language
Python
Stars
4.6k
Forks
600
Avg merge
5d 22h
Merged PRs (30d)
51

Description

This issue has been migrated from [#11637](https://github.com/matrix-org/synapse/issues/11637).

---

### Description

The AS scheduler keeps track of events to send to each AS in memory as they come in, only pushing these into the database (as an AS transaction) once any in-flight requests have been completed:

https://github.com/matrix-org/synapse/blob/c500bf37d660b08efb48501b7690dc4448b39eca/synapse/appservice/scheduler.py#L149-L173

### Steps to reproduce

This means AS events could be lost during the following series of events:

+ events come in, AS request begins
+ more events come in while AS is processing, these only exist in memory
+ a restart/crash/etc of the synapse or AS pusher process - the above AS events are lost forever

### Possible solution

Would it be possible to setup some kind of exit handling (`atexit` from stdlib?) that dumps any in-memory events into a new txn in the database before the process exits, this would prevent any loss of AS events.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.