element-hq / element-hq/synapse

State resolution of a large room but small deltas caused issues on event persister

Open
#12,378 0 comments 0 reactions 0 assignees View on GitHub
A-Performance A-State-Resolution T-Enhancement
Dominant language
Python
Stars
4.6k
Forks
600
Avg merge
5d 22h
Merged PRs (30d)
51

Description

This issue has been migrated from [#12378](https://github.com/matrix-org/synapse/issues/12378).

---

This led to all events being persisted on that worker to take a while to persist, heavily impacting our apdex. The room in question had 30K events, but the forks were very recent. The `3 biggest rooms for state-res` metrics had numbers such as `13.461s` CPU time and `6.54011s` DB time. During this period the reactor tick time ended up being large, which is likely what caused the delays persisting events in other rooms.

Some things we can do here:
* implement a delta state resolution algorithm
* speed up state res
* try and have state res yield to the reactor more often, so that it doesn't block the reactor

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.