element-hq / element-hq/synapse

federation device list updates cause high CPU on main process of matrix.org

Open
#8,613 0 comments 0 reactions 0 assignees View on GitHub
A-Device-List-Tracking A-Performance
Dominant language
Python
Stars
4.6k
Forks
600
Avg merge
5d 22h
Merged PRs (30d)
51

Description

This issue has been migrated from [#8613](https://github.com/matrix-org/synapse/issues/8613).

---

Something causes handling of m.device_list_updates over federation to take a lot of CPU time

See matrix.org homeserver.log from ~ 2020-10-21 12:45 to 2020-10-21 13:24 for logs

from a flamegraph, 95% of time was spent in synapse/storage/util/id_generators.py152

![100percent](https://user-images.githubusercontent.com/1917473/96735383-6f298d80-13b3-11eb-910b-00a43ae6ec5c.png)

I couldn't narrow it down to one particular request or homeserver that was causing this high CPU; there wasn't a spike in requests from a single homeserver as far as I could see; and request volumes were fairly low.

The database appeared unloaded at the time, and this CPU usage had a knock-on efffect on other endpoints relying on the main process.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.