element-hq / element-hq/synapse

slow cleanup in `device_lists_outbound_pokes`

Open
#16,480 0 comments 0 reactions 0 assignees View on GitHub
A-Database A-Performance S-Major T-Defect
Dominant language
Python
Stars
4.6k
Forks
600
Avg merge
5d 22h
Merged PRs (30d)
51

Description

This issue has been migrated from [#16480](https://github.com/matrix-org/synapse/issues/16480).

---

There is a cleanup job called `prune_old_outbound_device_pokes` (https://github.com/matrix-org/synapse/blob/v1.93.0/synapse/storage/databases/main/devices.py#L1297-L1388) which periodically attempts to clear out the `device_lists_outbound_pokes` table a bit.

The SELECT query can sometimes be very slow and IO-intensive. Sometimes it even takes longer than the 1 hour request timeout, meaning that it gets stuck without ever making any progress.

@erikjohnston suggests:

> I do kinda want to change this, so we have two tables: the current table, and a duplicate which only holds the latest destination/user -> stream ID

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.