element-hq / element-hq/synapse

High database usage by `get_auth_chain_ids_chains` after no particular event

Open
#17,470 15 comments 2 reactions 0 assignees View on GitHub
A-Database
Dominant language
Python
Stars
4.6k
Forks
600
Avg merge
5d 22h
Merged PRs (30d)
51

Description

### Description

The CPU usage of my server started to increase after no particular action on my end. After digging a little, I did find that the `get_auth_chain_ids_chains` transaction started to take a lot more time that previously.

Synapse exporter:
![2024-07-22-184751_1645x765_scrot](https://github.com/user-attachments/assets/0f69c841-2a6e-4a5b-8faa-9ee86ee27652)
![2024-07-22-191744_1640x763_scrot](https://github.com/user-attachments/assets/bc69facb-e347-4b03-929b-393b2eed20b1)

Postgres exporter:
![2024-07-22-185155_1644x1156_scrot](https://github.com/user-attachments/assets/66cfd7ff-20be-4742-b03d-f82c79cf12c4)

### Steps to reproduce

- Run synapse for a long time :sweat_smile:

### Homeserver

club1.fr

### Synapse Version

1.103.0

### Installation Method

Debian packages from packages.matrix.org

### Database

PostgreSQL single server, restored from a backup

### Workers

Single process

### Platform

- Debian 11 Bullseye
- `Linux club1.fr 5.10.0-31-amd64 #1 SMP Debian 5.10.221-1 (2024-07-14) x86_64 GNU/Linux`
- No VM, no Containers, Bare metal

### Configuration

Definitely some unusual config:

```yaml
# Message retention policy at the server level.
#
# Room admins and mods can define a retention period for their rooms using the
# 'm.room.retention' state event, and server admins can cap this period by setting
# the 'allowed_lifetime_min' and 'allowed_lifetime_max' config options.
#
# If this feature is enabled, Synapse will regularly look for and purge events
# which are older than the room's maximum retention period. Synapse will also
# filter events received over federation so that events that should have been
# purged are ignored and not stored again.
#
retention:
# The message retention policies feature is disabled by default. Uncomment the
# following line to enable it.
#
enabled: true

# Default retention policy. If set, Synapse will apply it to rooms that lack the
# 'm.room.retention' state event. Currently, the value of 'min_lifetime' doesn't
# matter much because Synapse doesn't take it into account yet.
#
default_policy:
min_lifetime: 1d
max_lifetime: 3y

media_retention:
local_media_lifetime: 2y
remote_media_lifetime: 90d

# When this option is enabled, the room "complexity" will be checked before a user
# joins a new remote room. If it is above the complexity limit, the server will
# disallow joining, or will instantly leave.
limit_remote_rooms:
enabled: true
complexity: 10
complexity_error: "Désolé, cette room est trop complèxe pour le serveur CLUB1."
admins_can_join: false

# disable presence completely
presence:
enabled: false
```

### Relevant log output

```shell
https://static.club1.fr/nicolas/random-other/log-synapse-2024-07-19.log
```

### Anything else that would be useful to know?

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.