element-hq / element-hq/synapse

Initial and incremental syncs with presence enabled eat up RAM until an OOM hits

Open
#13,901 0 comments 0 reactions 0 assignees View on GitHub
A-Memory-Usage A-Presence O-Uncommon S-Major T-Defect
Dominant language
Python
Stars
4.6k
Forks
600
Avg merge
5d 22h
Merged PRs (30d)
51

Description

This issue has been migrated from [#13901](https://github.com/matrix-org/synapse/issues/13901).

---

### Description

Synapse eat all ram (8GB) and swap until OOM hits.

### Steps to reproduce

There are 3 accounts on the server. This happens when one of the users connects (my account is an administrator). Did not connect to other rooms with a large number of connections before the problem occurred.

### Homeserver

tavda.org

### Synapse Version

{"server_version":"1.67.0","python_version":"3.10.6"}

### Installation Method

Debian packages from packages.matrix.org

### Platform

Ubuntu 22.04 LTS

### Relevant log output

```shell
Sep 26 14:27:29 nether2.network systemd[1]: matrix-synapse.service: A process of this unit has been killed by the OOM killer.
Sep 26 14:27:30 nether2.network systemd[1]: matrix-synapse.service: Main process exited, code=killed, status=9/KILL
Sep 26 14:27:30 nether2.network systemd[1]: matrix-synapse.service: Failed with result 'oom-kill'.
Sep 26 14:27:30 nether2.network systemd[1]: matrix-synapse.service: Consumed 29.476s CPU time.
Sep 26 14:27:33 nether2.network systemd[1]: matrix-synapse.service: Scheduled restart job, restart counter is at 15.
Sep 26 14:27:33 nether2.network systemd[1]: Stopped Synapse Matrix homeserver.
Sep 26 14:27:33 nether2.network systemd[1]: matrix-synapse.service: Consumed 29.476s CPU time.
Sep 26 14:27:33 nether2.network systemd[1]: Starting Synapse Matrix homeserver...
Sep 26 14:27:39 nether2.network synapse[8680]: root: [main] ***** STARTING SERVER *****
Sep 26 14:27:39 nether2.network synapse[8680]: root: [main] Server /opt/venvs/matrix-synapse/lib/python3.10/site-packages/synapse/app/homeserver.py version 1.67.0
Sep 26 14:27:39 nether2.network synapse[8680]: root: [main] Server hostname: tavda.org
Sep 26 14:27:39 nether2.network synapse[8680]: root: [main] Instance name: master
Sep 26 14:27:39 nether2.network synapse[8680]: synapse.app.homeserver: [main] Setting up server
Sep 26 14:27:39 nether2.network synapse[8680]: synapse.server: [main] Setting up.
Sep 26 14:27:39 nether2.network synapse[8680]: synapse.storage.databases: [main] [database config 'master']: Checking database server
Sep 26 14:27:39 nether2.network synapse[8680]: synapse.storage.databases: [main] [database config 'master']: Preparing for databases ['main', 'state']
Sep 26 14:27:39 nether2.network synapse[8680]: synapse.storage.prepare_database: [main] ['main', 'state']: Checking existing schema version
Sep 26 14:27:39 nether2.network synapse[8680]: synapse.storage.prepare_database: [main] ['main', 'state']: Existing schema is 72 (+8 deltas)
Sep 26 14:27:39 nether2.network synapse[8680]: synapse.storage.databases.main: [main] Checking database for consistency with configuration...
Sep 26 14:27:39 nether2.network synapse[8680]: synapse.storage.prepare_database: [main] Applying schema deltas for v72
Sep 26 14:27:39 nether2.network synapse[8680]: synapse.storage.prepare_database: [main] Schema now up to date
Sep 26 14:27:39 nether2.network synapse[8680]: synapse.storage.databases: [main] [database config 'master']: Starting 'main' database
```

### Anything else that would be useful to know?

During normal server operation, the top shows 7.5GB of free memory. The other two users can communicate with each other.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.