element-hq / element-hq/synapse
Very long query times for get_auth_chain_difference_chains
- Dominant language
- Python
- Stars
- 4.6k
- Forks
- 600
- Avg merge
- 5d 22h
- Merged PRs (30d)
- 51
Description
### Description
Since upgrading to v1.105.1 because of the security patch, we are seeing very high transaction times specifically for the transaction "get_auth_chain_difference_chains". In our case, we directly upgraded from v1.102.0 to v1.105.1, but heard from others with the same problem after upgrading from v1.104.0.
This could be a regression caused by https://github.com/element-hq/synapse/pull/17044 which modified code called through several indirections by "get_auth_chain_difference_chains" or a regression caused by the security patches.

### Steps to reproduce
- Run a large enough homseserver using Synapse v1.105.1 joined to many rooms
- Observe regular long-running transactions "get_auth_chain_difference_chains"
### Homeserver
kit.edu
### Synapse Version
1.105.1
### Installation Method
Debian packages from packages.matrix.org
### Database
postgresql 13.14-0+deb11u1
### Workers
Multiple workers
### Platform
```
root@main-1:~# cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 11 (bullseye)"
NAME="Debian GNU/Linux"
VERSION_ID="11"
VERSION="11 (bullseye)"
VERSION_CODENAME=bullseye
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"
root@main-1:~# uname -a
Linux main-1 5.10.0-28-amd64 #1 SMP Debian 5.10.209-2 (2024-01-31) x86_64 GNU/Linux
```
Synapse Main, Workers and the database all run in virtual machines.
### Configuration
Presence is enabled.
### Relevant log output
```shell
I'm not sure which logs would be helpful here and I can't find any relevant log entries in the affected code. Can add logs if you give me a hint which contexts I have to set to debug :)
```
### Anything else that would be useful to know?
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.