element-hq / element-hq/synapse

The `get_rooms_paginate` DB queries are slow

Open
#17,330 0 comments 0 reactions 0 assignees View on GitHub
A-Database A-Performance O-Uncommon S-Tolerable T-Defect
Dominant language
Python
Stars
4.6k
Forks
600
Avg merge
5d 22h
Merged PRs (30d)
51

Description

The `RoomWorkerStore.get_rooms_paginate` DB store method contains some slow SQL queries. This method is currently only called when hitting the [List Room Admin API](https://element-hq.github.io/synapse/latest/admin_api/rooms.html#list-room-api). The current codebase contains this query:

https://github.com/element-hq/synapse/blob/f1c4dfb08b530f2bfaf9c6723ce69ccd231a3370/synapse/storage/databases/main/room.py#L703-L723

Due to the two `INNER JOIN`s, the statement [takes about 700ms to execute](https://github.com/element-hq/synapse/pull/17276/files#r1633079477) on matrix.org (and this is after caching, as I ran the statement once before attempting to `ANALYZE` it).

https://github.com/element-hq/synapse/pull/17276/files#r1631162300 added a similar set of `INNER JOIN`s to the `count_sql` query.

These joins are necessary to access metadata about rooms, which is spread across multiple tables. For instance, the `rooms` table contains an `is_public` column, which allows for filtering room results based on whether a room is published in the public room directory.

Can we consolidate this information into a single table somehow? Or is there a faster way to access information across multiple tables than `INNER JOIN`s?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.