element-hq / element-hq/synapse

Optimize `/backfill` to respond faster

Open
#15,653 0 comments 0 reactions 0 assignees View on GitHub
A-Federation A-Messages-Endpoint A-Performance T-Enhancement
Dominant language
Python
Stars
4.6k
Forks
600
Avg merge
5d 22h
Merged PRs (30d)
51

Description

This issue has been migrated from [#15653](https://github.com/matrix-org/synapse/issues/15653).

---

Since `/backfill` is a big part of `/messages` performance, optimizing `/backfill` would help other servers respond to our `/backfill` requests faster which means we can respond to `/messages` faster.

From the examples below, it seems like we have these obvious opportunities to improve:

- [`get_backfill_events(...)`](https://github.com/matrix-org/synapse/blob/1903c7e5edccc86f6d28aed33dc2995b43d941b7/synapse/storage/databases/main/event_federation.py#L1402-L1533): Seems like a pretty slow query that we could probably turn into a recursive query.
- `get_current_state` -> `get_filtered_current_state_ids`: Takes ~200ms
- `is_host_joined`: uncached call took ~200ms
- `_get_state_groups_from_groups(...)`: A known slow thing
- Previous optimizations:
- https://github.com/matrix-org/synapse/pull/14527
- https://github.com/matrix-org/synapse/pull/15617

3.45s `/backfill?limit=100` request

934ms `/backfill?limit=100` request

Contributor guide

Open the contributing guide

Research direction

Start by profiling the /backfill?limit=100 entry point and reading get_backfill_events(...) in synapse/storage/databases/main/event_federation.py, then investigate the listed get_current_state, get_filtered_current_state_ids, is_host_joined, and _get_state_groups_from_groups(...) calls. Done means showing that /backfill responds faster than the reported 3.45s and 934ms examples while preserving its existing behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, performance
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.