element-hq / element-hq/synapse
We sometimes backfill on `/messages` requests when we shouldn't.
- Dominant language
- Python
- Stars
- 4.6k
- Forks
- 607
- Avg merge
- 5d 22h
- Merged PRs (30d)
- 51
Description
This issue has been migrated from [#10742](https://github.com/matrix-org/synapse/issues/10742).
---
This happens when we use a `from` token that just has a stream ordering part (which clients do the first time they start back paginating in a room after a `/sync` request), as the following function to convert a stream token to a topological token can lead to incorrect results:
https://github.com/matrix-org/synapse/blob/e2481dbe9325321d460037a2efe9b9ea2ac78057/synapse/storage/databases/main/stream.py#L747-L762
This is a particularly big problem for rooms like Matrix HQ, where repeated calls to `/messages` will always trigger a backfill and then time out, even though the server has enough events to return to the client.
The problem manifests when the server receives an old event, so that it ends up with an event with a recent stream ordering but an old depth, causing the above function to return the old depth.
Contributor guide
Research direction
Start at synapse/storage/databases/main/stream.py lines 747-762, focusing on conversion of a stream-only token to a topological token. Reproduce a /messages request after /sync with an old event having recent stream ordering and old depth; done means repeated requests return available events without triggering an unnecessary backfill or timing out.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend-api-design, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100