element-hq / element-hq/synapse

`/sync` incorrectly calculates state changes for non-gappy syncs with lazy-loading

Open
#17,050 0 comments 0 reactions 0 assignees View on GitHub
A-Lazy-Loading A-Sync T-Defect
Dominant language
Python
Stars
4.6k
Forks
600
Avg merge
5d 22h
Merged PRs (30d)
51

Description

Consider the following DAG:

```
E1
↗ ↖
| S2
|
--|---
|
E3
```

* Lazy-loading client does an initial /sync, which returns state excluding S2 for the reasons explained in #16940.
* Lazy-loading client now decides to load the full membership list in the room. This *still* excludes S2 due to #16940.
* Client now assumes that it has a full membership list, and that it will be kept up to date with future membership changes via `/sync`. (Per https://github.com/element-hq/element-web/issues/27285, this is technically incorrect, but normally close enough.)
* More events arrive:
```
E1
↗ ↖
| S2
| ↑
E3 |
↑ |
--|------|----
| |
E4 |
↖ /
E5
```
* Lazy-loading client does an incremental sync. Even though the state has changed between the previous and new token, S2 is *still* filtered out due to lazy-loading.

Now, unlike the gappy sync case, we can't really blame this on the client: there is no indication that events or state may be missing, though per https://github.com/matrix-org/matrix-spec/issues/942#issuecomment-1961038725 it would be nice if the spec actually told us the right thing to do here.

Anyway, I think really this is a consequence of #16940: if that were fixed, then we wouldn't be in the situation of needing to send S2 in the incremental sync.

I'm calling it out as a separate issue because:
* The trigger conditions are a bit different.
* We could work around it by disabling lazy-loading for all incremental syncs, and indeed maybe we should.

Contributor guide

Open the contributing guide

Research direction

Start at the /sync entry point and trace how non-gappy incremental syncs calculate state changes when lazy-loading excludes S2. Compare the described DAG across the initial, full-membership, and incremental syncs, using #16940 for context. Done means the incorrect state-change calculation is resolved or the intended workaround is explicitly established.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.