element-hq / element-hq/synapse
`/sync` incorrectly calculates state changes for non-gappy syncs with lazy-loading
- Dominant language
- Python
- Stars
- 4.6k
- Forks
- 600
- Avg merge
- 5d 22h
- Merged PRs (30d)
- 51
Description
Consider the following DAG:
```
E1
↗ ↖
| S2
|
--|---
|
E3
```
* Lazy-loading client does an initial /sync, which returns state excluding S2 for the reasons explained in #16940.
* Lazy-loading client now decides to load the full membership list in the room. This *still* excludes S2 due to #16940.
* Client now assumes that it has a full membership list, and that it will be kept up to date with future membership changes via `/sync`. (Per https://github.com/element-hq/element-web/issues/27285, this is technically incorrect, but normally close enough.)
* More events arrive:
```
E1
↗ ↖
| S2
| ↑
E3 |
↑ |
--|------|----
| |
E4 |
↖ /
E5
```
* Lazy-loading client does an incremental sync. Even though the state has changed between the previous and new token, S2 is *still* filtered out due to lazy-loading.
Now, unlike the gappy sync case, we can't really blame this on the client: there is no indication that events or state may be missing, though per https://github.com/matrix-org/matrix-spec/issues/942#issuecomment-1961038725 it would be nice if the spec actually told us the right thing to do here.
Anyway, I think really this is a consequence of #16940: if that were fixed, then we wouldn't be in the situation of needing to send S2 in the incremental sync.
I'm calling it out as a separate issue because:
* The trigger conditions are a bit different.
* We could work around it by disabling lazy-loading for all incremental syncs, and indeed maybe we should.
Contributor guide
Research direction
Start at the /sync entry point and trace how non-gappy incremental syncs calculate state changes when lazy-loading excludes S2. Compare the described DAG across the initial, full-membership, and incremental syncs, using #16940 for context. Done means the incorrect state-change calculation is resolved or the intended workaround is explicitly established.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api, backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100