element-hq / element-hq/synapse
MSC4354 Sticky Events sync and sliding sync should use a sharded token
- Dominant language
- Python
- Stars
- 4.6k
- Forks
- 600
- Avg merge
- 5d 22h
- Merged PRs (30d)
- 51
Description
Experimental feature tracking issue: #19409
Spawning from https://github.com/element-hq/synapse/pull/19365#discussion_r3017099461
---
When returning sticky events over sync and sliding sync, we currently use a single int as a stream token, but should change to using a sharded stream token.
As a very basic recap of https://element-hq.github.io/synapse/latest/development/synapse_architecture/streams.html#current-stream-id: a stream reader that uses a single int on a multi-writer stream must wait for all writers to finish persisting their work before advancing up to that position.
This means that the reader can be delayed in reading new facts in the stream.
Since it's event persistence that writes to both streams, this will probably cause a slightly undesirable situation where the event first appears in the timeline before appearing in the sticky stream (from the point of view of a stream reader serving sync).
As a result, users may see the sticky event sent down two (e.g.) consecutive sync responses, once in the timeline and once in the sticky section. This defeats the 'deduplication' logic intended in MSC4354, so we ideally want these two streams to be perfectly synchronised.
In other words, we need to use sharded tokens on the sticky events stream as well.
Contributor guide
Assessment
This issue has not been assessed yet.