element-hq / element-hq/synapse

MSC4354 Sticky Events sync and sliding sync should use a sharded token

Open
#19,661 0 comments 0 reactions 1 assignee Claimed by @reivilibre View on GitHub
T-Task
Dominant language
Python
Stars
4.6k
Forks
600
Avg merge
5d 22h
Merged PRs (30d)
51

Description

Experimental feature tracking issue: #19409

Spawning from https://github.com/element-hq/synapse/pull/19365#discussion_r3017099461

---

When returning sticky events over sync and sliding sync, we currently use a single int as a stream token, but should change to using a sharded stream token.

As a very basic recap of https://element-hq.github.io/synapse/latest/development/synapse_architecture/streams.html#current-stream-id: a stream reader that uses a single int on a multi-writer stream must wait for all writers to finish persisting their work before advancing up to that position.
This means that the reader can be delayed in reading new facts in the stream.

Since it's event persistence that writes to both streams, this will probably cause a slightly undesirable situation where the event first appears in the timeline before appearing in the sticky stream (from the point of view of a stream reader serving sync).
As a result, users may see the sticky event sent down two (e.g.) consecutive sync responses, once in the timeline and once in the sticky section. This defeats the 'deduplication' logic intended in MSC4354, so we ideally want these two streams to be perfectly synchronised.

In other words, we need to use sharded tokens on the sticky events stream as well.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.