element-hq / element-hq/synapse

Sync with no since param is not always up to date

Open
#14,444 1 comment 0 reactions 0 assignees View on GitHub
A-Sync O-Occasional S-Tolerable T-Defect
Dominant language
Python
Stars
4.6k
Forks
600
Avg merge
5d 22h
Merged PRs (30d)
51

Description

This issue has been migrated from [#14444](https://github.com/matrix-org/synapse/issues/14444).

---

### Description

If a message is posted in a room and you receive it and see it in a client (in element for example) and then run /sync from curl/postman with the same authorization token and no `since` param then it can sometimes exclude the most recent messages. If you then get the next_batch token from that sync response and use it in a /sync again as the `since` param then it may give you a message that you received in the previous sync again. This sync can also say that there is a highlighted message that you have already read (and set read marker to it). This issue goes away after 2-30 seconds, as if the server only updates latest sync data periodically.

The affect of this is that in my matrix client for example you see that in the room that you got a message in it says that there are unread mentions even when you have read the message. If you also enter the room then the message at the bottom is not the latest one and it has to wait until it fetches the latest one.

The /notifications api says that the message hasn't been read either.
This issue didn't happen long ago either, i dont remember exactly how long ago. Im not really sure if it is a bug or if it's intentionally behaving like this but it causes issues in my client.

It may only happen when using a filter, specifically this one: `{"presence":{"limit":0,"types":[""]},"account_data":{"types":["qm.emoji","m.direct"]},"room":{"state":{"not_types":["m.room.related_groups","m.room.power_levels","m.room.join_rules","m.room.history_visibility","m.room.canonical_alias","m.space.child"],"lazy_load_members":true},"timeline":{"lazy_load_members":true},"ephemeral":{"limit":0,"types":[""],"lazy_load_members":true},"account_data":{"types":["m.fully_read","m.tag","qm.last_read_message_timestamp"],"lazy_load_members":true}}}`

### Steps to reproduce

- Use two different account at the same time, two element windows in two different browsers work.
- While being in the same room use one of the accounts to @mention the other user.
- Open the account on the other window, scroll to the bottom and see that it has been read properly.
- Open postman/curl and run `https://matrix-server/_matrix/client/r0/sync?filter={"presence":{"limit":0,"types":[""]},"account_data":{"types":["qm.emoji","m.direct"]},"room":{"state":{"not_types":["m.room.related_groups","m.room.power_levels","m.room.join_rules","m.room.history_visibility","m.room.canonical_alias","m.space.child"],"lazy_load_members":true},"timeline":{"lazy_load_members":true},"ephemeral":{"limit":0,"types":[""],"lazy_load_members":true},"account_data":{"types":["m.fully_read","m.tag","qm.last_read_message_timestamp"],"lazy_load_members":true}}}&timeout=0` and with authorization token set (to the same account as the account that was mentioned).
- See that the latest m.room.message is not the latest message that was posted in the room. Also try using the next_batch at top level to do another request by setting the `since` param to see messages after that sync.

### Homeserver

matrix.org, midov.pl

### Synapse Version

1.71.0rc2 (b=matrix-org-hotfixes,39ea774cff,dirty), 1.69.0

### Installation Method

_No response_

### Platform

matrix.org

### Relevant log output

```shell
No helpful logs available, only steps to reproduce
```

### Anything else that would be useful to know?

_No response_

Contributor guide

Open the contributing guide

Research direction

Reproduce the report using the stated /sync endpoint, timeout=0, filter, authorization token, and two accounts, then compare the initial response with a follow-up using its next_batch as since. Check whether the latest m.room.message and highlighted/read state are current immediately after the mention; done means the initial sync is up to date and the follow-up does not duplicate events or report an already-read highlight.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend-api-design
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.