element-hq / element-hq/synapse
Sync with no since param is not always up to date
- Dominant language
- Python
- Stars
- 4.6k
- Forks
- 600
- Avg merge
- 5d 22h
- Merged PRs (30d)
- 51
Description
This issue has been migrated from [#14444](https://github.com/matrix-org/synapse/issues/14444).
---
### Description
If a message is posted in a room and you receive it and see it in a client (in element for example) and then run /sync from curl/postman with the same authorization token and no `since` param then it can sometimes exclude the most recent messages. If you then get the next_batch token from that sync response and use it in a /sync again as the `since` param then it may give you a message that you received in the previous sync again. This sync can also say that there is a highlighted message that you have already read (and set read marker to it). This issue goes away after 2-30 seconds, as if the server only updates latest sync data periodically.
The affect of this is that in my matrix client for example you see that in the room that you got a message in it says that there are unread mentions even when you have read the message. If you also enter the room then the message at the bottom is not the latest one and it has to wait until it fetches the latest one.
The /notifications api says that the message hasn't been read either.
This issue didn't happen long ago either, i dont remember exactly how long ago. Im not really sure if it is a bug or if it's intentionally behaving like this but it causes issues in my client.
It may only happen when using a filter, specifically this one: `{"presence":{"limit":0,"types":[""]},"account_data":{"types":["qm.emoji","m.direct"]},"room":{"state":{"not_types":["m.room.related_groups","m.room.power_levels","m.room.join_rules","m.room.history_visibility","m.room.canonical_alias","m.space.child"],"lazy_load_members":true},"timeline":{"lazy_load_members":true},"ephemeral":{"limit":0,"types":[""],"lazy_load_members":true},"account_data":{"types":["m.fully_read","m.tag","qm.last_read_message_timestamp"],"lazy_load_members":true}}}`
### Steps to reproduce
- Use two different account at the same time, two element windows in two different browsers work.
- While being in the same room use one of the accounts to @mention the other user.
- Open the account on the other window, scroll to the bottom and see that it has been read properly.
- Open postman/curl and run `https://matrix-server/_matrix/client/r0/sync?filter={"presence":{"limit":0,"types":[""]},"account_data":{"types":["qm.emoji","m.direct"]},"room":{"state":{"not_types":["m.room.related_groups","m.room.power_levels","m.room.join_rules","m.room.history_visibility","m.room.canonical_alias","m.space.child"],"lazy_load_members":true},"timeline":{"lazy_load_members":true},"ephemeral":{"limit":0,"types":[""],"lazy_load_members":true},"account_data":{"types":["m.fully_read","m.tag","qm.last_read_message_timestamp"],"lazy_load_members":true}}}&timeout=0` and with authorization token set (to the same account as the account that was mentioned).
- See that the latest m.room.message is not the latest message that was posted in the room. Also try using the next_batch at top level to do another request by setting the `since` param to see messages after that sync.
### Homeserver
matrix.org, midov.pl
### Synapse Version
1.71.0rc2 (b=matrix-org-hotfixes,39ea774cff,dirty), 1.69.0
### Installation Method
_No response_
### Platform
matrix.org
### Relevant log output
```shell
No helpful logs available, only steps to reproduce
```
### Anything else that would be useful to know?
_No response_
Contributor guide
Research direction
Reproduce the report using the stated /sync endpoint, timeout=0, filter, authorization token, and two accounts, then compare the initial response with a follow-up using its next_batch as since. Check whether the latest m.room.message and highlighted/read state are current immediately after the mention; done means the initial sync is up to date and the follow-up does not duplicate events or report an already-read highlight.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend-api-design
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100