matrix-org / matrix-org/matrix-spec

Consider changing `/messages` to use stream ordering for events

Open
#852 28 comments 7 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

feature
Dominant language
HTML
Stars
330
Forks
150
Avg merge
2h 21m
Merged PRs (30d)
3

Description

Today the /messages endpoint documentation does not say anything about the
ordering of the events. /sync hints at one possible ordering method:

> Events are ordered in this API according to the arrival time of the event on
> the homeserver. This can conflict with other APIs which order events based on
> their partial ordering in the event graph.

The current concept of ordering based on event graph is not ideal for the following reasons:

1. It's not clearly documented
- Which event is picked when an event has multiple prev events?
- What happens if a prev event is not known to the server
- What happens if an event has no prev events, or builds a circle of prev events
or is it's own prev event. This is not checked in the federation api.

2. You can't notify the client about updates
- Clients might cache events from /messages, but this list of events could be
incomplete because we don't know about events of a fork
- /sync can't be used to send these updates because it's response will be limited
if there are too many events

Possible solution:

/messages uses the same order as /sync, which is the order at which events
arrive at the server, not sorted at all.

This has the advantage of being a lot simpler and therefore doesn't have many
edge cases.

A problem is that Matrix events often arrive in batches from other servers, so
conversations might become unreadable because you first see all messages from
one user and then all messages from the other user.

This can be improved by adding more logic in the client (for example sort the
events by timestamp) and by improving the /messages api (maybe it can send prev
events to the client to improve ordering or call /context on them).

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the /messages and /sync endpoint documentation, then review the proposed use of arrival-time ordering and the noted /context interaction. Determine whether /messages should adopt /sync ordering and document the behavior and edge cases clearly, including how clients should handle event ordering.

Written by the indexing model from the issue text.

Assessment

Domain
api
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.