matrix-org / matrix-org/matrix-spec
Better specify how to validate events
- Dominant language
- HTML
- Stars
- 330
- Forks
- 150
- Avg merge
- 2h 21m
- Merged PRs (30d)
- 3
Description
We should clearly specify how to validate an event that has been received over federation, and how the receiving server should behave when validation fails.
The sorts of things we should consider checking include:
* json compliance:
* does it conform to the rules of canonicaljson (eg no floats, no bigints)?
* Does it have invalid UTF-8 (invalid byte sequences, overlong encodings, things which decode to the UTF-16 surrogates or to codepoints above U+10FFFF)?
* Does the JSON contain unicode non-characters? What about `U+0000`, is that allowed?
* Do the strings within the JSON contain `\uNNNN` sequences that decode to unpaired UTF-16 surrogates or unicode non-characters?
* schema compliance:
* whether it has all the required fields listed at https://matrix.org/docs/spec/server_server/unstable.html#persistent-data-unit-schema
* whether those fields (and the optional ones) have the expected types, and match the relevant grammars, including length limits (cf #1001)
* event-type-specific tests: eg, if it is a membership event, whether `content.membership` is a known value
* whether or not it has a `state_key`, depending on its event type
* does the event as a whole exceed the length limit?
* whether the `origin` matches the sender sending the event, for pushed transactions.
* whether the `sender` and `event_id` match the `origin` (except for join events)
* whether the `creator` on a create event matches the `sender` and the `room_id`
* whether it has a "sensible" number of `prev_events`
* signatures: whether the event has been signed by the origin (and the sender's server, where different).
* also, whether the signing key was valid at the time the signature was made.
* hashes: whether the event's content matches the content hash
* whether hashes in prev_events and auth_events correspond to the hashes on events we have (though this will probably be made much simpler by matrix-org/matrix-spec-proposals#1640)
Parts of this list overlap with the rules already set out for [Authorization of PDUs](https://matrix.org/docs/spec/server_server/unstable.html#authorization-of-pdus).
Synapse currently checks many, but not all, of the things on this list, so there may well be rooms in existence which do not conform. Starting to check them now is problematic in that it may prevent servers from participating in those existing rooms. We should therefore consider whether it is reasonable to apply any additional checks now, or which should wait for a future room version.
We should also define the behaviour when a validation check is not met: should we ignore such events, or reject incoming pushes? For pulls, should we try to get the events from elsewhere?
Contributor guide
Research direction
Start with the linked Persistent Data Unit schema and Authorization of PDUs sections, then compare them with the validation checks currently described for federation. Review Synapse's existing checks and the compatibility concerns in this issue; done means specifying the remaining validation rules, failure behavior for pushes and pulls, and any room-version constraints.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- json
- Domain
- distributed-systems, documentation, networking, security
- Issue type
- Documentation
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100