matrix-org / matrix-org/matrix-spec
Ability to migrate events between different formats.
Nobody has claimed this yet.
- Dominant language
- HTML
- Stars
- 330
- Forks
- 150
- Avg merge
- 2h 21m
- Merged PRs (30d)
- 3
Description
One of the biggest problems in rapid Matrix development turns out to be confusion over the extent to which it's acceptable to ship experimental features in clients which generate events with prefixed event types/fields. Currently there is a concern that if one ships features with prefixed event types too widely, the immutable nature of events means that the experimental feature will become a de facto part of the Matrix standard, and clients will have to implement it for the rest of time in order to render older room history - much as org.matrix.custom.html has done.
However: perhaps we are being too constrained by the idea that matrix events are immutable. What if we provided a mechanism to migrate events from an old format to a new one during a room version upgrade? So that room version N+1 re-imports all the events from prior versions of the room, having reexpressed them (e.g. JSON->CBOR, or unprefixing a prefix, etc)? For instance, the server triggering the upgrade could go through re-importing all the old events (obviously at the expense of manipulating history, but that comes with the territory - plus folks would always be able to compare against the old room history to verify that the migration was not malicious). For E2EE, you could rely on the user triggering the upgrade to have all the keys, and have them similarly re-submit all the messages (again at the expense of transport consistency, and the ugliness of the client having to be online during the migration). The migration could also be done incrementally in the background via [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716).
This feels like a pretty useful thing to have in the protocol, especially as we gear up to more invasive changes such as changing event shapes in order to support account portability, P2P, or more efficient event encodings.
Thoughts welcome :)
(This would also be useful for migrating historical data between encryption formats - see also https://github.com/matrix-org/matrix-doc/issues/3520)
EDIT: Thinking further, this is almost a hard requirement whenever a crypto vulnerability emerges (e.g. post-quantum) which requires everyone to shift to a more secure record of their conversation history...
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with MSC2716 and issue #3520, the references named in the proposal, then map the questions around room-version upgrades, event re-expression, and encrypted history migration. Done would require a protocol design that resolves the migration, verification, and E2EE constraints; the issue names no implementation files or tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- json
- Domain
- backend-api-design, cryptography, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100