element-hq / element-hq/synapse

Drop `state_events` and `rejections` tables

Open
#11,496 1 comment 0 reactions 1 assignee Claimed by @reivilibre View on GitHub
A-Database T-Task Z-Future-Maintenance
Dominant language
Python
Stars
4.6k
Forks
600
Avg merge
5d 22h
Merged PRs (30d)
51

Description

This issue has been migrated from [#11496](https://github.com/matrix-org/synapse/issues/11496).

---

Both `state_events` and `rejections` are tables which add a single piece of data to a subset of events (`state_key` and `reason`, respectively).

I believe this would better handled with nullable columns in the `events` table:
* `state_events` and `rejections` are *never* read from without a join to `events`, so we can save a join whereever we read from them
* it's not like adding the data to `events` is going to transform it from a small, cacheable table to something huge (so I don't expect much difference in the case where we *don't* join to `rejections` or `state_events`). In any case, when we look in `events` we normally want the rejection status too.
* obviously, populating three tables (and maintaining two extra indexes) is extra work to do when storing events.

However, getting from here to there is not an easy task by any means, and will take quite a bit of work to do in a way that doesn't break backwards compatibility. Here are the tasks as far as I can see them:

* [x] disambiguate any references to the `state_key` column, in preparation for adding new columns to `events`: #11497
* [x] Add `state_key` and `rejection_reason` columns to `events`, and populate them for all new events: #11792
* [x] Add a background update to populate `state_key` and `rejection_reason` for all existing events: #13215
* [ ] Add some code which determines if the background update has completed, and if so, sets a flag which makes reads use the new columns instead of joining to the old tables.
* [ ] Add some more code which drops the old tables and stops populating them.

Each of those steps will require SCHEMA_VERSION and/or SCHEMA_COMPAT version updates to ensure that we never break compatibility with the previous Synapse release.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.