element-hq / element-hq/synapse

Migrate localpart-only columns to full MXIDs

Open
#15,396 1 comment 0 reactions 0 assignees View on GitHub
A-Database T-Task Z-Future-Maintenance
Dominant language
Python
Stars
4.6k
Forks
600
Avg merge
5d 22h
Merged PRs (30d)
51

Description

This issue has been migrated from [#15396](https://github.com/matrix-org/synapse/issues/15396).

---

`profiles.user_id` and `user_filters.user_id` (are there more?) currently store localparts only, which is inconsistent with the rest of the schema. We'd like to move to storing full MXIDs.

### The plan
The migration will happen over two schema versions.

**Current schema version +0**
Can use databases up to and including schema version +1.

**Schema version +1**
Add a new column, called `full_user_id`, which is nullable and unique.
Add background update to populate it.
Modify reads to query both the localpart and full user IDs (localparts can't include `@`s or `:`s and so can't be confused with a full MXID)

Can use databases up to and including schema version +2.

TODO: we need to modify reads to `user_id` to check for both the full MXID and localpart, to work with schema version +2. Reads and writes should not touch `full_user_id`, otherwise Synapse won't be compatible with schema version +2, where there is no longer a `full_user_id` column.

**Schema version +2**
Force the background job to complete
TODO: figure out how. We may have to duplicate the background job logic in the schema migration.
NB: just because the background update has finished does not mean that `full_user_id` is fully populated. The server might have been rolled back to a previous Synapse version after it completed.
Make `full_user_id` non-nullable (see @\reivilibre's comment below about avoiding table locks)
Drop `user_id` and rename `full_user_id` to `user_id` in a schema migration.
Unqueue the background job, just in case it hasn't run yet/is still in progress.
Update reads to only query for the full user ID.
For this step, we want to audit usages of both `{profiles,user_filters}.user_id` and `{profiles,user_filters}.full_user_id` since there are still some reads that only use the old `user_id`.

Update `SCHEMA_COMPAT_VERSION` to schema version +1.
ie. code expecting schema version +0 or later cannot use the database, but code expecting schema version +1 or later can.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.