element-hq / element-hq/synapse

`synapse_port_db` assumes `events` is append-only, but it is not

Open
#12,007 0 comments 0 reactions 0 assignees View on GitHub
A-Database S-Tolerable T-Defect
Dominant language
Python
Stars
4.6k
Forks
600
Avg merge
5d 22h
Merged PRs (30d)
51

Description

This issue has been migrated from [#12007](https://github.com/matrix-org/synapse/issues/12007).

---

`synapse_port_db` has a list of tables that it believes to be "append-only". That is to say, if the script is re-run on an updated copy of the input database, it will not clear these tables, but just add new rows that have arrived since last time.

Included in this list is `events`, which is *generally* read-only, but we do sometimes update existing rows - for example, when we convert an outlier to a non-outlier.

This means that any event that is de-outliered between runs of `synapse_port_db` may still be marked as an outlier in the target database, even though we have full state for it.

I'm not entirely sure if this has any measurable impact. It's possible that it simply means we will redundantly de-outlier it again.

Contributor guide

Open the contributing guide

Research direction

Start at the synapse_port_db implementation and its append-only table list; inspect how events rows are copied on a rerun and how outlier status is handled. Reproduce a de-outlier between runs if possible, then check whether the target state matches the source and add or adjust coverage for the observed behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.