element-hq / element-hq/synapse

Database migrated with `synapse_port_db` fails consistency checks

Open
#19,467 3 comments 4 reactions 0 assignees View on GitHub
A-Database A-Portdb A-to-device-messages
Dominant language
Python
Stars
4.6k
Forks
600
Avg merge
5d 22h
Merged PRs (30d)
51

Description

### Description

Migrating a valid and recent sqlite database with the current version of `synapse_port_db` results in inconsistent PostgreSQL database state that fails consistency checks on startup.

### Steps to reproduce

I upgraded synapse to the current (docker container) release, stopped the synapse container gracefully, changed the config to use a PostgreSQL 17 database, and ran:

```sh
docker compose exec postgres createdb -U postgres --encoding=UTF8 --locale=C --template=template0 --owner=synapse synapse

docker compose run --rm --entrypoint "" -w /data synapse synapse_port_db --curses --sqlite-database homeserver.db --postgres-config homeserver.yaml`.
```

This took a while, but showed no errors.

After that, I restarted the synapse container (now with a valid `psycopg2` config) and got the following fatal error:

```
[...]
File "/usr/local/lib/python3.13/site-packages/synapse/storage/util/sequence.py", line 192, in check_consistency
raise IncorrectDatabaseSetup(
...<2 lines>...
)
synapse.storage.engines._base.IncorrectDatabaseSetup:
Postgres sequence 'device_inbox_sequence' is inconsistent with associated stream position
of 'to_device' in the 'stream_positions' table.

This is likely a programming error and should be reported at
https://github.com/matrix-org/synapse.

A temporary workaround to fix this error is to shut down Synapse (including
any and all workers) and run the following SQL:

DELETE FROM stream_positions WHERE stream_name = 'to_device';

This will need to be done every time the server is restarted.
```

I switched back to the original sqlite3 config and database file and let synapse run for a day, then I deleted and re-created the PostgreSQL database and repeated the process. Same result.

### Homeserver

defnull.de

### Synapse Version

1.147.0

### Installation Method

Docker (matrixdotorg/synapse)

### Database

Does not apply

### Workers

Single process

### Platform

docker composed based container setup, running on debian

### Configuration

Nothing special.

### Relevant log output

```shell
See above
```

### Anything else that would be useful to know?

I migrated everything in one go and stopped the synapse server for the entire duration, so any bugs related to incremental migrations or state changes between partial migrations can be ruled out.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.