element-hq / element-hq/synapse
Database migrated with `synapse_port_db` fails consistency checks
- Dominant language
- Python
- Stars
- 4.6k
- Forks
- 600
- Avg merge
- 5d 22h
- Merged PRs (30d)
- 51
Description
### Description
Migrating a valid and recent sqlite database with the current version of `synapse_port_db` results in inconsistent PostgreSQL database state that fails consistency checks on startup.
### Steps to reproduce
I upgraded synapse to the current (docker container) release, stopped the synapse container gracefully, changed the config to use a PostgreSQL 17 database, and ran:
```sh
docker compose exec postgres createdb -U postgres --encoding=UTF8 --locale=C --template=template0 --owner=synapse synapse
docker compose run --rm --entrypoint "" -w /data synapse synapse_port_db --curses --sqlite-database homeserver.db --postgres-config homeserver.yaml`.
```
This took a while, but showed no errors.
After that, I restarted the synapse container (now with a valid `psycopg2` config) and got the following fatal error:
```
[...]
File "/usr/local/lib/python3.13/site-packages/synapse/storage/util/sequence.py", line 192, in check_consistency
raise IncorrectDatabaseSetup(
...<2 lines>...
)
synapse.storage.engines._base.IncorrectDatabaseSetup:
Postgres sequence 'device_inbox_sequence' is inconsistent with associated stream position
of 'to_device' in the 'stream_positions' table.
This is likely a programming error and should be reported at
https://github.com/matrix-org/synapse.
A temporary workaround to fix this error is to shut down Synapse (including
any and all workers) and run the following SQL:
DELETE FROM stream_positions WHERE stream_name = 'to_device';
This will need to be done every time the server is restarted.
```
I switched back to the original sqlite3 config and database file and let synapse run for a day, then I deleted and re-created the PostgreSQL database and repeated the process. Same result.
### Homeserver
defnull.de
### Synapse Version
1.147.0
### Installation Method
Docker (matrixdotorg/synapse)
### Database
Does not apply
### Workers
Single process
### Platform
docker composed based container setup, running on debian
### Configuration
Nothing special.
### Relevant log output
```shell
See above
```
### Anything else that would be useful to know?
I migrated everything in one go and stopped the synapse server for the entire duration, so any bugs related to incremental migrations or state changes between partial migrations can be ruled out.
Contributor guide
Assessment
This issue has not been assessed yet.