element-hq / element-hq/synapse
PostgreSQL database schema incompatible with AWS DMS
- Dominant language
- Python
- Stars
- 4.6k
- Forks
- 600
- Avg merge
- 5d 22h
- Merged PRs (30d)
- 51
Description
This issue has been migrated from [#15583](https://github.com/matrix-org/synapse/issues/15583).
---
**Description:**
AWS DMS (Data Migration Service) is a tool to migrate databases around. It looks like Synapse is incompatible with using it due to some tables with LOB columns which don't have a primary key as per [DMS docs](https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Troubleshooting.html#CHAP_Troubleshooting.Oracle.LOBChanges).
It may be a good idea to add the primary keys to the relevant tables to become compatible with AWS DMS, unless there is a technical reason not to do so. The problematic tables are:

Example DMS log lines for `event_auth`:
>2023-05-11T13:46:37 [SOURCE_UNLOAD ]W: Column 'event_id' was removed from table definition 'public.event_auth': the column data type is LOB and the table has no primary key or unique index (metadatamanager.c:2492)
2023-05-11T13:46:37 [SOURCE_UNLOAD ]W: Column 'auth_id' was removed from table definition 'public.event_auth': the column data type is LOB and the table has no primary key or unique index (metadatamanager.c:2492)
2023-05-11T13:46:37 [SOURCE_UNLOAD ]W: Column 'room_id' was removed from table definition 'public.event_auth': the column data type is LOB and the table has no primary key or unique index (metadatamanager.c:2492)
Contributor guide
Research direction
Begin with the PostgreSQL schema for event_auth and the other tables shown in the linked image, then compare their keys with the AWS DMS LOB requirements and the cited DMS documentation. Confirm whether adding primary keys is technically safe; done means the affected tables are accepted by DMS without the reported SOURCE_UNLOAD warnings.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, postgresql
- Domain
- cloud, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100