element-hq / element-hq/synapse
Streams tables are never cleared out
- Dominant language
- Python
- Stars
- 4.6k
- Forks
- 600
- Avg merge
- 5d 22h
- Merged PRs (30d)
- 51
Description
This issue has been migrated from [#5888](https://github.com/matrix-org/synapse/issues/5888).
---
A number of 'stream' tables are appended-to, but are never cleared out. Such tables include:
* ~~cache_invalidation_stream_by_instance~~ (fixed by https://github.com/matrix-org/synapse/pull/15868)
* current_state_delta_stream
* ~~presence_stream~~ (does get cleaned out; has been ever since its creation in https://github.com/matrix-org/synapse/pull/582)
* ~~device_inbox~~ (not really a 'stream' table actually, so a separate problem: #3599)
* device_federation_outbox
* ~~local_group_updates~~ (removed in https://github.com/matrix-org/synapse/pull/12967)
* push_rules_stream
* ex_outlier_stream
* ~public_room_list_stream~ (now removed)
* `device_lists_changes_in_room` (probably not really a stream table, #13043)
Contributor guide
Research direction
Start by auditing the remaining tables listed in the issue: current_state_delta_stream, device_federation_outbox, push_rules_stream, and ex_outlier_stream. Trace how each table is written and whether any cleanup path exists; done requires an agreed, safe retention or cleanup approach for the tables that still accumulate rows.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, databases
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100