element-hq / element-hq/dendrite

Heavy Disk Usage of Postgres-Database

Open
#2,464 1 comment 0 reactions 0 assignees View on GitHub
C-Postgres C-SQLite T-Defect X-Performance
Dominant language
Go
Stars
965
Forks
101
PR merge metrics
No merged PRs in 30d

Description

*This issue was originally created by [**@genofire**](https://github.com/genofire) at .*

### Background information

- **Dendrite version or git SHA**: 0.8.5
- **Monolith or Polylith?**: Monolith
- **SQLite3 or Postgres?**: PostgreSQL
- **Running in Docker?**:
- **`go version`**:
- **Client used (if applicable)**:

### Description

- **What** is the problem: the database use 30GB (12GB more, after 2 days)
- **Who** is affected:
- **How** is this bug manifesting:
- **When** did this first appear:

```
dendrite_sum7=# SELECT nspname || '.' || relname AS "relation",
pg_size_pretty(pg_total_relation_size(C.oid)) AS "total_size"
FROM pg_class C
LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace)
WHERE nspname NOT IN ('pg_catalog', 'information_schema')
AND C.relkind <> 'i'
AND nspname !~ '^pg_toast'
ORDER BY pg_total_relation_size(C.oid) DESC
LIMIT 20;
relation | total_size
--------------------------------------------+------------
public.roomserver_state_block | 11 GB
public.syncapi_output_room_events | 6917 MB
public.roomserver_event_json | 914 MB
public.syncapi_current_room_state | 636 MB
public.roomserver_events | 287 MB
public.roomserver_previous_events | 234 MB
public.syncapi_memberships | 131 MB
public.syncapi_output_room_events_topology | 129 MB
public.roomserver_state_snapshots | 91 MB
public.federationsender_joined_hosts | 85 MB
public.roomserver_event_state_keys | 60 MB
public.roomserver_membership | 51 MB
public.syncapi_presence | 38 MB
public.keyserver_device_keys | 34 MB
public.keyserver_one_time_keys | 32 MB
public.federationsender_queue_edus | 14 MB
public.keyserver_cross_signing_sigs | 8832 kB
public.roomserver_redactions | 5632 kB
public.syncapi_backward_extremities | 5488 kB
public.pushserver_notifications | 5200 kB
(20 rows)

```

Contributor guide

Open the contributing guide

Research direction

Start with the PostgreSQL size query in the report and investigate the listed roomserver and syncapi tables in the Dendrite codebase. The report does not name source files, tests, or an expected fix; done would require identifying why usage grows and documenting or validating a specific remediation.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, postgresql
Domain
backend, databases
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.