High resource usage over time (memory leak?)

### Background information

- **Dendrite version or git SHA**: 0.13.8+79b87c7
- **SQLite3 or Postgres?**: Postgres
- **Running in Docker?**: No
- **`go version`**: go version go1.23.0 linux/amd64
- **Client used (if applicable)**: Schildichat, Element, Hydrogen, or CinnyChat is what most people use I believe on the HS

### Description

- **What** is the problem: Dendrite, when running over time, begins hogging ram over time. This results in not only a big RAM hog, but also a CPU hog, since I run zRAM (maybe also related to dendrite using the CPU? Idk.) Regardless, every week or so I have to restart Dendrite because it keeps eating more and more RAM, making the overall server performance worse.
- **Who** is affected: The server.
- **How** is this bug manifesting: It appears as Dendrite runs in long-term, slowly eating more and more RAM and/or swap.
- **When** did this first appear: I can't recall. I don't remember needing to restart dendrite before 1.18 I think.

### Steps to reproduce

- list the steps
- that reproduce the bug
- using hyphens as bullet points

- Run dendrite
- Use it for a week or so
- Watch the resource, mainly RAM, usage grow over time

This is its resource usage only 2 days later after its most recent restart. And it only creeps over time until I restart it:


It's weird.