element-hq / element-hq/dendrite
"enable_outbound" goes out of control and pegs the CPU
- Dominant language
- Go
- Stars
- 965
- Forks
- 101
- PR merge metrics
- No merged PRs in 30d
Description
*This issue was originally created by [**@IngwiePhoenix**](https://github.com/IngwiePhoenix) at .*
### Background information
- **Dendrite version or git SHA**: 0.10.8+76db8e90
- **Monolith or Polylith?**: Monolith
- **SQLite3 or Postgres?**: Postgres
- **Running in Docker?**: No
- **`go version`**: 1.19.3
- **Client used (if applicable)**: Element, Cinny
### Description
- **What** is the problem: When `presence.enable_outbound` is enabled, my VPS goes under complete load, showing an "Average load" of 40. In `htop`, this means all cores are nothing but red bars. That is, if my SSH connection even survives...
- **Who** is affected: My server (and my VPS' hypervisor, probably)
- **How** is this bug manifesting: `presence.enable_outbound: true` shows a lot of `m.presence` events being sent. My guess: There is no rate limiting in place, meaning it will just infinitly barf them out - and, depending on how many servers/rooms are involved, this can easily become a lot. I am not in `#matrix:matrix.org`, but only in a few small channels - the Dendrite channels amongst them, IPFS, a few old Gitter ones and Telegram 1:1s.
- **When** did this first appear: Today. I was moving between verrsions and upgraded from 0.10.6 to .7 and then .8. Once the bug manifested, I changed to building from `main` in hopes of this being fixed...Well, it's not.
### Steps to reproduce
```
git clone ...
git checkout tags/v0.10.8
mv .git .gitx # Works around a broken build
./build.sh
mv .gitx .git
nano .../dendrite.yaml # set presence.enable_outbound: true
systemctl start matrix-dendrite # I use a SystemD unit
```
Ultimatively, I'd like to be able to see some performance stats logged to tell what exactly is making my server go absolutely crazy. For now, I have this disabled and it seems fine. But for better or for worse, I would like to enable it eventually.
Thanks and kind regards,
Ingwie
Contributor guide
Research direction
Reproduce the issue using Dendrite 0.10.8 or the reported main build with `presence.enable_outbound: true`, then observe CPU usage and outbound `m.presence` events. Start by tracing the presence outbound path and use performance statistics or logs to identify the loop or send rate involved. Done means outbound presence no longer pegs the CPU and its behavior is covered by verification.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100