element-hq / element-hq/element-web

Element never flushes accumulated non-LL room members, and so slowly uses more ambient RAM.

Open
#25,264 1 comment 1 reaction 0 assignees View on GitHub
A-Performance O-Uncommon S-Major T-Defect
Dominant language
TypeScript
Stars
13.5k
Forks
2.8k
PR merge metrics
PR metrics pending

Description

### Steps to reproduce

1. My Element Desktop kept whitescreening overnight, as per https://github.com/vector-im/element-desktop/issues/680
2. It turns out my v8 heap was idling around 1.3GB (1.1GB after GC), perilously close to the 1.5GB limit - meaning that if sync accumulator persists overlap in sqlite when the laptop is asleep, we can OOM.
3. 1.3GB seems high; i'd expect more like 500MB
4. Turns out I had 418K RoomMember objects on the heap, which is massive.
5. On a fresh launch, i have 370K, which is still massive.
6. Turns out there are 158K room members pulled in from the accumulated sync response:
```
let count = 0; for (r of mxMatrixClientPeg.get().getRooms()) { count += Object.keys(r.currentState.members).length } console.log(count);
15:46:12.123 rageshake.ts:74 158074
```
Presumably we get ~2x of these on the heap - perhaps one for prevState and currentState per room, or similar.

So the problem seems to be that we never ever flush non-LL members from the sync accumulator. So if you idle in Matrix HQ in a login for a year, then the persisted room state gradually grows as it sees members appear in the timeline. (Lazily loaded members don't get persisted). So on my account it had gone from 7 members to 11,000 members - and similarly in other busy rooms, hence increasing the 'actual' LL subset of 17K members by ~10x up to 158K members.

While looking into this i saw some other memory leaks too:
* lots of detached HTML DOM nodes (e.g. 2152x detached HTMLDivElements)
* 2533x logger instances
* 16118x SummarizedNotificationState_SummarizedNotificationState instances
* window.mxSettingsStore is somehow retaining 140MB of native context (v8 JITed code?)
* client.reEmitters ends up with 480K-odd keys in its map, and grows forever, and is so unwieldy that trying to inspect it causes an OOM.

For the main bug (slowly leaking members), about the only actual solution that comes to mind that we could occasionally flush the accumulated m.room.member events, and toggle the `include_redundant_members` field on the /sync filter to persuade the server to start resending non-LL members to us. (Assuming the server flushes the LL LRU cache on seeing `include_redundant_members`)

Meanwhile, sliding sync (or a Hydrogen or rust-sdk style architecture where we don't store members in RAM) is clearly the better solution - i.e. killing the sync accumulator entirely.

I propose not fixing this, in favour of landing SS instead, and meanwhile the workaround is for powerusers to logout and login again once a year or so.

### Outcome

#### What did you expect?

App to not OOM overnight

#### What happened instead?

Many many OOMs.

### Operating system

macOS 13.3.1

### Browser information

Chrome

### URL for webapp

element.io

### Application version

Element Nightly version: 2023041901 Olm version: 3.2.12

### Homeserver

matrix.org

### Will you send logs?

No

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.