timeline scroll issues again

### Steps to reproduce

1. go into #foundation-office:matrix.org
2. jump to around https://matrix.to/#/!dSMpkVKGgQHlgBDSpo:matrix.org/$a34e4zp5TIkHnjIK7-XTIEbxYGR2QRhAWPcsh5ta3mg?via=matrix.org&via=element.io&via=envs.net
3. try to scroll down
4. refuses to scroll down and jumps back up

=====================================

1. try to scroll up much past https://matrix.to/#/!dSMpkVKGgQHlgBDSpo:matrix.org/$SQDCDjnsnOEWQjT5-mGEgwqQi55pqnZLrjQuumoNlew?via=matrix.org&via=element.io&via=envs.net
2. element hangs and crashes

### Outcome

#### What did you expect?

to be able to scroll the room timeline

#### What happened instead?

element either bugs out or hangs and crashes, appears to be some sort of ram leak

see a lot of spam of
```
FetchHttpApi: <-- GET https://matrix.envs.net/_matrix/client/v3/sync?filter=xxx&timeout=xxx&set_presence=xxx&since=xxx [517ms 200] rageshake.ts:74:16
```
and some
```
=$8q-fYysMuJ2gDBRiMx08ULIIlkdGS9BfCkgijCHU9SY that does not belong in timeline=!dSMpkVKGgQHlgBDSpo:matrix.org:2023-12-14T00:11:52.611Z timelineSet(threadId=undefined) rageshake.ts:74:16
Already have timeline for $qIctvOIk-sQQb20P-edKKW7g_o8OEULZECAmqm2Pue4 - joining timeline !dSMpkVKGgQHlgBDSpo:matrix.org:2023-12-14T00:11:52.611Z to !dSMpkVKGgQHlgBDSpo:matrix.org:2023-12-14T00:10:43.319Z rageshake.ts:74:16
```

in browser console

I'll try to send a /rageshake of the bug i get scrolling down into the affected area of the timeline, however i cannot send one of the hang and crash because it hangs the entire firefox tab and even operating the firefox console is impossible

screen recording: https://youtu.be/Kh36kpZC8wA

### Operating system

Fedora Linux 38 (KDE Plasma) x86_64

### Browser information

Mozilla Firefox version 120.0.1 (64 bit)

### URL for webapp

app.element.io

### Application version

Element version: 1.11.51 Crypto version: Olm 3.2.15

### Homeserver

envs.net

### Will you send logs?

Yes
```[tasklist]
### Tasks
```