Timeline problems: jumping up and down and infinitely loading

### Steps to reproduce

1. Enable `/jumptodate`
2. Join the Office of the Matrix.org Foundation
3. `/jumptodate 2024-09-01`
4. Try scrolling down. Notice frantic jumping of the timeline.
5. `/jumptodate 2024-09-02`
6. Try scrolling up. Notice infinite loading and slowdown.

### Outcome

#### What did you expect?

For the timeline to scroll, smoothly.

#### What happened instead?

It did not.

Some potentially related log snippets:

TimelinePanel.tsx:91 Uncaught TypeError: Cannot read properties of undefined (reading 'localTimestamp')
at vd (TimelinePanel.tsx:91:18)
at TimelinePanel.tsx:531:53
at Vt (lodash.js:816:11)
at $i (lodash.js:7372:14)
at Object.onUnfillRequest (TimelinePanel.tsx:531:19)
at ScrollPanel.tsx:469:17

rageshake.ts:77 RangeError: Invalid array length
at Array.push ()
at L.getEvents (timeline-window.ts:421:24)
at _d.getEvents (TimelinePanel.tsx:1696:49)
at TimelinePanel.tsx:627:49

### Operating system

Arch Linux

### Application version

Element version: 1.11.75 Crypto version: Rust SDK 0.7.1 (c8c9d15), Vodozemac 0.6.0

### How did you install the app?

Arch Linux repo

### Homeserver

_No response_

### Will you send logs?

Yes