Unread count goes from zero through the roof after restarting the app

### Steps to reproduce

1. Go to inbox zero in https://matrix.to/#/#ps-features-team:matrix.org
2. Restart and relaunch Element Desktop
3. Open the room
4. Notice how the unread count slowly jumps up to ~500

### Outcome

#### What did you expect?

Lots of messages are marked unread

#### What happened instead?

No unread count in the room

### Operating system

macOS

### Application version

Element Desktop nightly

### How did you install the app?

_No response_

### Homeserver

element.io

### Will you send logs?

Yes