Room list loses "latest" messages/constantly reorders itself and timelines clear and reload infinitely

### Steps to reproduce

1. Clear the cache
2. Watch the room list spool in, then out, and then constantly reorder itself
3. Open a room, observe the spinner and notice that occasionally the timeline appears briefly before disappearing again.

#### Other interesting thins to note
- Enabling/disabling auto-backpagination in developer options seems to make no difference

### Outcome

#### What did you expect?
Anything but this

#### What happened instead?
The app is borderline unusable until restarted. Room timelines are also sometimes *really* slow to load when opening rooms sometimes (3-5s)

I've DM'ed a video of this happening to Jorge and submitted a rageshake that mentions this issue

### Your phone model

Pixel 8 Pro

### Operating system version

Pixel something or other

### Application version and app store

Built with https://github.com/frebib/element-x-android/commits/c5955e8fb3

### Homeserver

nerdhouse.io; Synapse 1.153.0rc1 (at the time of writing)

### Will you send logs?

Yes

### Are you willing to provide a PR?

No