Huge message fetch delay in rooms

### Steps to reproduce

1. Open the app and let it sync
2. I tap on a room I want to check
3. Bunch of them is loaded already in advance, but when I go checking through all of the rooms some are unable to fetch new messages and get stuck on a loading loop.

### Outcome

#### What did you expect?
Load the messages after tapping the room under 3-5 secs

#### What happened instead?
Loading loop which has ended after tapping on the back button twice (after going back and forth) and waiting around 20-30 seconds

### Your phone model

OnePlus 8

### Operating system version

Android 13

### Application version and app store

Element X G-Play 25.04.0

### Homeserver

Matrix.org

### Will you send logs?

Yes

### Are you willing to provide a PR?

No