Ghost Rooms when changing DB Types

*This issue was originally created by [**@narashini**](https://github.com/narashini) at .*

### Background information

- **Dendrite version or git SHA**: 01d71a767dbd11fd7b17b948db6d3052f577c796
- **Monolith or Polylith?**: Monolith
- **SQLite3 or Postgres?**: Postgres
- **Running in Docker?**: Nope
- **`go version`**: go1.17.8 linux/arm64
- **Client used (if applicable)**: Element Desktop

### Description

- **What** is the problem: Ghost Rooms
- **Who** is affected: I would assume other people have had the issue, I've just come across it ~20m ago (I'm running the dendrite server.)
- **How** is this bug manifesting: When the server switched from SQlite to PostgreSQL, I got a room called "Red-eyed empire" added to my Historical list on Element. I never joined this room, on SQlite nor on the PostgreSQL DB's
- **When** did this first appear: 12.3.22 (uk format, DD.MM.YY)

### Steps to reproduce
(this is how i came across the bug)
- Setup a Dendrite server with SQlite
- Join a few rooms, chat in them, keep it running for around 20m
- Shut the server down
- Create a new PostgreSQL DB + user
- Set Dendrite up to use PostgreSQL
- Start the server
- Create yourself an admin user
- Login with element
- There should be a random room in the "Historical" section that you never joined on any network.

Describe how what happens differs from what you expected.
There should not be random ghost rooms when you change DB Types from SQLite to PostgreSQL.