/sync should not include a room in 'leave' and 'invite'

This issue has been migrated from [#14683](https://github.com/matrix-org/synapse/issues/14683).

---

### Description

When a User A kicks a User B and invites User B directly again, without User B syncing in between (because the client-app was closed for example), then the /sync response for User B includes the room in both `rooms.invite` and `rooms.leave`.

Since the client by default cannot rely on `origin_server_ts` for the order of events as there could be multiple servers with different time, some clients (like matrix-js-sdk) process the events in the wrong order (first the invite, then the leave) resulting in a wrong state on the client. The corresponding room cannot be used to chat, as the client thinks he left, when in reality he has an invitation to the room.

If the room is only inlcuded in either "leave" or "invite" (whichever is the newest event), then the client would always have the correct state

Here is a result of `/sync` for User B:

/sync Result

```json
{
"next_batch": "s127655_2818331_4835_288127_209897_85_169_14999_0",
"presence": {
"events": [
{
"type": "m.presence",
"sender": "@ct_a18c34c2-ea65-4359-956e-b4c237c96e47:chat.krz.tools",
"content": {
"presence": "online",
"last_active_ago": 17,
"currently_active": true
}
}
]
},
"device_lists": {
"left": [
"@ctbot:chat.krz.tools"
]
},
"device_one_time_keys_count": {
"signed_curve25519": 0
},
"org.matrix.msc2732.device_unused_fallback_key_types": [],
"device_unused_fallback_key_types": [],
"rooms": {
"invite": {
"!sQgeWUrcqXvxvzkUqw:chat.krz.tools": {
"invite_state": {
"events": [
{
"type": "m.room.avatar",
"state_key": "",
"content": {
"url": ""
},
"sender": "@ctbot:chat.krz.tools"
},
{
"type": "m.room.canonical_alias",
"state_key": "",
"content": {
"alias": "#ctg_4b1a00f5-7753-469f-bf95-6bb3b7c1414d:chat.krz.tools"
},
"sender": "@ctbot:chat.krz.tools"
},
{
"type": "m.room.create",
"state_key": "",
"content": {
"room_version": "9",
"creator": "@ctbot:chat.krz.tools"
},
"sender": "@ctbot:chat.krz.tools"
},
{
"type": "m.room.join_rules",
"state_key": "",
"content": {
"join_rule": "invite"
},
"sender": "@ctbot:chat.krz.tools"
},
{
"type": "m.room.name",
"state_key": "",
"content": {
"name": "Chattest"
},
"sender": "@ctbot:chat.krz.tools"
},
{
"type": "m.room.topic",
"state_key": "",
"content": {
"topic": "Gruppen-Chat einer Gruppe in testsilas.krz.tools"
},
"sender": "@ctbot:chat.krz.tools"
},
{
"type": "m.room.member",
"state_key": "@ctbot:chat.krz.tools",
"content": {
"membership": "join"
},
"sender": "@ctbot:chat.krz.tools"
},
{
"type": "m.room.member",
"sender": "@ctbot:chat.krz.tools",
"content": {
"membership": "invite",
"displayname": "Admin Admin"
},
"state_key": "@ct_a18c34c2-ea65-4359-956e-b4c237c96e47:chat.krz.tools",
"origin_server_ts": 1671178716243,
"unsigned": {
"replaces_state": "$LbfQGkLijKPasqYizadgfKnvLW_OKYK4izrqDLegduU",
"prev_content": {
"reason": null,
"membership": "leave"
},
"prev_sender": "@ctbot:chat.krz.tools",
"age": 41096
},
"event_id": "$elHhtOnT4Xy9mdNaHeJzVu5nVhKnRbjDEiFpN8BxBrc"
}
]
}
}
},
"leave": {
"!sQgeWUrcqXvxvzkUqw:chat.krz.tools": {
"timeline": {
"events": [
{
"type": "m.room.member",
"sender": "@ctbot:chat.krz.tools",
"content": {
"reason": null,
"membership": "leave"
},
"state_key": "@ct_a18c34c2-ea65-4359-956e-b4c237c96e47:chat.krz.tools",
"origin_server_ts": 1671178710797,
"unsigned": {
"replaces_state": "$pTOIJv3ViDwR7QyzOCLQatCwUyv5fQpNYSOaDIHXcdI",
"prev_content": {
"membership": "join",
"displayname": "Admin Admin"
},
"prev_sender": "@ct_a18c34c2-ea65-4359-956e-b4c237c96e47:chat.krz.tools",
"age": 46542
},
"event_id": "$LbfQGkLijKPasqYizadgfKnvLW_OKYK4izrqDLegduU"
}
],
"prev_batch": "s127652_2818310_4829_288126_209897_85_169_14999_0",
"limited": false
},
"state": {
"events": []
},
"account_data": {
"events": []
}
}
}
}
}
```

### Steps to reproduce

- User A and B are in a room together
- User B stops his client
- User A kicks User B
- User A invites User B back in
- User B starts his client
- Client of User B syncs and gets the wrong sync response

### Homeserver

chat.krz.tools

### Synapse Version

1.73.0

### Installation Method

I don't know

### Database

PostgreSQL and the rest I don't know

### Workers

Single process

### Platform

running in a vm as far as I know

### Configuration

_No response_

### Relevant log output

```shell
-
```

### Anything else that would be useful to know?

_No response_