Federation: history is broken when an invited member joins a room that he left previously

This issue has been migrated from [#3515](https://github.com/matrix-org/synapse/issues/3515).

---

TL;DR: history may be missing between a federated user being re-invited to a room and re-joining it.

### Description

A part of the room history is missing when an invited member joins a room that he left previously,
**if the room has been created on another homeserver than the member's one**.
Note: The e2e encryption is **disabled** in this room.

### Steps to reproduce

- Consider 2 users userA and userB, created on 2 different Homeservers.
- userA creates a room, history_visibility is shared (default value)
- userA invites userB
- userA sends some messages
- userB joins, he can read the messages OK
- userA and userB send some messages OK
- userB leaves the room
- userA sends a message M1
- userA invites again userB
- **userA sends a message M2**
- userB joins, the messages M1 and M2 are missing in the room history (I checked we did not get them in the server sync response after userB joined the room) **NOK**

Note1: The room history is restored correctly if we trigger an initial sync. Indeed the messages M1 and M2 are then provided in the server sync response.

Note2: You can send more than 1 message with the message M2. The problem is the same, userB don't receive them when he joins. He gets them only by triggering an initial sync.

Note3: **This issue is not reproduced when userA and userB belong to the same homeserver.**