Error: not invited when joining too quickly

This issue has been migrated from [#2367](https://github.com/matrix-org/synapse/issues/2367).

---

On the Matrix-IRCnet bridge some people have trouble sending a message to a person.
The cause seems to be that if the invited party joins too quickly, the inviting homeserver denies the join.
The bridge automatically joins, resulting in 307 ms between sending the invite and receiving the join.

The resulting response is `SynapseError: 403 - You are not invited to this room`.
[log.txt](https://github.com/matrix-org/matrix-appservice-irc/files/1144917/log.txt) contains the debug log for the two requests (creating/inviting and joining).

On the server, I see the following events:

```
| event_id | type | content
---------------------------+---------------------------+---------
$14999418670pJBSM:slxh.eu | m.room.create | {"creator":"@silex:slxh.eu"}
$14999418681RDRYI:slxh.eu | m.room.member | {"avatar_url":"mxc://slxh.eu/firYUbGCedTKEeYmayUSnKUz","displayname":"Silke","membership":"join"}
$14999418682SbYPV:slxh.eu | m.room.power_levels | {"ban":50,"events":{"m.room.avatar":50,"m.room.canonical_alias":50,"m.room.history_visibility":100,"m.room.name":50,"m.room.power_levels":100},"events_default":0,"invite":0,"kick":50,"redact":50,"state_default":50,"users":{"@_ircnet_^^_^^:irc.snt.utwente.nl":100,"@silex:slxh.eu":100},"users_default":0}
$14999418693oWJMx:slxh.eu | m.room.join_rules | {"join_rule":"invite"}
$14999418704rTiYM:slxh.eu | m.room.history_visibility | {"history_visibility":"shared"}
$14999418715zKlyB:slxh.eu | m.room.guest_access | {"guest_access":"can_join"}
$14999418736ROTWn:slxh.eu | m.room.member | {"avatar_url":null,"displayname":"^^_^^ (IRC)","is_direct":true,"membership":"invite"}
```

The server then creates (and signs) the following event.
Note that the `prev_events` is actually one event too early.

```json
{
"auth_events": [
["$14999418670pJBSM:slxh.eu", {
"sha256": "h4wQhafYBJ0hw0pByePewTZ/Wisd1mlYYy7b3+jYjkw"
}],
["$14999418682SbYPV:slxh.eu", {
"sha256": "lMBl4s8tP82N2cRY3AvmMeOpO4T8U0/YClTiQazi0V8"
}],
["$14999418693oWJMx:slxh.eu", {
"sha256": "aEkbGx/4vEZ6dPdm/+5PdLSEhnq7U0uDqyUIWyihYxU"
}]
],
"content": {
"membership": "join"
},
"depth": 7,
"event_id": "$14999418747tuojQ:slxh.eu",
"hashes": {
"sha256": "ZiMyi5EnmrKtVdUm0WpPmL6kD9EWLGhrDyKcDMcYZEU"
},
"origin": "slxh.eu",
"origin_server_ts": 1499941874430,
"prev_events": [
["$14999418715zKlyB:slxh.eu", {
"sha256": "VJCVgu/nkJJQ3/KAstVu/xbVyD8F+ve7iY4faRsje+U"
}]
],
"prev_state": [],
"room_id": "!pJHIzJXCnKbrJoTqKR:slxh.eu",
"sender": "@_ircnet_^^_^^:irc.snt.utwente.nl",
"signatures": {},
"state_key": "@_ircnet_^^_^^:irc.snt.utwente.nl",
"type": "m.room.member"
}
```