T&S: Fix retracted invites

### Your use case

I receive a (potentially) abusive invite. I just ignore it (doing nothing, leaving it in my invites inbox) for now because it seems sus.

After 2 weeks or so this room has been moderated/removed/user been deactivated - anyway somehow the invite has become invalid - and I can neither accept nor decline it anymore (let alone #3655).

### Have you considered any alternatives?

Clearing cache, relogging or whatever is *not* a fix.

### Additional context

Using SchildiNext 0.6.5.sc12-test5 sc_testing_fdroid

### Are you willing to provide a PR?

No