Panic in narrow_phase after "pause" using rapier_config.physics_pipeline_active = false

Hello!

Im getting a panic after resuming the RapierConfiguration `physics_pipeline_active` after having it set to false for a while. Basically, I pause the simulation as the player is presented with a level-up window. Once it is closed I resume the physics simulation (among other things). Every now and then (1 out of 100?) I get bitten by a panic on this line:

https://github.com/dimforge/bevy_rapier/blob/master/src/plugin/narrow_phase.rs#L307

My code causing the call to collider2() is basically:
```
for contact_pair in rapier_context.contacts_with(player_entity) {
let other_collider = if contact_pair.collider1() == player_entity {
contact_pair.collider2()
} else {
contact_pair.collider1()
};
...
}
```
The code lives in a system that runs every frame.

It feels like it might be some weird condition in the frame where bevy state is changed from ingame -> levelup and the removal of an entity that would have been collider2() in that same frame. Perhaps it mucks up the state in contacts vs what the bevy-rapier "sync removals" method removes. It seems like just as it unpauses and the system that panics runs it'll look at stale collider data that still has a collision with an entity that was removed in that frame where the gamestate changed (simulation paused). Am using version 0.13.1.