Bug switching backend between inline and qt or notebook

I routinely switch back and forth between `notebook` and `inline` backends. Occasionally the `notebook` backend does not show the figure but an empty space, and a static image is shown in the output area below.

Usually when this happens I get messages like this in the terminal:

```
[IPKernelApp] ERROR | No such comm: c1fa5e1e9de8413692476a2c8fdf34bb
```

It seems to me that the probability of seeing this bug increases with "heavy" plots. I tried to create a small notebook showing the problem:

https://gist.github.com/tritemio/a4554323fe0faca85c57

However most of the times the previous notebook only triggers the `No such comm` message but the `notebook` figure still shows correctly.

I have a much more complex code that triggers the bug much more often though.

attn: @tacaswell

Start with the linked reproduction gist and reproduce the failure while switching between the notebook and inline backends, watching for the terminal "No such comm" message. Done means the notebook figure renders instead of leaving empty space with a static image below, without the reported comm error.

Written by the indexing model from the issue text.