Async widget methods deadlock due to on_msg not being called

We are developing a Jupyter widget which makes it possible to [embed interactive Unfolded maps into notebooks](https://docs.unfolded.ai/jupyter):


Our widget is embedded via an iframe and so the communications with the JavaScript side are asynchronous. Some of the functions we want to add to the widget need to return data asynchronously. To communicate between Jupyter and JS we are using widget's `send` and `on_msg` methods. Our methods return `future`s. It works fine - the messages are being sent and received, but only if we do it in two steps:


When we are trying to use `await` so that we can have the results and do something with them within one cell, the execution is blocked forever:


We tried to investigate the cause. It turns out that the callback we register with `widget.on_msg` only gets called once the execution of the cell is finished. Hence, if we wait until we obtain the result of the calculation within the same cell where the method was called, we end up in a deadlock.

So far we've had no success in finding a workaround, despite trying many things. For instance, here we run the communication in a separate thread and wait until the query response is added to a blocking queue:

Calling the above function fails (after the 2 seconds timeout) with an error saying that the queue is empty:
And we don't see the `_receive` function being called.

However, if we check the queue immediately after getting this error, we can see that the queue is not empty - the response was added to it, apparently, *_just after_* the error caused the cell execution to be cancelled:

It might be the same issue as this one: https://github.com/ipython/ipython/issues/12786
but I am not completely sure. I tried installing and running the kernel with the fix from [this PR](https://github.com/ipython/ipykernel/pull/589), but I was getting errors related to traitlet messaging when trying to instantiate our widget.