Question about performance

Describe the issue

I realise there has already been several issues opened about the performance of ipympl (e.g. #55), compared to other libraries that render in jupyter. But I still don't fully understand why the performance is so much worse than using e.g. the %matplotlib qt backend.

Take for example the 'Looking glass' example from the matplotlib docs.
The ineractivity is very nice in the Qt backend (more than enough for our purposes, even if not quite as snappy as javascript based libs such as bokeh or plotly), but terrible in jupyter lab (laggy and leaving phantom parts of the ellipse around the plot).

Is there really no way to improve the performance of interactivity in ipympl? (I have tried to use fig.canvas.draw_idle() instead of .draw(), and it helps a little but we are still so far behind the Qt backend.

Many thanks.

Versions

 3.9.5 (default, Jun  4 2021, 12:28:51) 
[GCC 7.5.0]
ipympl version: 0.8.2
Selected Jupyter core packages...
IPython          : 7.29.0
ipykernel        : 6.5.0
ipywidgets       : 7.6.5
jupyter_client   : 7.0.6
jupyter_core     : 4.9.1
jupyter_server   : 1.11.2
jupyterlab       : 3.2.3
nbclient         : 0.5.8
nbconvert        : 6.3.0
nbformat         : 5.1.3
notebook         : 6.4.5
qtconsole        : not installed
traitlets        : 5.1.1
Known nbextensions:
  config dir: /home/nvaytet/software/miniconda3/etc/jupyter/nbconfig
    notebook section
      ipycanvas/extension  enabled 
      - Validating: OK
      ipyevents/extension  enabled 
      - Validating: OK
      jupyter-datawidgets/extension  enabled 
      - Validating: OK
      jupyter-matplotlib/extension  enabled 
      - Validating: OK
      jupyter-threejs/extension  enabled 
      - Validating: OK
      jupyter_bokeh/extension  enabled 
      - Validating: OK
      jupyter_dash/main  enabled 
      - Validating: OK
      jupyterlab-plotly/extension  enabled 
      - Validating: OK
      jupyter-js-widgets/extension  enabled 
      - Validating: OK
JupyterLab v3.2.3
/home/nvaytet/software/miniconda3/share/jupyter/labextensions
        ipycanvas v0.12.0 enabled OK
        ipyevents v2.0.1 enabled OK
        jupyter-matplotlib v0.10.2 enabled OK
        jupyterlab-datawidgets v7.0.0 enabled OK
        jupyterlab-plotly v5.4.0 enabled OK
        jupyter-threejs v2.3.0 enabled OK (python, pythreejs)
        @jupyter-widgets/jupyterlab-manager v3.0.1 enabled OK (python, jupyterlab_widgets)
        @bokeh/jupyter_bokeh v3.0.2 enabled OK (python, jupyter_bokeh)

Other labextensions (built into JupyterLab)
   app dir: /home/nvaytet/software/miniconda3/share/jupyter/lab
        jupyterlab-dash v0.4.0 enabled OK


Build recommended, please run `jupyter lab build`:
    jupyterlab-dash needs to be included in build

Start by reproducing the Looking Glass example in JupyterLab with ipympl and compare its interaction with the Qt backend. Examine the reported draw_idle behavior and the listed JupyterLab, ipympl, and widget versions; done means identifying a concrete performance cause and defining a measurable improvement in interactive rendering.