Plotly graphs in jupyter notebooks re-render entire page html

When attempting to run the first graph in the getting started documents, after following the steps here, the entire page html is output ontop of the graph like in the screenshot below. This occurs for all other graphs I've tried as well.

library(plotly)
fig <- plot_ly(midwest, x = ~percollege, color = ~state, type = "box")
fig

Steps to Reproduce:

1. pull the jupyter/r-notebook docker image from here with the command docker pull jupyter/r-notebook
   a. This allows you to skip the "add r-kernel" steps because the R kernel is included with the r-notebook. Specifically these two lines can be skipped:

devtools::install_github('IRkernel/IRkernel')
IRkernel::installspec()

Alternatively, start with the jupyter/minimal-notebook and do the steps to add the R kernel.

2. Run the docker image with docker run -p '8888:8888' -e JUPYTER_ENABLE_LAB=yes jupyter/r-notebook
3. Open an R console and run install.packages(c('repr', 'IRdisplay', 'pbdZMQ', 'devtools')) from the setup steps
4. Install pandoc - get to the container bash as root user. First get the container ID with docker ps then use the ID you see there in docker exec -u root -ti <container id> bash. Run apt update then apt install pandoc.
   a. Open the notebook from the link that step 2 output and you can verify pandoc is installed in /usr/bin/pandoc by opening a terminal and navigating to the /usr/bin directory.
5. Install plotly - either use the R console or the terminal. Personally, I used the terminal and ran conda install r-plotly. I attempted with both the latest version and also with v4.9.2.2. Both showed the issue.
6. Once that's all setup, open an R-kernel notebook and attempt to create any plotly graph. It will initially load the graph correctly, then a few seconds later, you will see the issue from the screenshot above.

Start with the linked R-in-Jupyter getting-started instructions and reproduce the example in a jupyter/r-notebook Docker container using an R-kernel notebook. Compare the initial graph rendering with the later output described in the issue; done means Plotly graphs render without appending the entire page HTML over the graph.