New users dashboard experience
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 778
- Avg merge
- 2h 50m
- Merged PRs (30d)
- 3
Description
I just helped a Dask user get the Dashboard running during the SciPy sprints. It was pretty painful and had a bunch of failures along the way that made it hard. I wanted to capture that here in case there are things we can do to improve this experience.
- The user was running Jupyter Lab on a remote machine via SSH, they set up SSH port forwarding to access Jupyter
- They created a Dask cluster with `client = Client()`
- **Failure 1** They clicked the link in the widget that said `http://localhost:8787/status`, which wasn't port forwarded
- **Failure 2** They added `8787` to the port forward list to fix this but some restriction in their org setup meant this didn't work (I think `8888` may have been on an allowlist but `8787` wasn't)
- **Failure 3** While testing they continued to run `client = Client()` which created many clusters and they were confused why the port kept changing
- @jsignell and I got involved at this point to help out. My suggestion was to use [jupyter-server-proxy](https://github.com/jupyterhub/jupyter-server-proxy) so that we could access the dashboard via Jupyter which was known to be working
- **Failure 4** We stopped Jupyter, ran `pip install jupyter-server-proxy` and started Jupyter again. Sadly Jupyter was running from the base environment and we pip installed into the activated environment. So this didn't take effect and took a long time to debug but we eventually got things installed in the right environment (with help from @yuvipanda).
- **Failure 5** Installing `jupyter-server-proxy` upgraded Jupyter which broke the user's Jupyter config and also took a while to debug.
- **Failure 6** Finally we got back into Jupyter, started the Dask cluster and the user clicked the link in the widget again, which was still saying `http://localhost:8787/status` and didn't work
- We then manually navigated to `http://localhost:8888/proxy/8787/status` and things worked 🎉
- Finally we added a config line to fix the widget `dask.config.set({"distributed.dashboard.link": "/proxy/{port}/status"})`
This was an unpleasant experience for a first-time Dask user. It was time-consuming to debug for three maintainers from across Dask and Jupyter.
Contributor guide
Assessment
This issue has not been assessed yet.