dask / dask/dask

Proxying Dask (Bokeh) Web Interface on AWS SageMaker

Open
#5,432 27 comments 7 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
13.9k
Forks
2k
PR merge metrics
No merged PRs in 30d

Description

Hi,
I am using Amazon SageMaker Instances to run self-contained JupyterLab environments.
I can run dask no problem, but it would be great to view the bokeh dashboard in my browser.
I have so far looked into two approaches (see below) but **would welcome any solutions/ideas anyone has**:

### 1. **Port Forwarding**: (as suggested [here](https://distributed.dask.org/en/latest/web.html#id1))
Unfortunately, (to the best of my knowledge), AWS Sagemaker Notebooks do not allow SSH access, and hence Port Forwarding is not an option
> NOTE: Google AI Notebook Instances **do** however and so this is a suitable solution for GCP `:8787/status`

### 2. Using a jupyter proxy extension [google-server-proxy](https://github.com/jupyterhub/jupyter-server-proxy)
This is close to a working solution, as I am able to view the dashboard, but am getting a websocket error
**When I attempt to access the dashboard using the proxy** `bokeh.protocol.exceptions.ProtocolError: No bokeh-protocol-version specified`
![image](https://user-images.githubusercontent.com/2926574/65598312-06df6100-df93-11e9-8aba-482ba06fc373.png)
**The exception raised from bokeh**
![image](https://user-images.githubusercontent.com/2926574/65598079-8b7daf80-df92-11e9-9fb1-0ea58e4ac21a.png)

#### Steps to reproduce
- Create a new notebook instance [SageMaker](https://eu-west-2.console.aws.amazon.com/sagemaker/home?region=eu-west-2#/notebook-instances/create)
- Install Dask + Bokeh in a py3.6 kernel
- Install jupyter-server-proxy extension
```
git clone --depth 1 https://github.com/jupyterhub/jupyter-server-proxy
cd jupyter-server-proxy/jupyterlab-server-proxy
npm install && npm run build && jupyter labextension link .
npm run build && jupyter lab build
```
- Create a new notebook and create a client
```python
from dask.distributed import Client
client = Client()
```
- Navigate to the proxied web-proxy:
```
# In web-browser
https:///proxy/8787/status
```
(Additional info on my runtime env)
![image](https://user-images.githubusercontent.com/2926574/65598017-66893c80-df92-11e9-8e3f-0c87b3d41931.png)

**Additional Comment**
I am a big advocate of dask and very much appreciate the ongoing work everyone is doing!

I understand this may not be the appropriate repo for this issue/feature-request and may not appear of high priority. However, I feel SageMaker Notebooks/GCP AI Notebook environments will be an increasing use case for lots of ML researchers and developers and, consequently, a well supported solution to this would be a useful addition to the docs.

Thanks,

David

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.