Authentication for dashboards
- Dominant language
- TypeScript
- Stars
- 329
- Forks
- 67
- PR merge metrics
- No merged PRs in 30d
Description
### The problem
Broadly speaking, this extension works by passing around URLs:
1. When a user has a cluster and clicks on the magnifying glass icon, it checks `client.dashboard_link` to find a URL for the dashboard, and uses that to fetch the individual plots endpoints and construct dashboard panes.
2. If a user creates a cluster, then tries to insert client connection code for that cluster into a notebook or text file, it grabs the scheduler address off of the cluster, and gives you a snippet like: `client = distributed.Client()`.
This URL-based system has gotten us pretty far, and has been useful for a lot of people. Both of the above work fine if your clusters aren't protected by authentication or other security measures. But if that is not the true (often the case for a cloud or HPC based cluster), then it will fail. That is to say, `Client("tls://some-url:8786")` will not connect without an SSL context, and a dashboard at `https://some-authenticated-service:8787/status` won't connect without authorization headers/tokens.
### Proposal
I don't think we will be able to (or will want to) cover authentication for all the manifold ways in which dask clusters are deployed today. But I think we can add an entrypoints-based plugin system to allow deployers of dask clusters to make sure that their services work with this package (similar to what @jacobtomlinson has done in `dask-ctl`, or what `intake` has done for drivers).
At it's most basic, I'm imagining a simple interface to allow library authors to say whether a given cluster belongs to them, and authenticate with their service as necessary. The basic flow:
#### Connecting to a dashboard
1. Given a dashboard URL, ask an entrypoint provider "does this belong to you?"
2. If yes, pass the provider a tornado request object for a dashboard plot and allow them to do whatever to it (probably add auth headers).
3. Make the request as before.
#### Cluster connection
The `ClusterManager` already sends information about a cluster to the frontend to show in the side pane. So it (or whatever replaces it, cf. #189 ) could
1. Ask an entrypoint provider "does this belong to you?"
2. If so, get a snippet for connecting to the cluster to send to the frontend.
3. That snippet can be inserted in to notebooks/text-files as before.
### Dask Gateway
`dask-gateway` also has functionality around authenticating and proxying remote clusters. Is it possible to adopt it as a dependency to handle more than what we do here (see some discussion in #135) ? My instinct is that we don't want to require dask-gateway clusters in all circumstances, and that it won't be able to cover all the use cases we want to be able to handle here (and by "handle" I mean, let plugin authors handle). The converse of having dask-gateway implement a plugin here, on the other hand (fixing #135), should be fairly straightforward. But I'm happy to get pushback on that if others disagree. Especially curious what @rabernat and @scottyhq think.
Contributor guide
Assessment
This issue has not been assessed yet.