dask / dask/dask-cloudprovider
Cloud logs observability improvements
- Dominant language
- Python
- Stars
- 147
- Forks
- 119
- PR merge metrics
- No merged PRs in 30d
Description
`dask-cloudprovider` integration might be tough unless the cloud is configured correctly. Often, the developer might see that the initial start of the cluster simply hung, and Dask waits infinitely for the scheduler to be started:
```
Creating scheduler instance
dask-5424b11a-scheduler
Internal IP: 10.128.0.220
External IP: 35.184.140.159
Waiting for scheduler to run at 35.184.140.159:8786
```
Usually, the problem is observed in cloud-init logs or docker container std output that can fetched if you connect to the instance directly via SSH and cat cloud-init logs:
```
cat /var/log/cloud-init-output.log
```
Or fetches the Docker container stdout:
```
docker logs
```
I propose to output these logs into a `distributed` logger.
Contributor guide
Assessment
This issue has not been assessed yet.