dask / dask/distributed

Enable scheduler to output worker logs to stdout

Open
#6,264 3 comments 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
1.7k
Forks
778
Avg merge
2h 50m
Merged PRs (30d)
3

Description

It is reasonably common for the scheduler to run on slightly different infrastructure to the workers in complex Dask deployments. For example the scheduler may live on a head node in an HPC while the workers run on the HPC nodes themselves.

In some cases the location of the scheduler is a little more user friendly, can be interactive, perhaps has more accessible log collection, etc.

It would be really helpful in those cases for the scheduler to emit the logs generated by the worker to stdout alongside its own logs. The scheduler already has access to those logs via the RPC, so I don't think it would be much work to periodically retrieve them and print them to stdout. I don't think this behaviour exists currently.

Perhaps the scheduler could have a flag like `dask-scheduler --output-worker-logs` to enable this behaviour.

I wanted to gather a little feedback on this proposal before I dive into implementing it. Thoughts?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.