dask / dask/distributed

Does dask distributed allow customized work.executor?

Open
#1,971 5 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.7k
Forks
778
Avg merge
2h 50m
Merged PRs (30d)
3

Description

Hello,

We are in a process of evaluating the possibilities of using dask and dask distributed in our Analytics Platform. However, there are some legacy problems that force us to customize work's executor. I found that WorkBase allows pass in executor, but Nanny doesn't. Any suggestion for how to pass in a customized executor?

`
class WorkerBase(ServerNode):

def __init__(self, scheduler_ip=None, scheduler_port=None,
scheduler_file=None, ncores=None, loop=None, local_dir=None,
services=None, service_ports=None, name=None,
reconnect=True, memory_limit='auto',
executor=None, resources=None, silence_logs=None,
death_timeout=None, preload=(), preload_argv=[], security=None,
contact_address=None, memory_monitor_interval='200ms', **kwargs):

self.executor = executor or ThreadPoolExecutor(self.ncores)
`

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.