dask / dask/distributed

Re-think IO loop handling

Open
#1,118 1 comment 0 reactions 1 assignee Claimed by @pitrou View on GitHub
Dominant language
Python
Stars
1.7k
Forks
778
Avg merge
2h 50m
Merged PRs (30d)
3

Description

I'd like to propose two things:
* deprecate the explicit `loop` argument in most APIs (public and internal), generalize the use of `IOLoop.current()` in places where a loop is still required (e.g. for calling `loop.add_callback`)
* sanitize our loop creation logic; for example, don't get the current IOLoop for a thread and start it in another thread...

Two recent Tornado idioms may help us: `IOLoop.current(instance=False)` (returns None if there is no IO loop registered for the current thread) and `IOLoop(make_current=False)` (creates a loop but doesn't register it as the current thread's loop).

The feasibility of this is not 100% sure, but I hope we can simplify our overly-complex and brittle loop handling code.

**Edit:** @mrocklin explained why explicit `loop` arguments can be useful. For example, we might want to start a Scheduler and other services, all in the same loop, albeit running in a separate thread from the main thread.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.