dask / dask/distributed

Possibility to use IPythom.embed() on workers

Open
#1,505 5 comments 0 reactions 0 assignees View on GitHub
documentation good first issue
Dominant language
Python
Stars
1.7k
Forks
778
Avg merge
2h 50m
Merged PRs (30d)
3

Description

Follow-up on our discussion with @mrocklin: It would be nice to use `IPython.embed()` directly in a worker for debugging purposes (in a cluster running locally). Example usage:

```python
from distributed import Client

def logic_to_run_on_worker():
# ... imagine complex logic here ...
# local embed for debugging
import IPython
IPython.embed()

client = Client("tcp://10.128.4.209:8786")
client.run(logic_to_run_on_worker)
```

This currently fails with

```
Traceback (most recent call last):
File "dask_test.py", line 9, in
client.run(logic_to_run_on_worker)
File "/home/PHI-TPS/fkeller/.virtualenvs/dask/local/lib/python2.7/site-packages/distributed/client.py", line 1819, in run
return self.sync(self._run, function, *args, **kwargs)
File "/home/PHI-TPS/fkeller/.virtualenvs/dask/local/lib/python2.7/site-packages/distributed/client.py", line 559, in sync
return sync(self.loop, func, *args, **kwargs)
File "/home/PHI-TPS/fkeller/.virtualenvs/dask/local/lib/python2.7/site-packages/distributed/utils.py", line 248, in sync
six.reraise(*error[0])
File "/home/PHI-TPS/fkeller/.virtualenvs/dask/local/lib/python2.7/site-packages/distributed/utils.py", line 236, in f
result[0] = yield make_coro()
File "/home/PHI-TPS/fkeller/.virtualenvs/dask/local/lib/python2.7/site-packages/tornado/gen.py", line 1055, in run
value = future.result()
File "/home/PHI-TPS/fkeller/.virtualenvs/dask/local/lib/python2.7/site-packages/tornado/concurrent.py", line 238, in result
raise_exc_info(self._exc_info)
File "/home/PHI-TPS/fkeller/.virtualenvs/dask/local/lib/python2.7/site-packages/tornado/gen.py", line 1069, in run
yielded = self.gen.send(value)
File "/home/PHI-TPS/fkeller/.virtualenvs/dask/local/lib/python2.7/site-packages/distributed/client.py", line 1773, in _run
six.reraise(*clean_exception(**resp))
File "dask_test.py", line 6, in logic_to_run_on_worker
IPython.embed()
File "/home/PHI-TPS/fkeller/.virtualenvs/dask/local/lib/python2.7/site-packages/IPython/terminal/embed.py", line 383, in embed
frame.f_code.co_filename, frame.f_lineno), **kwargs)
File "/home/PHI-TPS/fkeller/.virtualenvs/dask/local/lib/python2.7/site-packages/traitlets/config/configurable.py", line 412, in instance
inst = cls(*args, **kwargs)
File "/home/PHI-TPS/fkeller/.virtualenvs/dask/local/lib/python2.7/site-packages/IPython/terminal/embed.py", line 158, in __init__
super(InteractiveShellEmbed,self).__init__(**kw)
File "/home/PHI-TPS/fkeller/.virtualenvs/dask/local/lib/python2.7/site-packages/IPython/terminal/interactiveshell.py", line 439, in __init__
self.init_prompt_toolkit_cli()
File "/home/PHI-TPS/fkeller/.virtualenvs/dask/local/lib/python2.7/site-packages/IPython/terminal/interactiveshell.py", line 274, in init_prompt_toolkit_cli
output=create_output(true_color=self.true_color))
File "/home/PHI-TPS/fkeller/.virtualenvs/dask/local/lib/python2.7/site-packages/prompt_toolkit/interface.py", line 81, in __init__
self.input = input or StdinInput(sys.stdin)
File "/home/PHI-TPS/fkeller/.virtualenvs/dask/local/lib/python2.7/site-packages/prompt_toolkit/input.py", line 67, in __init__
assert self.stdin.isatty()
AssertionError
```

Maybe only a minor change is necessary to make this work and it could be optionally enabled via the Dask config.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.