ipython / ipython/ipykernel

async subprocess with live output (on Windows requires ProactorEventLoop which ipykernel doesn't use)

オープン
#391 コメント 2 件 リアクション 1 件 担当者 0 名 GitHub で見る
主要言語
Python
スター
734
フォーク
412
平均マージ
1日 5時間
マージ済み PR(30日)
8

説明

I'd like to be able to run subprocesses in the notebook and trace their output using asyncio. I thought that since the ipykernel is now using asyncio that it should just be a fairly simple await call.

https://blog.jupyter.org/ipython-7-0-async-repl-a35ce050f7f7

However, when I try to run the following example:

from https://fredrikaverpil.github.io/2017/06/20/async-and-await-with-subprocesses/
```python
async def run_command(*args):
"""Run command in subprocess

Example from:
http://asyncio.readthedocs.io/en/latest/subprocess.html
"""
# Create subprocess
process = await asyncio.create_subprocess_exec(
*args,
# stdout must a pipe to be accessible as process.stdout
stdout=asyncio.subprocess.PIPE)

# Status
print('Started:', args, '(pid = ' + str(process.pid) + ')')

# Wait for the subprocess to finish
stdout, stderr = await process.communicate()

# Progress
if process.returncode == 0:
print('Done:', args, '(pid = ' + str(process.pid) + ')')
else:
print('Failed:', args, '(pid = ' + str(process.pid) + ')')

# Result
result = stdout.decode().strip()

# Return stdout
return result
```

I get the following:

```
---------------------------------------------------------------------------
NotImplementedError Traceback (most recent call last)
cell_name in async-def-wrapper()

in run_command(*args, **kwargs)
6 """
7 # Create subprocess
----> 8 process = await asyncio.create_subprocess_exec(*args)
9
10 # Status

c:\python37\lib\asyncio\subprocess.py in create_subprocess_exec(program, stdin, stdout, stderr, loop, limit, *args, **kwds)
215 program, *args,
216 stdin=stdin, stdout=stdout,
--> 217 stderr=stderr, **kwds)
218 return Process(transport, protocol, loop)

c:\python37\lib\asyncio\base_events.py in subprocess_exec(self, protocol_factory, program, stdin, stdout, stderr, universal_newlines, shell, bufsize, *args, **kwargs)
1531 transport = await self._make_subprocess_transport(
1532 protocol, popen_args, False, stdin, stdout, stderr,
-> 1533 bufsize, **kwargs)
1534 if self._debug and debug_log is not None:
1535 logger.info('%s: %r', debug_log, transport)

c:\python37\lib\asyncio\base_events.py in _make_subprocess_transport(self, protocol, args, shell, stdin, stdout, stderr, bufsize, extra, **kwargs)
461 extra=None, **kwargs):
462 """Create subprocess transport."""
--> 463 raise NotImplementedError
464
465 def _write_to_self(self):

NotImplementedError:
```

My initial research shows that on Windows I have to use the Proactor event loop to do subprocesses? So does this mean that ipykernel simple doesn't use the event loop that I would need to do this? Or am I missing something?

コントリビューションガイド

コントリビューションガイドを開く

調査の方向性

Start by reproducing the Windows failure from the issue with ipykernel's asyncio integration and asyncio.create_subprocess_exec. Trace which event loop ipykernel uses and compare it with the loop required for subprocess support; done means asynchronous subprocess execution with live output works in the reported notebook scenario.

索引モデルが issue の本文から書いたものです。

評価

技術スタック
python
領域
backend, operating-systems
issue の種類
バグ
難易度
4/5
見積もり時間
3〜5日
活発さ
停滞
明瞭さ
おおむね明確
初心者へのやさしさ
35/100

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。