async subprocess with live output (on Windows requires ProactorEventLoop which ipykernel doesn't use)
- 主要言語
- Python
- スター
- 734
- フォーク
- 412
- 平均マージ
- 1日 5時間
- マージ済み PR(30日)
- 8
説明
I'd like to be able to run subprocesses in the notebook and trace their output using asyncio. I thought that since the ipykernel is now using asyncio that it should just be a fairly simple await call.
https://blog.jupyter.org/ipython-7-0-async-repl-a35ce050f7f7
However, when I try to run the following example:
from https://fredrikaverpil.github.io/2017/06/20/async-and-await-with-subprocesses/
```python
async def run_command(*args):
"""Run command in subprocess
Example from:
http://asyncio.readthedocs.io/en/latest/subprocess.html
"""
# Create subprocess
process = await asyncio.create_subprocess_exec(
*args,
# stdout must a pipe to be accessible as process.stdout
stdout=asyncio.subprocess.PIPE)
# Status
print('Started:', args, '(pid = ' + str(process.pid) + ')')
# Wait for the subprocess to finish
stdout, stderr = await process.communicate()
# Progress
if process.returncode == 0:
print('Done:', args, '(pid = ' + str(process.pid) + ')')
else:
print('Failed:', args, '(pid = ' + str(process.pid) + ')')
# Result
result = stdout.decode().strip()
# Return stdout
return result
```
I get the following:
```
---------------------------------------------------------------------------
NotImplementedError Traceback (most recent call last)
cell_name in async-def-wrapper()
in run_command(*args, **kwargs)
6 """
7 # Create subprocess
----> 8 process = await asyncio.create_subprocess_exec(*args)
9
10 # Status
c:\python37\lib\asyncio\subprocess.py in create_subprocess_exec(program, stdin, stdout, stderr, loop, limit, *args, **kwds)
215 program, *args,
216 stdin=stdin, stdout=stdout,
--> 217 stderr=stderr, **kwds)
218 return Process(transport, protocol, loop)
c:\python37\lib\asyncio\base_events.py in subprocess_exec(self, protocol_factory, program, stdin, stdout, stderr, universal_newlines, shell, bufsize, *args, **kwargs)
1531 transport = await self._make_subprocess_transport(
1532 protocol, popen_args, False, stdin, stdout, stderr,
-> 1533 bufsize, **kwargs)
1534 if self._debug and debug_log is not None:
1535 logger.info('%s: %r', debug_log, transport)
c:\python37\lib\asyncio\base_events.py in _make_subprocess_transport(self, protocol, args, shell, stdin, stdout, stderr, bufsize, extra, **kwargs)
461 extra=None, **kwargs):
462 """Create subprocess transport."""
--> 463 raise NotImplementedError
464
465 def _write_to_self(self):
NotImplementedError:
```
My initial research shows that on Windows I have to use the Proactor event loop to do subprocesses? So does this mean that ipykernel simple doesn't use the event loop that I would need to do this? Or am I missing something?
コントリビューションガイド
調査の方向性
Start by reproducing the Windows failure from the issue with ipykernel's asyncio integration and asyncio.create_subprocess_exec. Trace which event loop ipykernel uses and compare it with the loop required for subprocess support; done means asynchronous subprocess execution with live output works in the reported notebook scenario.
索引モデルが issue の本文から書いたものです。
評価
- 技術スタック
- python
- 領域
- backend, operating-systems
- issue の種類
- バグ
- 難易度
- 4/5
- 見積もり時間
- 3〜5日
- 活発さ
- 停滞
- 明瞭さ
- おおむね明確
- 初心者へのやさしさ
- 35/100