Hang in subprocess.Popen()
- Dominant language
- Python
- Stars
- 1.4k
- Forks
- 60
- PR merge metrics
- No merged PRs in 30d
Description
**[UPDATE: If you have this problem, the solution is to pass `close_fds=True` to `Popen()`]**
I've got a use case where a process being traced for type collection uses `subprocess.Popen()` to execute some helper program, and the `Popen()` call hangs at the line
```py
data = _eintr_retry_call(os.read, errpipe_read, 1048576)
```
in `Popen._execute_child()`. (This is Python 2.7 on Mac, i.e. POSIX.)
That pipe has `FD_CLOEXEC`, so the child is not hitting the `exec()`. Presumably this is because it hangs in a `Queue.put()` operation in ` _trace_dispatch()` (e.g. [here](https://github.com/dropbox/pyannotate/blob/master/pyannotate_runtime/collect_types.py#L853)).
I can think of a gross fix that monkey-patches `os.fork` to disable the profiling hook around the `fork()` so the child doesn't do this. But perhaps there's a more elegant solution (without using `os.register_at_fork()`, which is Python 3.7+ only)? Or the tracing hook could check the pid?
**[UPDATE:]** I can't repro this in a small test program. But it's real, and the `os.fork` monkey-patch fixes it. Not sure what to do about it yet, the monkey-patch seems risky.
Contributor guide
Assessment
This issue has not been assessed yet.