Creation and shutdown of InProcessKernel object leaks file descriptors.
- Dominant language
- Python
- Stars
- 734
- Forks
- 412
- Avg merge
- 1d 5h
- Merged PRs (30d)
- 8
Description
This issue is part bug report, part question.
The bug report: running the script below gives me a "Too many open files" exception on OS X 10.9. (Python 2.7.10, ipykernel 4.5.2, traitlets 4.3.1, zmq 16.0.0.) Using `lsof`, it looks as though every cycle of shell creation and destruction leaks two open unix sockets, and these sockets are (indirectly) kept alive until the end of the process via an `atexit` handler.
The question: what's the recommended way of cleaning up a (kernel_manager, kernel) pair created as below? I'm looking for something that releases all relevant system resources (threads, file handles, etc). Is there something more I can do or should be doing beyond the `shutdown_kernel` call?
Background: we have a large application that makes use of an in-process kernel. Several of the tests in our test suite create and destroy an IPython kernel. That leads to us running out of file handles later in the test suite; this is especially a problem on OS X, where the default number of file handles per process is 256. At the moment, we're artificially increasing that limit to get our test suite to pass (using `ulimit -n 1024`), but that doesn't seem like the right solution. :-)
```
import gc
from ipykernel.inprocess.manager import InProcessKernelManager
def create_and_destroy_shell():
kernel_manager = InProcessKernelManager()
kernel_manager.start_kernel()
kernel_manager.shutdown_kernel()
gc.collect()
# Raises "Too many open files" OSError on OS X, with the default
# process limits (<= 256 open files).
for _ in xrange(200):
create_and_destroy_shell()
```
Contributor guide
Assessment
This issue has not been assessed yet.