ipython / ipython/ipykernel

Creation and shutdown of InProcessKernel object leaks file descriptors.

Open
#219 8 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
734
Forks
412
Avg merge
1d 5h
Merged PRs (30d)
8

Description

This issue is part bug report, part question.

The bug report: running the script below gives me a "Too many open files" exception on OS X 10.9. (Python 2.7.10, ipykernel 4.5.2, traitlets 4.3.1, zmq 16.0.0.) Using `lsof`, it looks as though every cycle of shell creation and destruction leaks two open unix sockets, and these sockets are (indirectly) kept alive until the end of the process via an `atexit` handler.

The question: what's the recommended way of cleaning up a (kernel_manager, kernel) pair created as below? I'm looking for something that releases all relevant system resources (threads, file handles, etc). Is there something more I can do or should be doing beyond the `shutdown_kernel` call?

Background: we have a large application that makes use of an in-process kernel. Several of the tests in our test suite create and destroy an IPython kernel. That leads to us running out of file handles later in the test suite; this is especially a problem on OS X, where the default number of file handles per process is 256. At the moment, we're artificially increasing that limit to get our test suite to pass (using `ulimit -n 1024`), but that doesn't seem like the right solution. :-)

```
import gc
from ipykernel.inprocess.manager import InProcessKernelManager

def create_and_destroy_shell():
kernel_manager = InProcessKernelManager()
kernel_manager.start_kernel()
kernel_manager.shutdown_kernel()
gc.collect()

# Raises "Too many open files" OSError on OS X, with the default
# process limits (<= 256 open files).
for _ in xrange(200):
create_and_destroy_shell()
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.