jupyter notebook slows down after multiple executions and needs a kernel restart

Unfortunately, this error is not so easy to clearly reproduce and I'm not sure if it's a jupyter issue or an iPython kernel issue or maybe even due to the other libraries I'm using... But since it can be "fixed" by a kernel restart, I'm assuming it should be due to jupyter or the iPython kernel.

I'm training neural networks using the keras library on a Nvidia GPU on a local desktop computer running Linux Peppermint 8 (a derivative of ubuntu 17.4). I have jupyter notebook version 5.0.0 for python 2.7 installed via anaconda.

My problem is that at the beginning of executing the notebook, everything runs really fast and the neural networks train with less than 100ms per training epoch. After executing some cells multiple times, e.g. to test different parameters, after some time the training starts to slow down, taking at least twice as long and it gets worse the more often I run the cells. Then if I restart the kernel, when I execute all the cells again, everything is fast again.

I'm not really sure how this can happen. I assume it has something to do with the garbage collection (or whatever is being performed at a restart), but as I was saying I'm mostly reexecuting the same cells, not adding more, so it should overwrite the existing variables and not add stuff to the stack.

No repository file or test is identified. Start by reproducing the slowdown with Jupyter Notebook 5.0.0, Python 2.7, Keras, and repeated neural-network training cells, then compare behavior before and after a kernel restart to isolate Jupyter, the IPython kernel, or another library. Done means a reproducible cause and a targeted fix or regression test.

Written by the indexing model from the issue text.