EpistasisLab / EpistasisLab/tpot
TPOTClassifier training causes Jupyter kernel to die and segmentation fault when running from script.
- Dominant language
- Jupyter Notebook
- Stars
- 10.1k
- Forks
- 1.6k
- PR merge metrics
- No merged PRs in 30d
Description
Hi,
I'm running into several issues when fitting instances of `TPOTClassifier`.
When I run training in a Jupyter notebook, the kernel dies after a few rounds of training. The dataset is quite small (<1000 instances, <30 features). I also notice that the kernel dies more quickly when I increase the `population_size` and `generations` arguments beyond `20`. I'm setting `n_jobs=1` as I've read other people have this same issue when that parameter is anything but `1`. Here is the call:
```
tpot = TPOTClassifier(random_state=888, n_jobs=1,
generations=20, population_size=20,
verbosity=2, scoring='roc_auc')
tpot.fit(X_train, y_train)
```
I've also run the same code as a Python script. This results in a segmentation fault each time I run the script.
I've run the script while having `top` open in another bash shell and the memory consumption in the process does not exceed 1% of the overall available memory, so I don't think it's a memory issue.
I'm running this code in a kubernetes pod with the following resources:
```
Requests:
cpu: 1
memory: 1G
```
Here is the version of Python:
```
jovyan@jupyter-pod:~/work$ python --version
Python 3.6.6
```
Also, here is the result of calling `pip freeze`:
```
alembic==0.9.9
appdirs==1.4.3
asn1crypto==0.24.0async-generator==1.10
attrs==18.1.0
Automat==0.0.0
backcall==0.1.0
beautifulsoup4==4.6.1
bleach==2.1.3
bokeh==0.12.16
certifi==2018.4.16
cffi==1.11.5
chardet==3.0.4
cloudpickle==0.5.3
conda==4.5.8
constantly==15.1.0
cryptography==2.2.1
cycler==0.10.0
Cython==0.28.5
dask==0.18.2
deap==1.2.2
decorator==4.3.0
dill==0.2.8.2
entrypoints==0.2.3
fastcache==1.0.2
gmpy2==2.0.8
h5py==2.7.1
html5lib==1.0.1
hyperlink==17.3.1
idna==2.7
imageio==2.3.0
incremental==17.5.0
ipykernel==4.8.2
ipython==6.5.0
ipython-genutils==0.2.0
ipywidgets==7.2.1
jedi==0.12.1
Jinja2==2.10
jsonschema==2.6.0
jupyter-client==5.2.3
jupyter-core==4.4.0
jupyterhub==0.9.1
jupyterlab==0.33.7
jupyterlab-launcher==0.11.2
kiwisolver==1.0.1
llvmlite==0.23.0
Mako==1.0.7
MarkupSafe==1.0
matplotlib==2.2.2
mistune==0.8.3
nbconvert==5.3.1
nbformat==4.4.0
networkx==2.1
notebook==5.6.0
numba==0.38.1
numexpr==2.6.6
numpy==1.13.3
olefile==0.45.1
packaging==17.1
pamela==0.3.0
pandas==0.23.4
pandocfilters==1.4.2
parso==0.3.1
patsy==0.5.0
pexpect==4.6.0
pickleshare==0.7.4
Pillow==5.2.0
prometheus-client==0.3.0
prompt-toolkit==1.0.15
protobuf==3.5.2
ptyprocess==0.6.0
pyasn1==0.4.4
pyasn1-modules==0.2.1
pycosat==0.6.3
pycparser==2.18
pycurl==7.43.0.2
Pygments==2.2.0
pyOpenSSL==18.0.0
pyparsing==2.2.0
PySocks==1.6.8
python-dateutil==2.7.3
python-editor==1.0.3
python-oauth2==1.0.1
pytz==2018.5
PyWavelets==0.5.2
PyYAML==3.12
pyzmq==17.1.0
requests==2.19.1
rpy2==2.8.5
ruamel-yaml==0.15.44
scikit-image==0.14.0
scikit-learn==0.19.2
scipy==1.1.0
seaborn==0.9.0
Send2Trash==1.5.0
service-identity==17.0.0
simplegeneric==0.8.1
singledispatch==3.4.0.3
six==1.11.0
SQLAlchemy==1.2.10
statsmodels==0.9.0
stopit==1.1.2
sympy==1.1.1
terminado==0.8.1
testpath==0.3.1
toolz==0.9.0
tornado==5.1
TPOT==0.9.3
tqdm==4.25.0
traitlets==4.3.2
Twisted==18.7.0
update-checker==0.16
urllib3==1.23
vincent==0.4.4
wcwidth==0.1.7
webencodings==0.5
widgetsnbextension==3.2.1
xgboost==0.80
xlrd==1.1.0
zope.interface==4.5.0
```
If there are any other tests I can perform to help debug, let me know!
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.