GenericMappingTools / GenericMappingTools/pygmt
Support multiprocessing without importing gmt in each process
- Dominant language
- Python
- Stars
- 874
- Forks
- 255
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 40
Description
## Feedback or description of feature requested
When attempting to call a script using gmt-python by way of the python multiprocessing library (multiprocessing.pool), an error is returned, as shown below.
**Are you willing to help implement and maintain this feature?** No
## Full code that generated the error
```python
import gmt
import multiprocessing as mp
def gmt_fun(n): #a very simple function that uses gmt-python
fig = gmt.Figure()
fig.coast(R="-20/20/-20/20", J="M10i", G='gray', S="lightblue", B=True)
fig.savefig("{}.pdf".format(n), show=False)
#gmt_fun(5) #calls the function without multiprocessing - uncomment as desired to see that the function works on its own
with mp.Pool(2) as p: #calls the process with multiprocessing, using 2 cores
a = p.map(gmt_fun,[x for x in range(0,2)])
```
The code does work with the following change:
```python
import multiprocessing as mp
def gmt_fun(n): #a very simple function that uses gmt-python
import gmt
fig = gmt.Figure()
fig.coast(R="-20/20/-20/20", J="M10i", G='gray', S="lightblue", B=True)
fig.savefig("{}.pdf".format(n), show=False)
#gmt_fun(5) #calls the function without multiprocessing - uncomment as desired to see that the function works on its own
with mp.Pool(2) as p: #calls the process with multiprocessing, using 2 cores
a = p.map(gmt_fun,[x for x in range(0,2)])
```
## Full error message
```
gmt-python-session [ERROR]: Not available in classic mode
gmt-python-session [ERROR]: Not available in classic mode
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
File "/Users/norfordb/anaconda2/envs/gmt-python/lib/python3.6/multiprocessing/pool.py", line 119, in worker
result = (True, func(*args, **kwds))
File "/Users/norfordb/anaconda2/envs/gmt-python/lib/python3.6/multiprocessing/pool.py", line 44, in mapstar
return list(map(*args))
File "tweaker.py", line 6, in gmt_fun
fig = gmt.Figure()
File "/Users/norfordb/anaconda2/envs/gmt-python/lib/python3.6/site-packages/gmt/figure.py", line 64, in __init__
self._activate_figure()
File "/Users/norfordb/anaconda2/envs/gmt-python/lib/python3.6/site-packages/gmt/figure.py", line 85, in _activate_figure
lib.call_module("figure", "{} {}".format(self._name, fmt))
File "/Users/norfordb/anaconda2/envs/gmt-python/lib/python3.6/site-packages/gmt/clib/session.py", line 481, in call_module
module, status, self._error_message
gmt.exceptions.GMTCLibError: Module 'figure' failed with status code 30:
gmt-python-session [ERROR]: Not available in classic mode
"""
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "tweaker.py", line 13, in
a = p.map(gmt_fun,[x for x in range(0,2)])
File "/Users/norfordb/anaconda2/envs/gmt-python/lib/python3.6/multiprocessing/pool.py", line 266, in map
return self._map_async(func, iterable, mapstar, chunksize).get()
File "/Users/norfordb/anaconda2/envs/gmt-python/lib/python3.6/multiprocessing/pool.py", line 644, in get
raise self._value
gmt.exceptions.GMTCLibError: Module 'figure' failed with status code 30:
gmt-python-session [ERROR]: Not available in classic mode
```
## System information
* Operating system: OSX
* Python installation (Anaconda, system, ETS): Anaconda
* Version of GMT: 6.0.0a16
* Version of Python: 3.6.5
* Version of this package: 0.1a3+112.gefabb47
* If using conda, paste the output of `conda list` below:
```
# Output of conda list
appnope 0.1.0 py36_0 conda-forge
atomicwrites 1.1.5 py36_0 conda-forge
attrs 17.4.0 py_0 conda-forge
backcall 0.1.0 py_0 conda-forge
blas 1.1 openblas conda-forge
bokeh 0.13.0 py36_0 conda-forge
boost 1.66.0 py36_1 conda-forge
boost-cpp 1.66.0 1 conda-forge
bottleneck 1.2.1 py36h7eb728f_1 conda-forge
bzip2 1.0.6 1 conda-forge
ca-certificates 2018.1.18 0 conda-forge
cairo 1.14.10 0 conda-forge
certifi 2018.1.18 py36_0 conda-forge
cftime 1.0.0 py36_0 conda-forge
click 6.7 py_1 conda-forge
cloudpickle 0.5.3 py_0 conda-forge
curl 7.59.0 0 conda-forge
cycler 0.10.0 py36_0 conda-forge
Cython 0.28.4
cytoolz 0.9.0.1 py36h470a237_0 conda-forge
dask 0.18.2 py_0 conda-forge
dask-core 0.18.2 py_0 conda-forge
dcw-gmt 1.1.2 0 conda-forge
decorator 4.2.1 py36_0 conda-forge
distributed 1.22.0 py36_0 conda-forge
expat 2.2.5 0 conda-forge
fastkml 0.11
fftw 3.3.7 0 conda-forge
fontconfig 2.12.6 0 conda-forge
freetype 2.8.1 0 conda-forge
freexl 1.0.5 0 conda-forge
gdal 2.2.4 py36_0 conda-forge
geos 3.6.2 1 conda-forge
geotiff 1.4.2 1 conda-forge
gettext 0.19.8.1 0 conda-forge
ghostscript 9.22 0 conda-forge
giflib 5.1.4 0 conda-forge
glib 2.55.0 0 conda-forge
gmt 6.0.0a16 blas_openblas_200 [blas_openblas] conda-forge/label/dev
gmt-python 0.1a3+112.gefabb47
gshhg-gmt 2.3.7 0 conda-forge
h5netcdf 0.6.1 py_0 conda-forge
h5py 2.8.0 py36h470a237_0 conda-forge
hdf4 4.2.13 0 conda-forge
hdf5 1.10.1 2 conda-forge
heapdict 1.0.0 py36_0 conda-forge
icu 58.2 0 conda-forge
ipython 6.4.0 py36_0 conda-forge
ipython_genutils 0.2.0 py36_0 conda-forge
jedi 0.11.1 py36_0 conda-forge
jinja2 2.10 py_1 conda-forge
jpeg 9b 2 conda-forge
json-c 0.12.1 0 conda-forge
kealib 1.4.7 4 conda-forge
kiwisolver 1.0.1 py36_1 conda-forge
krb5 1.14.6 0 conda-forge
libdap4 3.18.3 2 conda-forge
libffi 3.2.1 3 conda-forge
libgdal 2.2.4 5 conda-forge
libgfortran 3.0.0 0 conda-forge
libiconv 1.15 0 conda-forge
libkml 1.3.0 6 conda-forge
libnetcdf 4.6.1 2 conda-forge
libpng 1.6.34 0 conda-forge
libpq 9.6.3 0 conda-forge
libspatialite 4.3.0a 19 conda-forge
libssh2 1.8.0 2 conda-forge
libtiff 4.0.9 0 conda-forge
libxml2 2.9.8 0 conda-forge
locket 0.2.0 py_2 conda-forge
markupsafe 1.0 py36_0 conda-forge
matplotlib 2.2.2 py36_1 conda-forge
more-itertools 4.1.0 py_0 conda-forge
msgpack-python 0.5.6 py36h2d50403_2 conda-forge
ncurses 5.9 10 conda-forge
netcdf4 1.4.0 py36_0 conda-forge
nose 1.3.7 py36_2 conda-forge
numpy 1.14.5
numpy 1.14.5 py36_blas_openblashd3ea46f_201 [blas_openblas] conda-forge
olefile 0.45.1 py36_0 conda-forge
openblas 0.2.20 7 conda-forge
openjpeg 2.3.0 2 conda-forge
openssl 1.0.2n 0 conda-forge
packaging 17.1 py_0 conda-forge
pandas 0.23.2
pandas 0.23.3 py36_0 conda-forge
parso 0.1.1 py_0 conda-forge
partd 0.3.8 py_1 conda-forge
pcre 8.41 1 conda-forge
pexpect 4.4.0 py36_0 conda-forge
pickleshare 0.7.4 py36_0 conda-forge
pillow 5.1.0 py36_0 conda-forge
pip 10.0.1
pip 9.0.3 py36_0 conda-forge
pixman 0.34.0 1 conda-forge
pluggy 0.6.0 py_0 conda-forge
poppler 0.61.1 3 conda-forge
poppler-data 0.4.8 0 conda-forge
proj4 4.9.3 5 conda-forge
prompt_toolkit 1.0.15 py36_0 conda-forge
psutil 5.4.6 py36_0 conda-forge
ptyprocess 0.5.2 py36_0 conda-forge
py 1.5.3 py_0 conda-forge
pygeoif 0.7
pygments 2.2.0 py36_0 conda-forge
pyparsing 2.2.0 py36_0 conda-forge
pytest 3.6.2 py36_0 conda-forge
pytest-mpl 0.9 py_0 conda-forge
python 3.6.5 0 conda-forge
python-dateutil 2.7.2 py_0 conda-forge
python-dateutil 2.7.3
pytz 2018.3 py_0 conda-forge
pytz 2018.5
pyyaml 3.12 py36_1 conda-forge
readline 7.0 0 conda-forge
scipy 1.0.1 py36_blas_openblas_200 [blas_openblas] conda-forge
setuptools 39.0.1 py36_0 conda-forge
simplegeneric 0.8.1 py36_0 conda-forge
six 1.11.0 py36_1 conda-forge
sortedcontainers 2.0.4 py_1 conda-forge
sqlite 3.20.1 2 conda-forge
tblib 1.3.2 py_1 conda-forge
tk 8.6.7 0 conda-forge
toolz 0.9.0 py_0 conda-forge
tornado 5.0.1 py36_1 conda-forge
traitlets 4.3.2 py36_0 conda-forge
wcwidth 0.1.7 py36_0 conda-forge
wheel 0.31.0 py36_0 conda-forge
xarray 0.10.8 py36_0 conda-forge
xarray 0.10.7
xerces-c 3.2.0 0 conda-forge
xz 5.2.3 0 conda-forge
yaml 0.1.7 0 conda-forge
zict 0.1.3 py_0 conda-forge
zlib 1.2.11 0 conda-forge
```
Contributor guide
Assessment
This issue has not been assessed yet.