sequential_distribute_dump fails

sequential_distribute_dump fails for large domains in cPickle with

File "/usr/local/lib/python2.7/dist-packages/anuga/parallel/sequential_distribute.py", line 246, in sequential_distribute_dump
    cPickle.dump( tostore, f, protocol=cPickle.HIGHEST_PROTOCOL)
SystemError: error return without exception set

Probably related:
https://github.com/numpy/numpy/issues/2396
https://bugs.python.org/issue11564
https://stackoverflow.com/questions/34091717/avoid-cpickle-error-alternatives-to-cpickle-on-python-2-7

As using python3 isn't easily done it seems, i am unable to save large domains. (roughly >100,000,000 tris) - which is rather annoying when trying multiple variants on same large mesh.

My only advice to other users at this moment would be to not use sequential_distribute_dump to save the combined domain and the parts, but only the partial (per-proc) domains. these might just be small enough to work.

As to how to circumvent the error without upgrading anuga to python3, using numpy.save()/numpy.savez() is the most often suggested solution. (not sure if anuga uses numpy arrays for fields internally)

Start with anuga/parallel/sequential_distribute.py around line 246 and reproduce the cPickle failure using a large domain. Review the linked Python and NumPy reports for relevant constraints. Done means sequential_distribute_dump can save domains larger than roughly 100,000,000 triangles without this error.