MPI parallelisation fails with DLPOLY engine

**Description of the error**
The script examples/argon-dlp.py works correctly when executed by `python argon-dlp.py`
However, the same script fails when run via `mpirun`, independent of the number of CPUs used.

NOTE: I am running the examples in the standard Docker container created by the MDMC workflow.
My Docker settings give the container access to up to 5 CPU cores.

**Describe the expected result**
I would expect the script to produce the same result using MPI as it does running normally.

**Describe the actual result**
Segmentation fault, in every test case. Details below.

**Suggested fix**
The problem could be caused by MDMC, dlpoly-py, or some interplay between the two.

**Additional details**

I use examples/argon-dlp.py script from MDMC for testing.

**Test 1**
Input:
`python argon-dlp.py`

Result:
Completes the optimisation, runs 10 steps and exits.

**Test 2**
`mpirun -np 1 --allow-run-as-root python -m mpi4py argon-dlp.py`

Result
```
root@89993f60ee3f:/workspaces/MDMCv0.2_pilot/examples# mpirun -np 1 --allow-run-as-root python3 -m mpi4py argon-dlp.py
Supported DL_POLY version 4.10
Universe created with:
Dimensions [30.76, 30.76, 30.76]
Force field None
Number of atoms 0

--------------------------------------------------------------------------
It looks like orte_init failed for some reason; your parallel process is
likely to abort. There are many reasons that a parallel process can
fail during orte_init; some of which are due to configuration or
environment problems. This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
Open MPI developer):

getting local rank failed
--> Returned value No permission (-17) instead of ORTE_SUCCESS
--------------------------------------------------------------------------
--------------------------------------------------------------------------
It looks like orte_init failed for some reason; your parallel process is
likely to abort. There are many reasons that a parallel process can
fail during orte_init; some of which are due to configuration or
environment problems. This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
Open MPI developer):

orte_ess_init failed
--> Returned value No permission (-17) instead of ORTE_SUCCESS
--------------------------------------------------------------------------
--------------------------------------------------------------------------
It looks like MPI_INIT failed for some reason; your parallel process is
likely to abort. There are many reasons that a parallel process can
fail during MPI_INIT; some of which are due to configuration or environment
problems. This failure appears to be an internal failure; here's some
additional information (which may only be relevant to an Open MPI
developer):

ompi_mpi_init: ompi_rte_init failed
--> Returned "No permission" (-17) instead of "Success" (0)
--------------------------------------------------------------------------
DLPOLY.Z is not DLPoly, run may not work
Simulation created with dlpoly engine and settings:
temperature 120.0
numprocs 4.0

[89993f60ee3f:06740] OPAL ERROR: Unreachable in file ext2x_client.c at line 109
*** An error occurred in MPI_Init
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
*** and potentially your MPI job)
[89993f60ee3f:06740] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!
update coordinates from /workspaces/MDMCv0.2_pilot/examples/minim/REVCON
Unable to find file: /workspaces/MDMCv0.2_pilot/examples/minim/REVCON
[89993f60ee3f:06744] OPAL ERROR: Unreachable in file ext2x_client.c at line 109
*** An error occurred in MPI_Init
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
*** and potentially your MPI job)
[89993f60ee3f:06744] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!
update coordinates from /workspaces/MDMCv0.2_pilot/examples/equil/REVCON
Unable to find file: /workspaces/MDMCv0.2_pilot/examples/equil/REVCON
Control created with:
Minimizer MMC
FoM type ChiSquaredExpError
Number of observables 1
Number of parameters 2

Step FoM Change state epsilon (#2) sigma (#3)
[89993f60ee3f:06748] OPAL ERROR: Unreachable in file ext2x_client.c at line 109
*** An error occurred in MPI_Init
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
*** and potentially your MPI job)
[89993f60ee3f:06748] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!
update coordinates from /workspaces/MDMCv0.2_pilot/examples/dlprun/REVCON
Unable to find file: /workspaces/MDMCv0.2_pilot/examples/dlprun/REVCON
Traceback (most recent call last):
File "/usr/local/lib/python3.9/runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/local/lib/python3.9/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/usr/local/lib/python3.9/site-packages/mpi4py/__main__.py", line 7, in
main()
File "/usr/local/lib/python3.9/site-packages/mpi4py/run.py", line 198, in main
run_command_line(args)
File "/usr/local/lib/python3.9/site-packages/mpi4py/run.py", line 47, in run_command_line
run_path(sys.argv[0], run_name='__main__')
File "/usr/local/lib/python3.9/runpy.py", line 268, in run_path
return _run_module_code(code, init_globals, run_name,
File "/usr/local/lib/python3.9/runpy.py", line 97, in _run_module_code
_run_code(code, mod_globals, init_globals,
File "/usr/local/lib/python3.9/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "argon-dlp.py", line 74, in
control.refine(n_steps=10)
File "/usr/local/lib/python3.9/site-packages/MDMC/control/control.py", line 365, in refine
self.step() # advance the refinement by one step
File "/usr/local/lib/python3.9/site-packages/MDMC/control/control.py", line 422, in step
fom = self._generate_FoM()
File "/usr/local/lib/python3.9/site-packages/MDMC/control/control.py", line 478, in _generate_FoM
self._calculate_observables(self.simulation, self.observable_pairs)
File "/usr/local/lib/python3.9/site-packages/MDMC/control/control.py", line 574, in _calculate_observables
trj = simulation.engine.convert_trajectory()
File "/usr/local/lib/python3.9/site-packages/MDMC/MD/engine_facades/dlpoly_engine.py", line 426, in convert_trajectory
with open(self.dlpoly.control['io_file_history'], "r", encoding="ascii") as f:
FileNotFoundError: [Errno 2] No such file or directory: '/workspaces/MDMCv0.2_pilot/examples/dlprun/HISTORY'
--------------------------------------------------------------------------
MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD
with errorcode 1.

NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.
You may or may not see output from other processes, depending on
exactly when Open MPI kills them.
--------------------------------------------------------------------------
[89993f60ee3f:06699] *** Process received signal ***
[89993f60ee3f:06699] Signal: Segmentation fault (11)
[89993f60ee3f:06699] Signal code: Address not mapped (1)
[89993f60ee3f:06699] Failing at address: 0x30
[89993f60ee3f:06699] [ 0] linux-vdso.so.1(__kernel_rt_sigreturn+0x0)[0xffff93a367c0]
[89993f60ee3f:06699] [ 1] /usr/lib/aarch64-linux-gnu/libpmix.so.2(pmix_ptl_base_lost_connection+0x748)[0xffff90f83888]
[89993f60ee3f:06699] [ 2] /usr/lib/aarch64-linux-gnu/libpmix.so.2(pmix_ptl_base_recv_handler+0x1e4)[0xffff90f86074]
[89993f60ee3f:06699] [ 3] /usr/lib/aarch64-linux-gnu/libevent-2.1.so.6(+0x236ac)[0xffff936e76ac]
[89993f60ee3f:06699] *** End of error message ***
Segmentation fault
```

**Test 3**
Input:
```
export OMPI_MCA_btl_vader_single_copy_mechanism=none
export PMIX_MCA_gds=hash
mpirun -np 1 --allow-run-as-root python3 -m mpi4py argon-dlp.py
```

Result:
```
Supported DL_POLY version 4.10
Universe created with:
Dimensions [30.76, 30.76, 30.76]
Force field None
Number of atoms 0

Simulation created with dlpoly engine and settings:
temperature 120.0
numprocs 4.0

Folder minim exists, over-writing.
[89993f60ee3f:07206] OPAL ERROR: Unreachable in file ext2x_client.c at line 109
*** An error occurred in MPI_Init
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
*** and potentially your MPI job)
[89993f60ee3f:07206] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!
update coordinates from /workspaces/MDMCv0.2_pilot/examples/minim/REVCON
Unable to find file: /workspaces/MDMCv0.2_pilot/examples/minim/REVCON
Folder equil exists, over-writing.
[89993f60ee3f:07210] OPAL ERROR: Unreachable in file ext2x_client.c at line 109
*** An error occurred in MPI_Init
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
*** and potentially your MPI job)
[89993f60ee3f:07210] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!
update coordinates from /workspaces/MDMCv0.2_pilot/examples/equil/REVCON
Unable to find file: /workspaces/MDMCv0.2_pilot/examples/equil/REVCON
Control created with:
Minimizer MMC
FoM type ChiSquaredExpError
Number of observables 1
Number of parameters 2

Step FoM Change state epsilon (#2) sigma (#3)
[89993f60ee3f:07214] OPAL ERROR: Unreachable in file ext2x_client.c at line 109
*** An error occurred in MPI_Init
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
*** and potentially your MPI job)
[89993f60ee3f:07214] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!
update coordinates from /workspaces/MDMCv0.2_pilot/examples/dlprun2/REVCON
Unable to find file: /workspaces/MDMCv0.2_pilot/examples/dlprun2/REVCON
Traceback (most recent call last):
File "/usr/local/lib/python3.9/runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/local/lib/python3.9/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/usr/local/lib/python3.9/site-packages/mpi4py/__main__.py", line 7, in
main()
File "/usr/local/lib/python3.9/site-packages/mpi4py/run.py", line 198, in main
run_command_line(args)
File "/usr/local/lib/python3.9/site-packages/mpi4py/run.py", line 47, in run_command_line
run_path(sys.argv[0], run_name='__main__')
File "/usr/local/lib/python3.9/runpy.py", line 268, in run_path
return _run_module_code(code, init_globals, run_name,
File "/usr/local/lib/python3.9/runpy.py", line 97, in _run_module_code
_run_code(code, mod_globals, init_globals,
File "/usr/local/lib/python3.9/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "argon-dlp.py", line 74, in
control.refine(n_steps=10)
File "/usr/local/lib/python3.9/site-packages/MDMC/control/control.py", line 365, in refine
self.step() # advance the refinement by one step
File "/usr/local/lib/python3.9/site-packages/MDMC/control/control.py", line 422, in step
fom = self._generate_FoM()
File "/usr/local/lib/python3.9/site-packages/MDMC/control/control.py", line 478, in _generate_FoM
self._calculate_observables(self.simulation, self.observable_pairs)
File "/usr/local/lib/python3.9/site-packages/MDMC/control/control.py", line 574, in _calculate_observables
trj = simulation.engine.convert_trajectory()
File "/usr/local/lib/python3.9/site-packages/MDMC/MD/engine_facades/dlpoly_engine.py", line 426, in convert_trajectory
with open(self.dlpoly.control['io_file_history'], "r", encoding="ascii") as f:
FileNotFoundError: [Errno 2] No such file or directory: '/workspaces/MDMCv0.2_pilot/examples/dlprun2/HISTORY'
--------------------------------------------------------------------------
MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD
with errorcode 1.

NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.
You may or may not see output from other processes, depending on
exactly when Open MPI kills them.
--------------------------------------------------------------------------
[89993f60ee3f:07180] *** Process received signal ***
[89993f60ee3f:07180] Signal: Segmentation fault (11)
[89993f60ee3f:07180] Signal code: Address not mapped (1)
[89993f60ee3f:07180] Failing at address: 0x30
[89993f60ee3f:07180] [ 0] linux-vdso.so.1(__kernel_rt_sigreturn+0x0)[0xffffb4d177c0]
[89993f60ee3f:07180] [ 1] /usr/lib/aarch64-linux-gnu/libpmix.so.2(pmix_ptl_base_lost_connection+0x748)[0xffffb2264888]
[89993f60ee3f:07180] [ 2] /usr/lib/aarch64-linux-gnu/libpmix.so.2(pmix_ptl_base_recv_handler+0x1e4)[0xffffb2267074]
[89993f60ee3f:07180] [ 3] /usr/lib/aarch64-linux-gnu/libevent-2.1.so.6(+0x236ac)[0xffffb49c86ac]
[89993f60ee3f:07180] *** End of error message ***
Segmentation fault
```

**Test 4**
Input:
```
export OMPI_MCA_btl_vader_single_copy_mechanism=none
export PMIX_MCA_gds=hash
mpirun -np 2 --allow-run-as-root python3 -m mpi4py argon-dlp.py
```

Result:
```
Supported DL_POLY version 4.10
Supported DL_POLY version 4.10
Universe created with:
Dimensions [30.76, 30.76, 30.76]
Force field None
Number of atoms 0

/usr/local/lib/python3.9/site-packages/MDMC/MD/structures.py:944: UserWarning: No cutoff was set for the Coulombic interaction of this atom. The default cutoff of 10 Angstrom will be used. To set a cutoff, provide the argument cutoff=[value] when initialising the Atom object.
warnings.warn("No cutoff was set for the Coulombic interaction of this atom."
: Coulombic interaction for the Atom objectinitialized with the Coulomb interaction function.
Universe created with:
Dimensions [30.76, 30.76, 30.76]
Force field None
Number of atoms 0

DLPOLY.Z is not DLPoly, run may not work
Simulation created with dlpoly engine and settings:
temperature 120.0
numprocs 4.0

Folder minim exists, over-writing.
Simulation created with dlpoly engine and settings:
temperature 120.0
numprocs 4.0

Folder minim exists, over-writing.
[89993f60ee3f:07644] *** Process received signal ***
[89993f60ee3f:07644] Signal: Segmentation fault (11)
[89993f60ee3f:07644] Signal code: Address not mapped (1)
[89993f60ee3f:07644] Failing at address: 0xaaaaee64d5e0
[89993f60ee3f:07644] [ 0] linux-vdso.so.1(__kernel_rt_sigreturn+0x0)[0xffff9aa217c0]
[89993f60ee3f:07644] [ 1] /usr/lib/aarch64-linux-gnu/openmpi/lib/openmpi3/mca_btl_vader.so(mca_btl_vader_poll_handle_frag+0x15c)[0xffff90518114]
[89993f60ee3f:07644] [ 2] /usr/lib/aarch64-linux-gnu/openmpi/lib/openmpi3/mca_btl_vader.so(+0x4358)[0xffff90518358]
[89993f60ee3f:07644] [ 3] /usr/lib/aarch64-linux-gnu/libopen-pal.so.40(opal_progress+0x38)[0xffff91c66678]
[89993f60ee3f:07644] [ 4] /usr/lib/aarch64-linux-gnu/libopen-pal.so.40(ompi_sync_wait_mt+0x134)[0xffff91c6e35c]
[89993f60ee3f:07644] [ 5] /usr/lib/aarch64-linux-gnu/libmpi.so.40(ompi_request_default_wait+0x200)[0xffff91e0efb0]
[89993f60ee3f:07644] [ 6] /usr/lib/aarch64-linux-gnu/libmpi.so.40(ompi_comm_set+0x6c)[0xffff91df354c]
[89993f60ee3f:07644] [ 7] /usr/lib/aarch64-linux-gnu/libmpi.so.40(ompi_dpm_connect_accept+0x970)[0xffff91dfb228]
[89993f60ee3f:07644] [ 8] /usr/lib/aarch64-linux-gnu/libmpi.so.40(MPI_Comm_spawn+0x480)[0xffff91e2e750]
[89993f60ee3f:07644] [ 9] /usr/local/lib/python3.9/site-packages/mpi4py/MPI.cpython-39-aarch64-linux-gnu.so(+0x8f6b4)[0xffff91f6e6b4]
[89993f60ee3f:07644] [10] /usr/local/lib/libpython3.9.so.1.0(+0x11ab90)[0xffff9a758b90]
[89993f60ee3f:07644] [11] /usr/local/lib/libpython3.9.so.1.0(_PyObject_MakeTpCall+0x144)[0xffff9a72534c]
[89993f60ee3f:07644] [12] /usr/local/lib/libpython3.9.so.1.0(_PyEval_EvalFrameDefault+0x47e8)[0xffff9a7adf10]
[89993f60ee3f:07644] [13] /usr/local/lib/libpython3.9.so.1.0(+0x16a82c)[0xffff9a7a882c]
[89993f60ee3f:07644] [14] /usr/local/lib/libpython3.9.so.1.0(_PyFunction_Vectorcall+0x1cc)[0xffff9a725d24]
[89993f60ee3f:07644] [15] /usr/local/lib/libpython3.9.so.1.0(+0xe9c94)[0xffff9a727c94]
[89993f60ee3f:07644] [16] /usr/local/lib/libpython3.9.so.1.0(_PyEval_EvalFrameDefault+0x11d4)[0xffff9a7aa8fc]
[89993f60ee3f:07644] [17] /usr/local/lib/libpython3.9.so.1.0(+0x16a82c)[0xffff9a7a882c]
[89993f60ee3f:07644] [18] /usr/local/lib/libpython3.9.so.1.0(_PyFunction_Vectorcall+0x1cc)[0xffff9a725d24]
[89993f60ee3f:07644] [19] /usr/local/lib/libpython3.9.so.1.0(+0xe9c94)[0xffff9a727c94]
[89993f60ee3f:07644] [20] /usr/local/lib/libpython3.9.so.1.0(PyVectorcall_Call+0xb8)[0xffff9a726c88]
[89993f60ee3f:07644] [21] /usr/local/lib/libpython3.9.so.1.0(_PyEval_EvalFrameDefault+0x2050)[0xffff9a7ab778]
[89993f60ee3f:07644] [22] /usr/local/lib/libpython3.9.so.1.0(+0x16a82c)[0xffff9a7a882c]
[89993f60ee3f:07644] [23] /usr/local/lib/libpython3.9.so.1.0(_PyFunction_Vectorcall+0x1cc)[0xffff9a725d24]
[89993f60ee3f:07644] [24] /usr/local/lib/libpython3.9.so.1.0(+0xe9c94)[0xffff9a727c94]
[89993f60ee3f:07644] [25] /usr/local/lib/libpython3.9.so.1.0(PyVectorcall_Call+0xb8)[0xffff9a726c88]
[89993f60ee3f:07644] [26] /usr/local/lib/libpython3.9.so.1.0(_PyEval_EvalFrameDefault+0x2050)[0xffff9a7ab778]
[89993f60ee3f:07644] [27] /usr/local/lib/libpython3.9.so.1.0(+0x16a82c)[0xffff9a7a882c]
[89993f60ee3f:07644] [28] /usr/local/lib/libpython3.9.so.1.0(_PyFunction_Vectorcall+0x1cc)[0xffff9a725d24]
[89993f60ee3f:07644] [29] /usr/local/lib/libpython3.9.so.1.0(+0xe9c94)[0xffff9a727c94]
[89993f60ee3f:07644] *** End of error message ***
[89993f60ee3f:07639] *** Process received signal ***
[89993f60ee3f:07639] Signal: Segmentation fault (11)
[89993f60ee3f:07639] Signal code: Address not mapped (1)
[89993f60ee3f:07639] Failing at address: 0x30
[89993f60ee3f:07639] [ 0] linux-vdso.so.1(__kernel_rt_sigreturn+0x0)[0xffff8684a7c0]
[89993f60ee3f:07639] [ 1] /usr/lib/aarch64-linux-gnu/libpmix.so.2(pmix_ptl_base_lost_connection+0x748)[0xffff83d97888]
[89993f60ee3f:07639] [ 2] /usr/lib/aarch64-linux-gnu/libpmix.so.2(pmix_ptl_base_recv_handler+0x1e4)[0xffff83d9a074]
[89993f60ee3f:07639] [ 3] /usr/lib/aarch64-linux-gnu/libevent-2.1.so.6(+0x236ac)[0xffff864fb6ac]
[89993f60ee3f:07639] *** End of error message ***
Segmentation fault
```