Azure / Azure/cyclecloud-slurm
Problem using the default image
- Dominant language
- Python
- Stars
- 84
- Forks
- 56
- Avg merge
- 3d 17h
- Merged PRs (30d)
- 1
Description
I'm not sure if I'm doing something wrong, but I'm utterly failing to use the default Slurm image to do some basic MPI.
I've stood up a simple cyclecloud-slurm setup with the default image (Cycle CentOS 7) for all node types.
I can run a simple single node MPI job as long as I use Intel MPI:
srun -n2 mpirun ./hello
That works fine.
But as soon as I try to use more than one node, I get all sorts of infiniband related error messages.
If I try to use a machinetype that supports Infiniband I end up with no functioning MPI at all, as it appears to not get installed.
Try even a single node MPI test with OpenMPI and I get errors possibly related to hostname issues:
```
--------------------------------------------------------------------------
ORTE has lost communication with a remote daemon.
HNP daemon : [[22239,0],0] on node ip-0A000406
Remote daemon: [[22239,0],1] on node hpc-1
```
Looks like OpenMPI is working with the Slurm node name (hpc-1) rather than that actual hostname (ip-0A000406), and then possibly getting upset as a result?
If I've missed something really obvious, please do point it out :)
Thanks,
John
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.