Example to create and access a PHDF5 dataset under Unify

Nobody has claimed this yet.

I'm creating this example to document a simple tests that creates and accesses a simple dataset in Parallel-HDF5, under Unify.

The original example is taken from the HDF5 website: https://support.hdfgroup.org/ftp/HDF5/examples/parallel/Dataset.c

Based on that example, the changes made for Unify are these:
a) Change the filename from SDS.h5 to ufs:/unifyfs/SDS.h5
b) Call to a function to mount /unifyfs after MPI_Init;
c) Call to unifyfs_umount() at the end.

On Quartz, after "load module hdf5-parallel", I build this example with

h5pcc -o prog-gotcha mpi_prog.c -I${UNIFYFS}/include -L${UNIFYFS}/lib -lunifyfs_gotcha ${UNIFYFS}/lib64/libgotcha.so

This example, run with 8 processors on 2 nodes, used to work fine with the "new-margotree" branch of Unify. It is no longer working with the recent dev version (as of Nov.20/2020). It only works in one node.

This is the modified source code:

```
/*
* This example writes data to the HDF5 file.
* Number of processes is assumed to be 1 or multiples of 2 (up to 8)
*/

#include "hdf5.h"
#include "stdlib.h"

#define H5FILE_NAME "ufs:/unifyfs/SDS.h5"
#define DATASETNAME "IntArray"
#define NX 8 /* dataset dimensions */
#define NY 5
#define RANK 2

int
main (int argc, char **argv)
{
/*
* HDF5 APIs definitions
*/
hid_t file_id, dset_id; /* file and dataset identifiers */
hid_t filespace; /* file and memory dataspace identifiers */
hsize_t dimsf[] = {NX, NY}; /* dataset dimensions */
int *data; /* pointer to data buffer to write */
hid_t plist_id; /* property list identifier */
int i;
herr_t status;

/*
* MPI variables
*/
int mpi_size, mpi_rank;
MPI_Comm comm = MPI_COMM_WORLD;
MPI_Info info = MPI_INFO_NULL;

/*
* Initialize MPI
*/
MPI_Init(&argc, &argv); my_unify_mount();
MPI_Comm_size(comm, &mpi_size);
MPI_Comm_rank(comm, &mpi_rank);

/*
* Initialize data buffer
*/
data = (int *) malloc(sizeof(int)*dimsf[0]*dimsf[1]);
for (i=0; i < dimsf[0]*dimsf[1]; i++) {
data[i] = i;
}
/*
* Set up file access property list with parallel I/O access
*/
plist_id = H5Pcreate(H5P_FILE_ACCESS);
H5Pset_fapl_mpio(plist_id, comm, info);

/*
* Create a new file collectively and release property list identifier.
*/
file_id = H5Fcreate(H5FILE_NAME, H5F_ACC_TRUNC, H5P_DEFAULT, plist_id);
H5Pclose(plist_id);

/*
* Create the dataspace for the dataset.
*/
filespace = H5Screate_simple(RANK, dimsf, NULL);

/*
* Create the dataset with default properties and close filespace.
*/
dset_id = H5Dcreate(file_id, DATASETNAME, H5T_NATIVE_INT, filespace,
H5P_DEFAULT, H5P_DEFAULT, H5P_DEFAULT);
/*
* Create property list for collective dataset write.
*/
plist_id = H5Pcreate(H5P_DATASET_XFER);
H5Pset_dxpl_mpio(plist_id, H5FD_MPIO_COLLECTIVE);

/*
* To write dataset independently use
*
* H5Pset_dxpl_mpio(plist_id, H5FD_MPIO_INDEPENDENT);
*/

status = H5Dwrite(dset_id, H5T_NATIVE_INT, H5S_ALL, H5S_ALL,
plist_id, data);
free(data);

/*
* Close/release resources.
*/
H5Dclose(dset_id);
H5Sclose(filespace);
H5Pclose(plist_id);
H5Fclose(file_id);
unifyfs_unmount();
MPI_Finalize();

return 0;
}

/* --------------------------------------------------------------- */
#include

int my_unify_mount()
{
int ret = 0, rank, nranks;

MPI_Comm_size(MPI_COMM_WORLD, &nranks);
MPI_Comm_rank(MPI_COMM_WORLD, &rank);

ret = unifyfs_mount("/unifyfs", rank, nranks, 0);
if (ret) {
printf("[%d] unifyfs_mount failed (return = %d)\n", rank, ret);
exit(-1);
}
}

/* --------------------------------------------------------------- */

```

Start with the modified mpi_prog.c example and its h5pcc build command after loading parallel HDF5 on Quartz. Run it with 8 processors across two nodes and compare the result with a single-node run, focusing on the MPI initialization, UnifyFS mount, and ufs:/unifyfs/SDS.h5 access. Done means the dataset example works across two nodes on the current development version, or the regression is narrowed to a reproducible failure.

Written by the indexing model from the issue text.