NemoLite2D benchmarking results on HPC-level hardware

Open
#41 5 comments 0 reactions 3 assignees View on GitHub

@rupertford is already working on this.

Since Jun 9, 2020.

Assessment

This issue has not been assessed yet.

Description

enhancement

I think I (and @sergisiso ?) may be reaching the point where we're reaching some level of maturity in the newer versions of the NemoLite2D, and I think it would be best for us to begin to collate results. As previously discussed I think an idea was a paper discussing comparative benchmarks of various parallel systems applied to the NemoLite2D benchmark.

As far as I understand it, we have the following versions:
Manual versions:

  • Fortran OpenMP
  • Fortran serial
  • Fortran/C OpenCL (?)
  • Regent
  • C++ OpenMP
  • C++ Kokkos

PSYclone generated:

  • OpenMP
  • OpenACC

My proposal would be then the following benchmarking results:

  • Fortran OpenMP version with gcc9 on Skylake

  • Fortran OpenMP version with intel/? on Skylake

  • Fortran Serial version with gcc9 on Skylake

  • Fortran Serial version with intel/? on Skylake

  • Regent version on Skylake

  • C++ OpenMP version with gcc9 on Skylake

  • C++ OpenMP version with Intel on Skylake

  • C++ Kokkos version on Skylake

  • PSYclone OpenMP generated version on Skylake

  • OpenCL version on appropriate hardware (@arporter) - I assume we want both the GPU in Glados and on ScafellPike? FPGA also an option.

  • OpenACC version on GPU

  • PSYclone generated OpenACC version on GPU

I think what we should record for each set of results is:

  1. git hash for the commit
  2. Compiler version & Compile flags used (where appropriate, e.g. Regent version will only have the flags passed to the install.py)
  3. Hardware (should be simple)
  4. Runtimes at various threadcounts, I'd think at least 1/2/4/8/16/32 for CPUs, and strong scalability plots/parallel efficiency plots corresponding to those results. I would expect the scalability results should be with respect to the faster of the serial version or OpenMP version for the given compiler where appropriate (or plots corresponding to both). For GPUs I assume we should run various tests and just show the best achievable runtimes and the appropriate data for reproducability.
  5. Runtime parameters should be tested where appropriate too - e.g. OpenMP Schedule options.

Does this all seem reasonable?

Dominant language
Fortran
Stars
7
Forks
5
PR merge metrics
No merged PRs in 30d

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from stfc/PSycloneBench

All issues in stfc/PSycloneBench

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.