NemoLite2D benchmarking results on HPC-level hardware
@rupertford is already working on this.
Since Jun 9, 2020.
Assessment
This issue has not been assessed yet.
Description
I think I (and @sergisiso ?) may be reaching the point where we're reaching some level of maturity in the newer versions of the NemoLite2D, and I think it would be best for us to begin to collate results. As previously discussed I think an idea was a paper discussing comparative benchmarks of various parallel systems applied to the NemoLite2D benchmark.
As far as I understand it, we have the following versions:
Manual versions:
- Fortran OpenMP
- Fortran serial
- Fortran/C OpenCL (?)
- Regent
- C++ OpenMP
- C++ Kokkos
PSYclone generated:
- OpenMP
- OpenACC
My proposal would be then the following benchmarking results:
-
Fortran OpenMP version with gcc9 on Skylake
-
Fortran OpenMP version with intel/? on Skylake
-
Fortran Serial version with gcc9 on Skylake
-
Fortran Serial version with intel/? on Skylake
-
Regent version on Skylake
-
C++ OpenMP version with gcc9 on Skylake
-
C++ OpenMP version with Intel on Skylake
-
C++ Kokkos version on Skylake
-
PSYclone OpenMP generated version on Skylake
-
OpenCL version on appropriate hardware (@arporter) - I assume we want both the GPU in Glados and on ScafellPike? FPGA also an option.
-
OpenACC version on GPU
-
PSYclone generated OpenACC version on GPU
I think what we should record for each set of results is:
- git hash for the commit
- Compiler version & Compile flags used (where appropriate, e.g. Regent version will only have the flags passed to the
install.py) - Hardware (should be simple)
- Runtimes at various threadcounts, I'd think at least 1/2/4/8/16/32 for CPUs, and strong scalability plots/parallel efficiency plots corresponding to those results. I would expect the scalability results should be with respect to the faster of the serial version or OpenMP version for the given compiler where appropriate (or plots corresponding to both). For GPUs I assume we should run various tests and just show the best achievable runtimes and the appropriate data for reproducability.
- Runtime parameters should be tested where appropriate too - e.g. OpenMP Schedule options.
Does this all seem reasonable?
- Dominant language
- Fortran
- Stars
- 7
- Forks
- 5
- PR merge metrics
- No merged PRs in 30d
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from stfc/PSycloneBench
-
help wanted
Difficulty 4/5 3-5 days Newbie friendliness 25/100
stfc/PSycloneBench#98 · 11 comments ·
-
Difficulty 4/5 3-5 days Newbie friendliness 20/100
stfc/PSycloneBench#97 · 10 comments ·
-
Difficulty 4/5 3-5 days Newbie friendliness 38/100
stfc/PSycloneBench#94 · 1 comment ·
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
stfc/PSycloneBench#92 ·
-
stfc/PSycloneBench#84 · 1 assignee ·