E4S-Project / E4S-Project/testsuite

Slate+rocm test fails on crusher

Open
#59 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Fortran
Stars
21
Forks
33
Avg merge
3m
Merged PRs (30d)
1

Description

@G-Ragghianti @mgates3

The slate standalone test defined here: https://github.com/E4S-Project/testsuite/tree/master/validation_tests/slate-rocm fails when run on the slate build installed as part of the e4s 22.11 deployment on crusher using these variants with the console output below:

-- linux-sles15-zen3 / gcc@11.2.0 -------------------------------
edojdwe slate@2022.07.00~cuda~ipo+mpi+openmp+rocm+shared amdgpu_target=gfx90a build_system=cmake build_type=RelWithDebInfo
7ej4aoh     blaspp@2022.07.00~cuda~ipo+openmp+rocm+shared amdgpu_target=gfx90a build_system=cmake build_type=RelWithDebInfo
c6gpjyk         cmake@3.24.2~doc+ncurses+ownlibs~qt build_system=generic build_type=Release
igbrz2c             ncurses@6.3~symlinks+termlib abi=none build_system=autotools
savxweu                 pkgconf@1.8.0 build_system=autotools
kq7i44v             openssl@1.1.1s~docs~shared build_system=generic certs=mozilla
6ki4n47                 ca-certificates-mozilla@2022-10-11 build_system=generic
ucjrwtm                 perl@5.36.0+cpanm+shared+threads build_system=generic
gqdvawb                     berkeley-db@18.1.40+cxx~docs+stl build_system=autotools patches=26090f4,b231fcc
g2bpsoz                     bzip2@1.0.8~debug~pic+shared build_system=generic
rnafwos                         diffutils@3.8 build_system=autotools
xfogkcu                             libiconv@1.16 build_system=autotools libs=shared,static
otqsxvg                     gdbm@1.23 build_system=autotools
6mvf2em                         readline@8.1.2 build_system=autotools
76b2zrq                     zlib@1.2.13+optimize+pic+shared build_system=makefile
bzm57qy         hip@5.2.0~ipo build_system=cmake build_type=Release patches=959d1fe
e5ldtkh         hsa-rocr-dev@5.2.0+image~ipo+shared build_system=cmake build_type=Release patches=71e6851
mm6mnhr         llvm-amdgpu@5.2.0~ipo~link_llvm_dylib~llvm_dylib~openmp+rocm-device-libs build_system=cmake build_type=Release patches=a08bbe1
bgpvt5g         openblas@0.3.21~bignuma~consistent_fpcsr+fortran~ilp64+locking+pic+shared build_system=makefile patches=d3d9b15 symbol_suffix=none threads=openmp
g2sf37k         rocblas@5.2.0~ipo+tensile amdgpu_target=auto build_system=cmake build_type=Release patches=81591d9
oaykapp     cray-mpich@8.1.17+wrappers build_system=generic
izppu2z     lapackpp@2022.07.00~cuda~ipo+rocm+shared amdgpu_target=gfx90a build_system=cmake build_type=RelWithDebInfo
orsl6og         rocsolver@5.2.0~ipo+optimal amdgpu_target=auto build_system=cmake build_type=Release
slate+rocm %gcc: edojdwe
terminate called after throwing an instance of 'std::out_of_range'
  what():  map::at
terminate called after throwing an instance of 'std::out_of_range'
  what():  map::at
terminate called after throwing an instance of 'std::out_of_range'
  what():  map::at
srun: error: crusher124: tasks 1-3: Aborted
srun: launch/slurm: _step_signal: Terminating StepId=230307.0
slurmstepd: error: *** STEP 230307.0 ON crusher124 CANCELLED AT 2022-12-14T18:10:38 ***
srun: error: crusher124: task 0: Terminated
srun: Force Terminated StepId=230307.0

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the standalone test in validation_tests/slate-rocm and reproduce it on the slate build from the E4S 22.11 deployment on crusher, using the listed variants and console output as the baseline. Trace the std::out_of_range map::at failure and verify that the validation test completes without the abort on the affected environment.

Written by the indexing model from the issue text.

Assessment

Tech stack
cmake
Domain
hpc, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.