ORNL / ORNL/ReSolve

Bug: 38/49 tests fail on CUDA when using address sanitizers.

Open
#387 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
C++
Stars
84
Forks
13
PR merge metrics
No merged PRs in 30d

Description

Description how to reproduce the bug
cmake -DCMAKE_BUILD_TYPE=RelWithDebInfo -DRESOLVE_USE_ASAN=yes -DRESOLVE_USE_UBSAN=yes -DRESOLVE_USE_KLU=yes -DRESOLVE_USE_CUDA=yes -DRESOLVE_USE_LUSOL=yes -DCMAKE_CUDA_ARCHITECTURES=70 -DCMAKE_INSTALL_PREFIX=../bin -DSUITESPARSE_DIR=/usr/lib/x86_64-linux-gnu -DSUITESPARSE_INCLUDE_DIR=/usr/include/suitesparse  ../ReSolve/
make -j
ctest -j

Memory leaks such as:

Direct leak of 2996 byte(s) in 4 object(s) allocated from:
    #0 0x7fb54b596887 in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:145
    #1 0x7fb515e8b810  (/usr/local/cuda/lib64/libcusparse.so.12+0xa8b810)

Direct leak of 408 byte(s) in 1 object(s) allocated from:
    #0 0x7fb54b596a57 in __interceptor_calloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:154
    #1 0x7fb4d318c004  (/lib/x86_64-linux-gnu/libcuda.so.1+0x2d5004)
    #2 0x7fb4d31cb151  (/lib/x86_64-linux-gnu/libcuda.so.1+0x314151)
    #3 0x7fb4d31bc92f in cuInit (/lib/x86_64-linux-gnu/libcuda.so.1+0x30592f)
    #4 0x7fb532afd8c9  (/usr/local/cuda/lib64/libcublas.so.12+0x4fd8c9)
    #5 0x7fb532afda47  (/usr/local/cuda/lib64/libcublas.so.12+0x4fda47)
    #6 0x7fb51460fee7 in __pthread_once_slow nptl/pthread_once.c:116

The same errors do not appear when the address sanitizer is not enabled, or alternatively during CPU only builds.

Re::Solve version

All versions

System and environment details

Build environment:

  • pdesx system
  • CUDA 12.9
  • gcc 11.4.0
Additional information

This seems like NVIDIA's error.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the report with the provided CMake options on CUDA 12.9 and run ctest -j; compare the ASAN CUDA results with CPU-only and non-ASAN builds. Inspect the reported sanitizer leaks from libcusparse, libcuda, and libcublas, and establish whether the failures are project defects or NVIDIA runtime behavior; done requires a confirmed cause and documented test outcome.

Written by the indexing model from the issue text.

Assessment

Tech stack
cmake, cpp
Domain
build-system, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.