QMCPACK / QMCPACK/qmcpack

Suspicious QP short test failures with anomalously large errors

Open
#5,172 5 comments 0 reactions 0 assignees View on GitHub
stale
Dominant language
C++
Stars
403
Forks
154
Avg merge
1d 13h
Merged PRs (30d)
90

Description

**Describe the bug**

For the excited state case, the C4 multideterminant tests seem to be wrong in mixed precision. The ground state calculation looks more reliable, but there are still some large deviations. Possibly there is a bug here, possibly the test is not well enough designed and doesn't get reliable error bars. It looks like outlier energies can be obtained, indicating a possible bug.

I have been going through the short tests to "tame" them. Most failures would need only modest increase in error bar to pass, e.g. are failing at 3.1 sigma. These QP ones have noticeably much larger deviations.

@kgasperich , @anbenali ? Any insights/guesses?

short-C4_md-ae-excited-qp-vmc-1-16-totenergy
https://cdash.qmcpack.org/testSummary.php?project=1&name=short-C4_md-ae-excited-qp-vmc-1-16-totenergy&date=2024-09-19 (52 sigma failure for one of the gcc14-mixed builds!)

short-C4_md-ae-excited-qp-vmc-1-16-variance
https://cdash.qmcpack.org/testSummary.php?project=1&name=short-C4_md-ae-excited-qp-vmc-1-16-variance&date=2024-09-19

short-C4_md-ae-excited-qp-vmc-1-16-kinetic
https://cdash.qmcpack.org/testSummary.php?project=1&name=short-C4_md-ae-excited-qp-vmc-1-16-kinetic&date=2024-09-19

Ground state seems more reliable, but still some large deviations:

short-C4_md-ae-ground-qp-vmc-1-16-variance
https://cdash.qmcpack.org/testSummary.php?project=1&name=short-C4_md-ae-ground-qp-vmc-1-16-variance&date=2024-09-19

**To Reproduce**

Standard ctest runs with develop

**Expected behavior**

Test passes reliably.

**System:**
CPU systems

Contributor guide

Open the contributing guide

Research direction

Start by running the standard ctest suite on develop and inspect the named short-C4_md-ae-excited-qp-vmc-1-16 and short-C4_md-ae-ground-qp-vmc-1-16 tests, using the linked CDash results to compare mixed-precision builds. Trace the anomalous energy, variance, and kinetic deviations to determine whether the calculation or test error bars are at fault; done means the tests pass reliably or the confirmed defect and appropriate test correction are identified.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
hpc, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.