Suspicious QP short test failures with anomalously large errors
- Dominant language
- C++
- Stars
- 403
- Forks
- 154
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 90
Description
**Describe the bug**
For the excited state case, the C4 multideterminant tests seem to be wrong in mixed precision. The ground state calculation looks more reliable, but there are still some large deviations. Possibly there is a bug here, possibly the test is not well enough designed and doesn't get reliable error bars. It looks like outlier energies can be obtained, indicating a possible bug.
I have been going through the short tests to "tame" them. Most failures would need only modest increase in error bar to pass, e.g. are failing at 3.1 sigma. These QP ones have noticeably much larger deviations.
@kgasperich , @anbenali ? Any insights/guesses?
short-C4_md-ae-excited-qp-vmc-1-16-totenergy
https://cdash.qmcpack.org/testSummary.php?project=1&name=short-C4_md-ae-excited-qp-vmc-1-16-totenergy&date=2024-09-19 (52 sigma failure for one of the gcc14-mixed builds!)
short-C4_md-ae-excited-qp-vmc-1-16-variance
https://cdash.qmcpack.org/testSummary.php?project=1&name=short-C4_md-ae-excited-qp-vmc-1-16-variance&date=2024-09-19
short-C4_md-ae-excited-qp-vmc-1-16-kinetic
https://cdash.qmcpack.org/testSummary.php?project=1&name=short-C4_md-ae-excited-qp-vmc-1-16-kinetic&date=2024-09-19
Ground state seems more reliable, but still some large deviations:
short-C4_md-ae-ground-qp-vmc-1-16-variance
https://cdash.qmcpack.org/testSummary.php?project=1&name=short-C4_md-ae-ground-qp-vmc-1-16-variance&date=2024-09-19
**To Reproduce**
Standard ctest runs with develop
**Expected behavior**
Test passes reliably.
**System:**
CPU systems
Contributor guide
Research direction
Start by running the standard ctest suite on develop and inspect the named short-C4_md-ae-excited-qp-vmc-1-16 and short-C4_md-ae-ground-qp-vmc-1-16 tests, using the linked CDash results to compare mixed-precision builds. Trace the anomalous energy, variance, and kinetic deviations to determine whether the calculation or test error bars are at fault; done means the tests pass reliably or the confirmed defect and appropriate test correction are identified.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- hpc, testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 38/100