catchorg / catchorg/Catch2

'ApprovalTests' fails in Catch 2.10.0

Open
#1,780 4 comments 0 reactions 0 assignees View on GitHub
3rd party bug
Dominant language
C++
Stars
21.5k
Forks
3.5k
Avg merge
3d 16h
Merged PRs (30d)
2

Description

**Describe the bug**
CTest failure

**Reproduction steps**
just type 'ctest -j 2 --output-on-failure'

**Platform information:**
- OS: Windows 10
- Compiler+version Mingw32-w64
- Catch version: 2.10.0 latest version

**Additional context**
CTest output:
```
19/19 Test #12: ApprovalTests ..........................***Failed Error regular expression found in output. Regex=[Results differed] 1.88 sec
Running approvals against executable:
C:/Users/Amit/source/repos/Catch2/build/projects/SelfTest.exe

console.std:
Results matched

console.sw:
--- console.sw.approved.txt

+++ console.sw.unapproved.txt

- -0.0 and 0 are within 2.22045e-12% of each other
+ -0.0 and 0 are within 2.22045e-012% of each other
- 0.0 and 2.22507e-308 are within 2.22045e-12% of each other
+ 0.0 and 2.22507e-308 are within 2.22045e-012% of each other
- 0.0f and 1.17549e-38 are within 0.00119209% of each other
+ 0.0f and 1.17549e-038 are within 0.00119209% of each other

****************************
Results differed

console.swa4:
Results matched

junit.sw:
Results matched

xml.sw:
--- xml.sw.approved.txt

+++ xml.sw.unapproved.txt

- -0.0 and 0 are within 2.22045e-12% of each other
+ -0.0 and 0 are within 2.22045e-012% of each other
- 0.0 and 2.22507e-308 are within 2.22045e-12% of each other
+ 0.0 and 2.22507e-308 are within 2.22045e-012% of each other
- 0.0f and 1.17549e-38 are within 0.00119209% of each other
+ 0.0f and 1.17549e-038 are within 0.00119209% of each other

****************************
Results differed

compact.sw:
--- compact.sw.approved.txt

+++ compact.sw.unapproved.txt

-Matchers.tests.cpp:: passed: -0., WithinRel(0.) for: -0.0 and 0 are within 2.22045e-12% of each other
-Matchers.tests.cpp:: passed: v1, WithinRel(v2) for: 0.0 and 2.22507e-308 are within 2.22045e-12% of each other
+Matchers.tests.cpp:: passed: -0., WithinRel(0.) for: -0.0 and 0 are within 2.22045e-012% of each other
+Matchers.tests.cpp:: passed: v1, WithinRel(v2) for: 0.0 and 2.22507e-308 are within 2.22045e-012% of each other
-Matchers.tests.cpp:: passed: v1, WithinRel(v2) for: 0.0f and 1.17549e-38 are within 0.00119209% of each other
+Matchers.tests.cpp:: passed: v1, WithinRel(v2) for: 0.0f and 1.17549e-038 are within 0.00119209% of each other

****************************
Results differed

If these differences are expected, run approve.py to approve new baselines.

95% tests passed, 1 tests failed out of 19

Total Test time (real) = 1.93 sec

The following tests FAILED:
12 - ApprovalTests (Failed)
Errors while running CTest

```

Is this an expected behavior?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.