Test failed using the JUnit reporter but not using the console or xml reporter
- Dominant language
- C++
- Stars
- 21.5k
- Forks
- 3.5k
- Avg merge
- 3d 16h
- Merged PRs (30d)
- 2
Description
**Describe the bug**
When I run the following test, the last check `CHECK( i == 3 )` fails when I use the JUnit reporter (passing the `-r junit` argument to the binary), but passes when I use the xml or console reporter.
```c++
// Stupid test to understand the behaviour of the std::filesystem library
auto it2 = std::filesystem::directory_iterator("C:/tests/sample_dir", ec);
int i = 0;
for ( ; it2 != std::filesystem::directory_iterator() ; it2.increment(ec)) {
REQUIRE( !ec );
CHECK( it2 != std::filesystem::directory_iterator() );
++i;
}
CHECK( i == 3 ); // <-- This fails using the JUnit reporter but not with the xml and console reporter
```
**Expected behavior**
The test should pass for all reporters
**Reproduction steps**
1. Create a project to generate a binary and add the code below.
2. Create on your computer a folder and add 3 items (files or folders). If you are under macOS, you may have to create only two items because the Finder application will create a `.DS_Store` file (or you can adapt the test in consequence)
3. Run the binary using the argument `-r console` (pass)
4. Run the binary using the argument `-r xml` (pass)
5. Run the binary using the argument `-r junit` (fail)
```c++
// Copy/paste from a larger CPP file. This should work as is.
#define CATCH_CONFIG_MAIN
#include
#include
#include
TEST_CASE("std::filesystem::directory_iterator") {
SECTION("using a directory") {
std::error_code ec;
auto it2 = std::filesystem::directory_iterator("C:/tests/sample_dir", ec);
int i = 0;
for ( ; it2 != std::filesystem::directory_iterator() ; it2.increment(ec)) {
REQUIRE( !ec );
CHECK( it2 != std::filesystem::directory_iterator() );
++i;
}
CHECK( i == 3 );
}
}
```
**Platform information:**
- Windows
- OS: Win10 (1809)
- Compiler+version: MSVC 2019
- Catch version: **v2.10.2** (tested with v2.10.0 too)
- Build type: Debug
- macOS
- OS: macOS 10.14
- Compiler+version: XCode 11
- Catch version: **v2.10.0**
- Build type: Debug
- Azure (image windows-2019)
- OS: Windows Server 2019
- Compiler+version: Visual Studio 16 2019
- Catch version: **v2.10.0**
- Build type: Debug
- Azure (image vs2017-win2016)
- OS: Windows Server 2016
- Compiler+version: Visual Studio 15 2017
- Catch version: **v2.10.0**
- Build type: Debug
**Supplementary information:**
I discovered this problem does not happen when I comment the first CHECK command.
The following test does not fail with the JUnit reporter
```c++
std::error_code ec;
auto it2 = std::filesystem::directory_iterator("C:/tests/sample_dir", ec);
int i = 0;
for ( ; it2 != std::filesystem::directory_iterator() ; it2.increment(ec)) {
REQUIRE( !ec );
// CHECK( it2 != std::filesystem::directory_iterator() ); // <-- Pass with the JUnit reporter if this line is commented!
++i;
}
CHECK( i == 3 );
```
Contributor guide
Assessment
This issue has not been assessed yet.