google / google/googletest

[Bug]: Tests are reported as completed when GTEST_SKIP is used inside an environment's SetUp

Open
#4,653 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
39.5k
Forks
10.9k
Avg merge
6d 13h
Merged PRs (30d)
1

Description

### Describe the issue

When adding a global test environment whose `SetUp` function calls `GTEST_SKIP`, the actual tests are still reported as executed in the test program's output, and as `completed` in the XML output.

Running the attached source results in the following output :
```
$ ./foobar --gtest_output=xml:output.xml
[==========] Running 2 tests from 2 test suites.
[----------] Global test environment set-up.
/home/daniel/zabawki/poligon5/main.cpp:8: Skipped
skipping everything

skipping everything

[----------] Global test environment tear-down
[==========] 2 tests from 2 test suites ran. (0 ms total)
[ PASSED ] 2 tests.
```

Meanwhile, the XML reports that the test results were completed and that the execution of each test took place at "time zero", though the root test suite's timestamp is set correctly :
```xml






```

I expected the usage of a `GTEST_SKIP` in a global test environment to be equivalent to a `GTEST_SKIP` in every single test case, and the output to be the following :

```
$ ./foobar
[==========] Running 2 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 1 test from Foo
[ RUN ] Foo.Bar
/home/daniel/zabawki/poligon5/main.cpp:13: Skipped
foobar

[ SKIPPED ] Foo.Bar (0 ms)
[----------] 1 test from Foo (0 ms total)

[----------] 1 test from Bar
[ RUN ] Bar.Foo
/home/daniel/zabawki/poligon5/main.cpp:18: Skipped
barfoo

[ SKIPPED ] Bar.Foo (0 ms)
[----------] 1 test from Bar (0 ms total)

[----------] Global test environment tear-down
[==========] 2 tests from 2 test suites ran. (0 ms total)
[ PASSED ] 0 tests.
[ SKIPPED ] 2 tests, listed below:
[ SKIPPED ] Foo.Bar
[ SKIPPED ] Bar.Foo
```

In which case, the XML output also includes appropriate elements to indicate that test execution was skipped (and the timestamps are correct, too) :
```xml










```

### Steps to reproduce the problem

```cpp
#include

struct Env : public ::testing::Environment
{
void SetUp() override
{
GTEST_SKIP() << "skipping everything";
}
};

TEST(Foo, Bar)
{
ASSERT_EQ(1, 2);
}

TEST(Bar, Foo)
{
ASSERT_EQ(2, 1);
}

int main(int argc, char **argv)
{
::testing::InitGoogleTest(&argc, argv);
::testing::AddGlobalTestEnvironment(new Env);
return RUN_ALL_TESTS();
}
```

### What version of GoogleTest are you using?

`d144031940543e15423a25ae5a8a74141044862f`

### What operating system and version are you using?

Arch Linux

### What compiler and version are you using?

gcc version 14.2.1 20240910

### What build system are you using?

cmake version 3.30.5

### Additional context

_No response_

Contributor guide

Open the contributing guide

Research direction

Start with the minimal C++ reproducer in the issue, focusing on global Environment::SetUp and the GTEST_SKIP path. Trace how test execution results and XML output are recorded after setup is skipped, then verify that the console and XML report each test as skipped with appropriate timestamps and skip details.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.