google / google/googletest

[Bug]: Throwing from `OnTestPartResult` after when handling an exception causes entire gtest run to exit abnormally

Open
#4,791 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
39.5k
Forks
10.9k
Avg merge
6d 13h
Merged PRs (30d)
1

Description

### Describe the issue

I'm using this well known trick of turning assertion failures in subroutines into exceptions documented at https://google.github.io/googletest/advanced.html#asserting-on-subroutines-with-an-exception

```cpp
class ThrowListener : public testing::EmptyTestEventListener {
void OnTestPartResult(const testing::TestPartResult& result) override {
if (result.type() == testing::TestPartResult::kFatalFailure) {
throw testing::AssertionException(result);
}
}
};
int main(int argc, char** argv) {
...
testing::UnitTest::GetInstance()->listeners().Append(new ThrowListener);
return RUN_ALL_TESTS();
}
```

We also have a test case for it: https://github.com/google/googletest/blob/b007c54f2944e193ac44fba1bc997cb65826a0b9/googletest/test/gtest_assert_by_exception_test.cc

However, it changes the gtest behavior when test cases throw exceptions.

By default, if any test throws an exception from the test body, gtest will consider it as failed and continue to run remaining tests followed by tear down/cleanup.

However, if the above mentioned trick is used, gtest will stop abnormally.

For example, if I change the test linked above

```diff
--- i/googletest/test/gtest_assert_by_exception_test.cc
+++ w/googletest/test/gtest_assert_by_exception_test.cc
@@ -57,6 +57,10 @@ void Fail(const char* msg) {

static void AssertFalse() { ASSERT_EQ(2, 3) << "Expected failure"; }

+TEST(Test, First) {
+ throw std::runtime_error("an exception which should fail a single test");
+}
+
// Tests that an assertion failure throws a subclass of
// std::runtime_error.
TEST(Test, Test) {
```

I get the following behavior:

```
$ bazel run //googletest/test:gtest_assert_by_exception_test
INFO: Analyzed target //googletest/test:gtest_assert_by_exception_test (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
Target //googletest/test:gtest_assert_by_exception_test up-to-date:
bazel-bin/googletest/test/gtest_assert_by_exception_test
INFO: Elapsed time: 0.030s, Critical Path: 0.00s
INFO: 1 process: 1 internal.
INFO: Build completed successfully, 1 total action
INFO: Running command line: external/bazel_tools/tools/test/test-setup.sh googletest/test/gtest_assert_by_exception_test
exec ${PAGER:-/usr/bin/less} "$0" || exit 1
Executing tests from //googletest/test:gtest_assert_by_exception_test
-----------------------------------------------------------------------------
[==========] Running 3 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 3 tests from Test
[ RUN ] Test.First
unknown file: Failure
C++ exception with description "an exception which should fail a single test" thrown in the test body.

FAILURE: Should have continued with other tests, but did not.
→ nv at dev in /home/nv/src/googletest on git:main[!]
```

If I remove the `ThrowListener`, then the result is closer to expectation:

```
$ bazel run //googletest/test:gtest_assert_by_exception_test
INFO: Analyzed target //googletest/test:gtest_assert_by_exception_test (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
Target //googletest/test:gtest_assert_by_exception_test up-to-date:
bazel-bin/googletest/test/gtest_assert_by_exception_test
INFO: Elapsed time: 0.517s, Critical Path: 0.48s
INFO: 3 processes: 1 internal, 2 linux-sandbox.
INFO: Build completed successfully, 3 total actions
INFO: Running command line: external/bazel_tools/tools/test/test-setup.sh googletest/test/gtest_assert_by_exception_test
exec ${PAGER:-/usr/bin/less} "$0" || exit 1
Executing tests from //googletest/test:gtest_assert_by_exception_test
-----------------------------------------------------------------------------
[==========] Running 3 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 3 tests from Test
[ RUN ] Test.First
unknown file: Failure
C++ exception with description "an exception which should fail a single test" thrown in the test body.

[ FAILED ] Test.First (0 ms)
[ RUN ] Test.Test
googletest/test/gtest_assert_by_exception_test.cc:78: Failure
Expected equality of these values:
3
4

googletest/test/gtest_assert_by_exception_test.cc:60: Failure
Expected equality of these values:
2
3
Expected failure

FAILURE: A failed assertion should've thrown but didn't.
```

I believe the problem to be around here https://github.com/google/googletest/blob/c67de117379f4d1c889c7581a0a76aa0979c2083/googletest/src/gtest.cc#L2715 which causes the throw from an unexpected place aborting the whole thing.

The (ugly) workaround I found is to match on the failure message and not throw the AssertionException in that case. I.e.

```diff
--- i/googletest/test/gtest_assert_by_exception_test.cc
+++ w/googletest/test/gtest_assert_by_exception_test.cc
@@ -39,7 +39,8 @@

class ThrowListener : public testing::EmptyTestEventListener {
void OnTestPartResult(const testing::TestPartResult& result) override {
- if (result.type() == testing::TestPartResult::kFatalFailure) {
+ if (result.type() == testing::TestPartResult::kFatalFailure
+ && std::string_view(result.message()).find("C++ exception with description") == std::string_view::npos) {
throw testing::AssertionException(result);
}
}
```

Consider this as both a bug and a feature request. As I'm not sure this can be fixed out of the box without introducing new mechanisms. Btw, it might be interesting to expose a "hook" for letting the user C++ exceptions and building `TestPartResult` themselves. I.e. it might be possible to extract source locations from these exceptions.

### Steps to reproduce the problem

Apply the following change to main (i.e. at 3983f67e32fb3e9294487b9d4f9586efa6e5d088)

```diff
$ git diff -p --stat | cat -v
googletest/test/gtest_assert_by_exception_test.cc | 4 ++++
1 file changed, 4 insertions(+)

diff --git i/googletest/test/gtest_assert_by_exception_test.cc w/googletest/test/gtest_assert_by_exception_test.cc
index f507eac4..a1e642ca 100644
--- i/googletest/test/gtest_assert_by_exception_test.cc
+++ w/googletest/test/gtest_assert_by_exception_test.cc
@@ -57,6 +57,10 @@ void Fail(const char* msg) {

static void AssertFalse() { ASSERT_EQ(2, 3) << "Expected failure"; }

+TEST(Test, First) {
+ throw std::runtime_error("an exception which should fail a single test");
+}
+
// Tests that an assertion failure throws a subclass of
// std::runtime_error.
TEST(Test, Test) {
```

Run the test with

```sh
bazel test //googletest/test:gtest_assert_by_exception_test
```

Expect the test to succeed but instead it fails with the following incomplete output:

```
exec ${PAGER:-/usr/bin/less} "$0" || exit 1
Executing tests from //googletest/test:gtest_assert_by_exception_test
-----------------------------------------------------------------------------
[==========] Running 3 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 3 tests from Test
[ RUN ] Test.First
unknown file: Failure
C++ exception with description "an exception which should fail a single test" thrown in the test body.

FAILURE: Should have continued with other tests, but did not.
```

### What version of GoogleTest are you using?

3983f67e32fb3e9294487b9d4f9586efa6e5d088 (latest at the time of writing)

### What operating system and version are you using?

Linux dev 6.12.27+bpo-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.12.27-1~bpo12+1 (2025-05-19) x86_64 GNU/Linux

### What compiler and version are you using?

gcc (Debian 12.2.0-14+deb12u1) 12.2.0

### What build system are you using?

bazel 8.3.1

### Additional context

_No response_

Contributor guide

Open the contributing guide

Research direction

Start with googletest/test/gtest_assert_by_exception_test.cc and run bazel test //googletest/test:gtest_assert_by_exception_test using the reported added test case. Then read googletest/src/gtest.cc around the referenced line near 2715 to trace how exceptions from OnTestPartResult are handled. Done means the test-body exception is reported as one failed test, remaining tests continue, and the assertion-exception behavior still works.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.