google / google/googletest

The strange 'TEST EXECUTE FAILED' only on iOS

Open
#3,845 2 comments 0 reactions 0 assignees View on GitHub
bug More Information Requested
Dominant language
C++
Stars
39.5k
Forks
10.9k
Avg merge
6d 13h
Merged PRs (30d)
1

Description

In `license_test.cpp` only one test. And it looks lie this:
```cpp
TEST(Test, License) {
#if !defined(WITH_LICENSE)
GTEST_SKIP();
#else
checkLicense();
#endif
}
```
**In case, if the macro `WITH_LICENSE` is not defined.**

On Linux (android and windows), I am observing something like this:
```sh
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from Test
[ RUN ] Test.License
[ SKIPPED ] Test.License (0 ms)
[----------] 1 test from Test (0 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (0 ms total)
[ PASSED ] 0 tests.
[ SKIPPED ] 1 test, listed below:
[ SKIPPED ] Test.License
```

But on iOS (iOS 13.4 Deployment 11.0 / arm64) it falls each time with log:
```sh
Executed 1 test, with 0 failures (0 unexpected) in 0.017 (0.017) seconds
Test Suite 'License' started at 2022-05-16 11:44:35.799
Test Case '-[Test License]' started.
.../license_test.cpp:103: error: -[Test License] : Skipped
.../ios/GoogleTests.mm:158: error: -[Test License] : ((totalTestsRun) equal to (1)) failed: ("0") is not equal to ("1") - Expected to run a single test for filter "Test.License"
Test Case '-[Test License]' failed (0.002 seconds).
Test Suite 'Test' failed at 2022-05-16 11:44:35.802.
...
** TEST EXECUTE FAILED **
Testing started on 'iPhone X black'
```

**Compiler and build system information:**
```sh
-- [polly] Used toolchain: iOS 13.4 Deployment 11.0 / arm64 / clang / c++11 support
debug CMAKE_CURRENT_LIST_DIR = /Users/admin/dev/polly
-- The C compiler identification is AppleClang 11.0.3.11030032
-- The CXX compiler identification is AppleClang 11.0.3.11030032
```

**The version of googletest is - v1.10**

Contributor guide

Open the contributing guide

Research direction

Start with the skip path in license_test.cpp and the iOS test handling in ios/GoogleTests.mm, then reproduce the issue with the reported iOS 13.4 and GoogleTest v1.10 setup. Compare how the skipped test is counted on iOS versus the shown Linux output. Done means the license test can be skipped on iOS without producing TEST EXECUTE FAILED or an unexpected totalTestsRun failure.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, ios
Domain
mobile-dev, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.