The strange 'TEST EXECUTE FAILED' only on iOS
- Dominant language
- C++
- Stars
- 39.5k
- Forks
- 10.9k
- Avg merge
- 6d 13h
- Merged PRs (30d)
- 1
Description
In `license_test.cpp` only one test. And it looks lie this:
```cpp
TEST(Test, License) {
#if !defined(WITH_LICENSE)
GTEST_SKIP();
#else
checkLicense();
#endif
}
```
**In case, if the macro `WITH_LICENSE` is not defined.**
On Linux (android and windows), I am observing something like this:
```sh
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from Test
[ RUN ] Test.License
[ SKIPPED ] Test.License (0 ms)
[----------] 1 test from Test (0 ms total)
[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (0 ms total)
[ PASSED ] 0 tests.
[ SKIPPED ] 1 test, listed below:
[ SKIPPED ] Test.License
```
But on iOS (iOS 13.4 Deployment 11.0 / arm64) it falls each time with log:
```sh
Executed 1 test, with 0 failures (0 unexpected) in 0.017 (0.017) seconds
Test Suite 'License' started at 2022-05-16 11:44:35.799
Test Case '-[Test License]' started.
.../license_test.cpp:103: error: -[Test License] : Skipped
.../ios/GoogleTests.mm:158: error: -[Test License] : ((totalTestsRun) equal to (1)) failed: ("0") is not equal to ("1") - Expected to run a single test for filter "Test.License"
Test Case '-[Test License]' failed (0.002 seconds).
Test Suite 'Test' failed at 2022-05-16 11:44:35.802.
...
** TEST EXECUTE FAILED **
Testing started on 'iPhone X black'
```
**Compiler and build system information:**
```sh
-- [polly] Used toolchain: iOS 13.4 Deployment 11.0 / arm64 / clang / c++11 support
debug CMAKE_CURRENT_LIST_DIR = /Users/admin/dev/polly
-- The C compiler identification is AppleClang 11.0.3.11030032
-- The CXX compiler identification is AppleClang 11.0.3.11030032
```
**The version of googletest is - v1.10**
Contributor guide
Research direction
Start with the skip path in license_test.cpp and the iOS test handling in ios/GoogleTests.mm, then reproduce the issue with the reported iOS 13.4 and GoogleTest v1.10 setup. Compare how the skipped test is counted on iOS versus the shown Linux output. Done means the license test can be skipped on iOS without producing TEST EXECUTE FAILED or an unexpected totalTestsRun failure.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, ios
- Domain
- mobile-dev, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100