bazelbuild / bazelbuild/rules_apple
ios_ui_test runner is flaky
- Dominant language
- Starlark
- Stars
- 593
- Forks
- 334
- Avg merge
- 16h 48m
- Merged PRs (30d)
- 9
Description
We're attempting to make use of `ios_ui_test` on the MDC iOS repo and encountering fairly flaky builds as a result.
The errors we're seeing are:
- "Timed out waiting 120 seconds for simulator to boot, current state is 1."
- "(ipc/mig) server died"
Example failure logs from [this PR](https://github.com/material-components/material-components-ios/pull/4182):
```
FAIL: //components/BottomSheet:app_tests (see /Volumes/BuildData/tmpfs/tmp/bazel/3eb3c42fcd20d50a831da151c89283a1/execroot/__main__/bazel-out/darwin-fastbuild/testlogs/components/BottomSheet/app_tests/test.log)
INFO: From Testing //components/BottomSheet:app_tests:
==================== Test output for //components/BottomSheet:app_tests:
2018-05-17 20:09:38,551 Will consider the test as test type xctest to run.
2018-05-17 20:09:38,680 The signing options only works on sdk iphoneos, but current sdk is iphonesimulator
2018-05-17 20:09:38,680 Generating xctestrun file.
2018-05-17 20:09:38,681 Generating dummy project.
2018-05-17 20:09:38,694 Failed to import biplist module. Will use tool /usr/libexec/PlistBuddy to handle the binary format plist.
2018-05-17 20:09:38,823 Dummy project is generated.
2018-05-17 20:09:38,824 Preparing build products directory /tmpfs/tmp/test_runner_work_dir.BsxgNY/TEST_ROOT for dummy project.
2018-05-17 20:09:38,825 Running `xcodebuild build-for-testing` with dummy project.
built_product_dir = /tmpfs/tmp/test_runner_work_dir.BsxgNY/TEST_ROOT
derived_data_path = /tmpfs/tmp/test_runner_work_dir.BsxgNY/dummyproject_derived_data
2018-05-17 20:09:42,480 Failed to import biplist module. Will use tool /usr/libexec/PlistBuddy to handle the binary format plist.
2018-05-17 20:09:42,494 Creating a new simulator:
Name: NEW_IPHONE_5_8_1
OS: iOS 8.1
Type: iPhone 5
2018-05-17 20:09:43,427 Created new simulator 4CC7829A-504A-44BC-AEDD-FF2CBB0671CD.
2018-05-17 20:09:43,427 Running test-without-building with device 4CC7829A-504A-44BC-AEDD-FF2CBB0671CD
User defaults from command line:
IDEDerivedDataPathOverride = /Volumes/BuildData/tmpfs/tmp/bazel/3eb3c42fcd20d50a831da151c89283a1/bazel-sandbox/8061246070028729815/execroot/__main__/bazel-out/darwin-fastbuild/bin/components/BottomSheet/app_tests.runfiles/__main__/Volumes/BuildData/tmpfs/tmp/bazel/3eb3c42fcd20d50a831da151c89283a1/bazel-sandbox/8061246070028729815/execroot/__main__/bazel-out/darwin-fastbuild/testlogs/components/BottomSheet/app_tests/test.outputs
IDETestRunSpecificationPath = /Volumes/BuildData/tmpfs/tmp/test_runner_work_dir.BsxgNY/TEST_ROOT/xctestrun.plist
2018-05-17 20:09:45.060 xcodebuild[22583:77096] IDETestOperationsObserverDebug: Writing diagnostic log for test session to:
/Volumes/BuildData/tmpfs/tmp/bazel/3eb3c42fcd20d50a831da151c89283a1/bazel-sandbox/8061246070028729815/execroot/__main__/bazel-out/darwin-fastbuild/bin/components/BottomSheet/app_tests.runfiles/__main__/Volumes/BuildData/tmpfs/tmp/bazel/3eb3c42fcd20d50a831da151c89283a1/bazel-sandbox/8061246070028729815/execroot/__main__/bazel-out/darwin-fastbuild/testlogs/components/BottomSheet/app_tests/test.outputs/Logs/Test/3BFD2A73-57BA-4E0F-A947-F991A82931B7/Session-app_tests-2018-05-17_200945-XNT4Yh.log
2018-05-17 20:09:45.060 xcodebuild[22583:77018] [MT] IDETestOperationsObserverDebug: (83847887-5130-4C8E-BE87-12079E1E327A) Beginning test session app_tests-83847887-5130-4C8E-BE87-12079E1E327A at 2018-05-17 20:09:45.060 with Xcode 8E3004b on target {
SimDevice: SimDevice : NEW_IPHONE_5_8_1 (4CC7829A-504A-44BC-AEDD-FF2CBB0671CD) : state={ Shutdown } deviceType={ SimDeviceType : com.apple.CoreSimulator.SimDeviceType.iPhone-5 } runtime={ SimRuntime : 8.1 (12B411) - com.apple.CoreSimulator.SimRuntime.iOS-8-1 }
} (8.1 (12B411))
2018-05-17 20:11:56.767 xcodebuild[22583:77018] [MT] IDETestOperationsObserverDebug: (83847887-5130-4C8E-BE87-12079E1E327A) Failed to make test runner session:
Error Domain=com.apple.dt.xctest.error Code=7 "Timed out waiting 120 seconds for simulator to boot, current state is 1." UserInfo={NSLocalizedDescription=Timed out waiting 120 seconds for simulator to boot, current state is 1.}
2018-05-17 20:11:56.771 xcodebuild[22583:77018] Error Domain=IDETestOperationsObserverErrorDomain Code=3 "Timed out waiting 120 seconds for simulator to boot, current state is 1. If you believe this error represents a bug, please attach the log file at /Volumes/BuildData/tmpfs/tmp/bazel/3eb3c42fcd20d50a831da151c89283a1/bazel-sandbox/8061246070028729815/execroot/__main__/bazel-out/darwin-fastbuild/bin/components/BottomSheet/app_tests.runfiles/__main__/Volumes/BuildData/tmpfs/tmp/bazel/3eb3c42fcd20d50a831da151c89283a1/bazel-sandbox/8061246070028729815/execroot/__main__/bazel-out/darwin-fastbuild/testlogs/components/BottomSheet/app_tests/test.outputs/Logs/Test/3BFD2A73-57BA-4E0F-A947-F991A82931B7/Session-app_tests-2018-05-17_200945-XNT4Yh.log" UserInfo={NSLocalizedDescription=Timed out waiting 120 seconds for simulator to boot, current state is 1. If you believe this error represents a bug, please attach the log file at /Volumes/BuildData/tmpfs/tmp/bazel/3eb3c42fcd20d50a831da151c89283a1/bazel-sandbox/8061246070028729815/execroot/__main__/bazel-out/darwin-fastbuild/bin/components/BottomSheet/app_tests.runfiles/__main__/Volumes/BuildData/tmpfs/tmp/bazel/3eb3c42fcd20d50a831da151c89283a1/bazel-sandbox/8061246070028729815/execroot/__main__/bazel-out/darwin-fastbuild/testlogs/components/BottomSheet/app_tests/test.outputs/Logs/Test/3BFD2A73-57BA-4E0F-A947-F991A82931B7/Session-app_tests-2018-05-17_200945-XNT4Yh.log}
2018-05-17 20:12:13,601 The xcodebuild command got stuck and has not started test in 150. Will kill the command directly.
** BUILD INTERRUPTED **
2018-05-17 20:12:13,622 xcodebuild command can not launch test on device/simulator in 150s.
2018-05-17 20:12:13,628 Failed to parse test summaries /Volumes/BuildData/tmpfs/tmp/bazel/3eb3c42fcd20d50a831da151c89283a1/bazel-sandbox/8061246070028729815/execroot/__main__/bazel-out/darwin-fastbuild/bin/components/BottomSheet/app_tests.runfiles/__main__//Volumes/BuildData/tmpfs/tmp/bazel/3eb3c42fcd20d50a831da151c89283a1/bazel-sandbox/8061246070028729815/execroot/__main__/bazel-out/darwin-fastbuild/testlogs/components/BottomSheet/app_tests/test.outputs/Logs/Test/3BFD2A73-57BA-4E0F-A947-F991A82931B7_TestSummaries.plist: The field TestableSummaries:0:Tests:0 can not be found in the target object. The object content is []
2018-05-17 20:12:13,651 Simulator 4CC7829A-504A-44BC-AEDD-FF2CBB0671CD has already shut down.
2018-05-17 20:12:14,116 Deleted simulator 4CC7829A-504A-44BC-AEDD-FF2CBB0671CD.
2018-05-17 20:12:14,116 Done.
================================================================================
```
Contributor guide
Assessment
This issue has not been assessed yet.