flutter / flutter/devtools

`eval_and_inspect_test` running on web device failing on bots

Open
#7,145 0 comments 0 reactions 0 assignees View on GitHub
integration tests P2 testing
Dominant language
Dart
Stars
1.7k
Forks
404
Avg merge
6d 17h
Merged PRs (30d)
18

Description

On web, it passes but then hangs indefinitely and requires `ctrl+c` to quit the test:

On the CI bots, it fails: https://github.com/flutter/devtools/actions/runs/7730688845/job/21076721585?pr=7144

Logs when running locally

```
dart run integration_test/run_tests.dart --target=integration_test/test/live_connection/eval_and_inspect_test.dart --test-app-device=chrome
ChromeDriver - Starting ChromeDriver 120.0.6099.109 (3419140ab665596f21b385ce136419fde0924272-refs/branch-heads/6099@{#1483}) on port 4444
ChromeDriver - Only local connections are allowed.
ChromeDriver - Please see https://chromedriver.chromium.org/security-considerations for suggestions on keeping ChromeDriver safe.
ChromeDriver - ChromeDriver was started successfully.
FlutterDriveProcess - Resolving dependencies...
FlutterDriveProcess - _fe_analyzer_shared 61.0.0 (67.0.0 available)
FlutterDriveProcess - analyzer 5.13.0 (6.4.1 available)
FlutterDriveProcess - archive 3.4.9 (3.4.10 available)
FlutterDriveProcess - build_resolvers 2.4.1 (2.4.2 available)
FlutterDriveProcess - build_runner 2.4.7 (2.4.8 available)
FlutterDriveProcess - build_runner_core 7.2.11 (7.3.0 available)
FlutterDriveProcess - built_value 8.8.0 (8.9.0 available)
FlutterDriveProcess - code_builder 4.8.0 (4.10.0 available)
FlutterDriveProcess - coverage 1.7.1 (1.7.2 available)
FlutterDriveProcess - dart_style 2.3.2 (2.3.4 available)
FlutterDriveProcess - dds_service_extensions 1.6.2 (1.7.0 available)
FlutterDriveProcess - ! devtools_app_shared 0.0.9 from path ../devtools_app_shared (overridden)
FlutterDriveProcess - ! devtools_extensions 0.0.13 from path ../devtools_extensions (overridden)
FlutterDriveProcess - ! devtools_shared 6.0.4 from path ../devtools_shared (overridden)
FlutterDriveProcess - ! devtools_test 2.32.0-dev.8 from path ../devtools_test (overridden)
FlutterDriveProcess - ffi 2.1.0 (2.1.2 available)
FlutterDriveProcess - file_selector 1.0.1 (1.0.2 available)
FlutterDriveProcess - file_selector_android 0.5.0+4 (0.5.0+7 available)
FlutterDriveProcess - file_selector_ios 0.5.1+7 (0.5.1+8 available)
FlutterDriveProcess - file_selector_platform_interface 2.6.1 (2.6.2 available)
FlutterDriveProcess - flutter_lints 2.0.3 (3.0.1 available)
FlutterDriveProcess - flutter_markdown 0.6.18+2 (0.6.18+3 available)
FlutterDriveProcess - flutter_riverpod 2.0.0-dev.9 (2.4.9 available)
FlutterDriveProcess - http 1.1.2 (1.2.0 available)
FlutterDriveProcess - image 4.1.3 (4.1.4 available)
FlutterDriveProcess - intl 0.18.1 (0.19.0 available)
FlutterDriveProcess - js 0.6.7 (0.7.0 available)
FlutterDriveProcess - lints 2.1.1 (3.0.0 available)
FlutterDriveProcess - markdown 7.1.1 (7.2.1 available)
FlutterDriveProcess - mime 1.0.4 (1.0.5 available)
FlutterDriveProcess - mockito 5.4.3 (5.4.4 available)
FlutterDriveProcess - path_provider_platform_interface 2.1.1 (2.1.2 available)
FlutterDriveProcess - plugin_platform_interface 2.1.7 (2.1.8 available)
FlutterDriveProcess - pointer_interceptor 0.9.3+7 (0.10.1 available)
FlutterDriveProcess - pointycastle 3.7.3 (3.7.4 available)
FlutterDriveProcess - riverpod 2.0.0-dev.9 (2.4.9 available)
FlutterDriveProcess - shared_preferences_foundation 2.3.4 (2.3.5 available)
FlutterDriveProcess - shared_preferences_platform_interface 2.3.1 (2.3.2 available)
FlutterDriveProcess - source_gen 1.4.0 (1.5.0 available)
FlutterDriveProcess - state_notifier 0.7.2+1 (1.0.0 available)
FlutterDriveProcess - test 1.24.9 (1.25.2 available)
FlutterDriveProcess - test_api 0.6.1 (0.7.0 available)
FlutterDriveProcess - test_core 0.5.9 (0.6.0 available)
FlutterDriveProcess - url_launcher 6.2.1 (6.2.4 available)
FlutterDriveProcess - url_launcher_android 6.2.0 (6.2.2 available)
FlutterDriveProcess - url_launcher_ios 6.2.1 (6.2.4 available)
FlutterDriveProcess - url_launcher_linux 3.1.0 (3.1.1 available)
FlutterDriveProcess - url_launcher_platform_interface 2.2.0 (2.3.1 available)
FlutterDriveProcess - url_launcher_web 2.2.1 (2.2.3 available)
FlutterDriveProcess - url_launcher_windows 3.1.0 (3.1.1 available)
FlutterDriveProcess - vm_service 13.0.0 (14.0.0 available)
FlutterDriveProcess - vm_snapshot_analysis 0.7.2 (0.7.6 available)
FlutterDriveProcess - web_socket_channel 2.4.0 (2.4.3 available)
FlutterDriveProcess - win32 5.1.0 (5.2.0 available)
FlutterDriveProcess - xdg_directories 1.0.3 (1.0.4 available)
FlutterDriveProcess - xml 6.4.2 (6.5.0 available)
FlutterDriveProcess - Got dependencies!
FlutterDriveProcess - 52 packages have newer versions incompatible with dependency constraints.
FlutterDriveProcess - Try `flutter pub outdated` for more information.
FlutterDriveProcess - Launching integration_test/test/live_connection/eval_and_inspect_test.dart on Chrome in debug mode...
FlutterDriveProcess - Waiting for connection from debug service on Chrome... 36.9s
FlutterDriveProcess - This app is linked to the debug service: ws://127.0.0.1:61690/nYoYCJ_UnPI=/ws
FlutterDriveProcess - Debug service listening on ws://127.0.0.1:61690/nYoYCJ_UnPI=/ws
FlutterDriveProcess - 00:00 +0: (setUpAll)
FlutterDriveProcess - 00:00 +1: eval with scope in inspector window
ChromeDriver - [1706729325.630][WARNING]: This version of ChromeDriver has not been tested with Chrome version 121.
FlutterDriveProcess - devtools server not available (200)
FlutterDriveProcess - DevTools version 2.32.0-dev.8.
FlutterDriveProcess - No DTD uri provided from the server during initialization.
FlutterDriveProcess - TEST STATUS: verify that we can connect to an app
FlutterDriveProcess - Attempted to call extension 'ext.flutter.inspector.structuredErrors', but no service with that name exists
FlutterDriveProcess - getSourceReport: (-32602) Source report kind Coverage not supported
FlutterDriveProcess - TEST STATUS: switching to inspector screen (icon IconData(U+0F12B))
FlutterDriveProcess - TEST STATUS: testing basic evaluation
FlutterDriveProcess - TEST STATUS:
FlutterDriveProcess - attempt #0, tapping
FlutterDriveProcess - TEST STATUS: Found 1 widget with type "AutoCompleteSearchField": [
FlutterDriveProcess - AutoCompleteSearchField(state: _AutoCompleteSearchFieldState#171b3),
FlutterDriveProcess - ]
FlutterDriveProcess - TEST STATUS: testing variable assignment
FlutterDriveProcess - TEST STATUS:
FlutterDriveProcess - attempt #0, tapping
FlutterDriveProcess - TEST STATUS: Found 1 widget with type "AutoCompleteSearchField": [
FlutterDriveProcess - AutoCompleteSearchField(state: _AutoCompleteSearchFieldState#8fb6d),
FlutterDriveProcess - ]
FlutterDriveProcess - TEST STATUS:
FlutterDriveProcess - attempt #0, tapping
FlutterDriveProcess - TEST STATUS: Found 1 widget with type "AutoCompleteSearchField": [
FlutterDriveProcess - AutoCompleteSearchField(state: _AutoCompleteSearchFieldState#3c3a9),
FlutterDriveProcess - ]
FlutterDriveProcess - TEST STATUS:
FlutterDriveProcess - attempt #0, tapping
FlutterDriveProcess - TEST STATUS: Found 1 widget with type "AutoCompleteSearchField": [
FlutterDriveProcess - AutoCompleteSearchField(state: _AutoCompleteSearchFieldState#e44cc),
FlutterDriveProcess - ]
FlutterDriveProcess - TEST STATUS: testing eval on widget tree node
FlutterDriveProcess - TEST STATUS:
FlutterDriveProcess - attempt #0, tapping
FlutterDriveProcess - TEST STATUS: Found 1 widget with widget matching predicate descending from widget with key [<'Summary Tree'>]: [
FlutterDriveProcess - RichText(softWrap: wrapping at box width, overflow: ellipsis, maxLines: unlimited, text: "FloatingActionButton", dependencies: [Directionality, _LocalizationsScope-[GlobalKey#35bb9]], renderObject: RenderParagraph#5c590 relayoutBoundary=up2),
FlutterDriveProcess - ]
FlutterDriveProcess - TEST STATUS:
FlutterDriveProcess - attempt #0, tapping
FlutterDriveProcess - TEST STATUS: Found 1 widget with type "AutoCompleteSearchField": [
FlutterDriveProcess - AutoCompleteSearchField(state: _AutoCompleteSearchFieldState#c419f),
FlutterDriveProcess - ]
FlutterDriveProcess - 02:04 +2: (tearDownAll)
FlutterDriveProcess - 02:04 +3: All tests passed!
FlutterDriveProcess - result {"result":"true","failureDetails":[]}
FlutterDriveProcess - All tests passed.
FlutterDriveProcess - Application finished.
```

Logs when running on CI bot

```
FlutterDriveProcess - Launching integration_test/test/live_connection/eval_and_inspect_test.dart on Web Server in debug mode...
FlutterDriveProcess - Waiting for connection from debug service on Web Server... 78.8s
FlutterDriveProcess - integration_test/test/live_connection/eval_and_inspect_test.dart is being served at http://localhost:49649
FlutterDriveProcess - The web-server device requires the Dart Debug Chrome extension for debugging. Consider using the Chrome or Edge devices for an improved development workflow.
FlutterDriveProcess - result {"result":"false","failureDetails":["{\"methodName\":\"eval and browse in inspector window\",\"details\":\"══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞═════════════════\\nThe following TestFailure was thrown running a test:\\nExpected: exactly one matching candidate\\n Actual: _TextContainingWidgetFinder:\\n Which: means none were found but one was expected\\n\\nWhen the exception was thrown, this was the stack:\\ndart-sdk/lib/_internal/js_dev_runtime/private/ddc_runtime/errors.dart 328:3 throw_\\npackages/matcher/src/expect/prints_matcher.dart.js 464:22 fail\\npackages/matcher/src/expect/prints_matcher.dart.js 461:12 _expect\\npackages/matcher/src/expect/prints_matcher.dart.js 396:12 expect$\\npackages/flutter_test/src/test_text_input_key_handler.dart.js 10436:12 expect$\\neval_utils.dart.js 89:23 testEval\\ndart-sdk/lib/_internal/js_dev_runtime/patch/async_patch.dart 45:50 \\ndart-sdk/lib/async/zone.dart 1407:47 _rootRunUnary\\ndart-sdk/lib/async/zone.dart 1308:19 runUnary\\ndart-sdk/lib/async/future_impl.dart 162:18 handleValue\\ndart-sdk/lib/async/future_impl.dart 838:44 handleValueCallback\\ndart-sdk/lib/async/future_impl.dart 867:13 _propagateToListeners\\ndart-sdk/lib/async/future_impl.dart 643:5 [_completeWithValue]\\ndart-sdk/lib/async/future_impl.dart 713:7 \\ndart-sdk/lib/async/zone.dart 1399:13 _rootRun\\ndart-sdk/lib/async/zone.dart 1301:19 run\\ndart-sdk/lib/async/zone.dart 1209:7 runGuarded\\ndart-sdk/lib/async/zone.dart 1249:23 callback\\ndart-sdk/lib/async/schedule_microtask.dart 40:11 _microtaskLoop\\ndart-sdk/lib/async/schedule_microtask.dart 49:5 _startMicrotaskLoop\\ndart-sdk/lib/_internal/js_dev_runtime/patch/async_patch.dart 181:7 \\n\\nThe test description was:\\n eval and browse in inspector window\\n═════════════════════════════════════════════════════════════════\\n\"}"]}
FlutterDriveProcess - Failure Details:
FlutterDriveProcess - Failure in method: eval and browse in inspector window
FlutterDriveProcess - ══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞═════════════════
FlutterDriveProcess - The following TestFailure was thrown running a test:
FlutterDriveProcess - Expected: exactly one matching candidate
FlutterDriveProcess - Actual: _TextContainingWidgetFinder:
FlutterDriveProcess - Which: means none were found but one was expected
FlutterDriveProcess -
FlutterDriveProcess - When the exception was thrown, this was the stack:
FlutterDriveProcess - dart-sdk/lib/_internal/js_dev_runtime/private/ddc_runtime/errors.dart 328:3 throw_
FlutterDriveProcess - packages/matcher/src/expect/prints_matcher.dart.js 464:22 fail
FlutterDriveProcess - packages/matcher/src/expect/prints_matcher.dart.js 461:12 _expect
FlutterDriveProcess - packages/matcher/src/expect/prints_matcher.dart.js 396:12 expect$
FlutterDriveProcess - packages/flutter_test/src/test_text_input_key_handler.dart.js 10436:12 expect$
FlutterDriveProcess - eval_utils.dart.js 89:23 testEval
FlutterDriveProcess - dart-sdk/lib/_internal/js_dev_runtime/patch/async_patch.dart 45:50
FlutterDriveProcess - dart-sdk/lib/async/zone.dart 1407:47 _rootRunUnary
FlutterDriveProcess - dart-sdk/lib/async/zone.dart 1308:19 runUnary
FlutterDriveProcess - dart-sdk/lib/async/future_impl.dart 162:18 handleValue
FlutterDriveProcess - dart-sdk/lib/async/future_impl.dart 838:44 handleValueCallback
FlutterDriveProcess - dart-sdk/lib/async/future_impl.dart 867:13 _propagateToListeners
FlutterDriveProcess - dart-sdk/lib/async/future_impl.dart 643:5 [_completeWithValue]
FlutterDriveProcess - dart-sdk/lib/async/future_impl.dart 713:7
FlutterDriveProcess - dart-sdk/lib/async/zone.dart 1399:13 _rootRun
FlutterDriveProcess - dart-sdk/lib/async/zone.dart 1301:19 run
FlutterDriveProcess - dart-sdk/lib/async/zone.dart 1209:7 runGuarded
FlutterDriveProcess - dart-sdk/lib/async/zone.dart 1249:23 callback
FlutterDriveProcess - dart-sdk/lib/async/schedule_microtask.dart 40:11 _microtaskLoop
FlutterDriveProcess - dart-sdk/lib/async/schedule_microtask.dart 49:5 _startMicrotaskLoop
FlutterDriveProcess - dart-sdk/lib/_internal/js_dev_runtime/patch/async_patch.dart 181:7
FlutterDriveProcess -
FlutterDriveProcess - The test description was:
FlutterDriveProcess - eval and browse in inspector window
FlutterDriveProcess - ═════════════════════════════════════════════════════════════════
FlutterDriveProcess -
FlutterDriveProcess - end of failure 1
FlutterDriveProcess -
FlutterDriveProcess -
FlutterDriveProcess -
FlutterDriveProcess - Application finished.
Unhandled exception:
Exception: Test 'eval and browse in inspector window' failed: ══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞═════════════════
The following TestFailure was thrown running a test:
Expected: exactly one matching candidate
Actual: _TextContainingWidgetFinder:
Which: means none were found but one was expected

When the exception was thrown, this was the stack:
dart-sdk/lib/_internal/js_dev_runtime/private/ddc_runtime/errors.dart 328:3 throw_
packages/matcher/src/expect/prints_matcher.dart.js 464:22 fail
packages/matcher/src/expect/prints_matcher.dart.js 461:12 _expect
packages/matcher/src/expect/prints_matcher.dart.js 396:12 expect$
packages/flutter_test/src/test_text_input_key_handler.dart.js 10436:12 expect$
eval_utils.dart.js 89:23 testEval
dart-sdk/lib/_internal/js_dev_runtime/patch/async_patch.dart 45:50
dart-sdk/lib/async/zone.dart 1407:47 _rootRunUnary
dart-sdk/lib/async/zone.dart 1308:19 runUnary
dart-sdk/lib/async/future_impl.dart 162:18 handleValue
dart-sdk/lib/async/future_impl.dart 838:44 handleValueCallback
dart-sdk/lib/async/future_impl.dart 867:13 _propagateToListeners
dart-sdk/lib/async/future_impl.dart 643:5 [_completeWithValue]
dart-sdk/lib/async/future_impl.dart 713:7
dart-sdk/lib/async/zone.dart 1399:13 _rootRun
dart-sdk/lib/async/zone.dart 1301:19 run
dart-sdk/lib/async/zone.dart 1209:7 runGuarded
dart-sdk/lib/async/zone.dart 1249:23 callback
dart-sdk/lib/async/schedule_microtask.dart 40:11 _microtaskLoop
dart-sdk/lib/async/schedule_microtask.dart 49:5 _startMicrotaskLoop
dart-sdk/lib/_internal/js_dev_runtime/patch/async_patch.dart 181:7

The test description was:
eval and browse in inspector window
═════════════════════════════════════════════════════════════════
.

result {"result":"false","failureDetails":["{\"methodName\":\"eval and browse in inspector window\",\"details\":\"══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞═════════════════\\nThe following TestFailure was thrown running a test:\\nExpected: exactly one matching candidate\\n Actual: _TextContainingWidgetFinder:\\n Which: means none were found but one was expected\\n\\nWhen the exception was thrown, this was the stack:\\ndart-sdk/lib/_internal/js_dev_runtime/private/ddc_runtime/errors.dart 328:3 throw_\\npackages/matcher/src/expect/prints_matcher.dart.js 464:22 fail\\npackages/matcher/src/expect/prints_matcher.dart.js 461:12 _expect\\npackages/matcher/src/expect/prints_matcher.dart.js 396:12 expect$\\npackages/flutter_test/src/test_text_input_key_handler.dart.js 10436:12 expect$\\neval_utils.dart.js 89:23 testEval\\ndart-sdk/lib/_internal/js_dev_runtime/patch/async_patch.dart 45:50 \\ndart-sdk/lib/async/zone.dart 1407:47 _rootRunUnary\\ndart-sdk/lib/async/zone.dart 1308:19 runUnary\\ndart-sdk/lib/async/future_impl.dart 162:18 handleValue\\ndart-sdk/lib/async/future_impl.dart 838:44 handleValueCallback\\ndart-sdk/lib/async/future_impl.dart 867:13 _propagateToListeners\\ndart-sdk/lib/async/future_impl.dart 643:5 [_completeWithValue]\\ndart-sdk/lib/async/future_impl.dart 713:7 \\ndart-sdk/lib/async/zone.dart 1399:13 _rootRun\\ndart-sdk/lib/async/zone.dart 1301:19 run\\ndart-sdk/lib/async/zone.dart 1209:7 runGuarded\\ndart-sdk/lib/async/zone.dart 1249:23 callback\\ndart-sdk/lib/async/schedule_microtask.dart 40:11 _microtaskLoop\\ndart-sdk/lib/async/schedule_microtask.dart 49:5 _startMicrotaskLoop\\ndart-sdk/lib/_internal/js_dev_runtime/patch/async_patch.dart 181:7 \\n\\nThe test description was:\\n eval and browse in inspector window\\n═════════════════════════════════════════════════════════════════\\n\"}"]}
Failure Details:
Failure in method: eval and browse in inspector window
══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞═════════════════
The following TestFailure was thrown running a test:
Expected: exactly one matching candidate
Actual: _TextContainingWidgetFinder:
Which: means none were found but one was expected

When the exception was thrown, this was the stack:
dart-sdk/lib/_internal/js_dev_runtime/private/ddc_runtime/errors.dart 328:3 throw_
packages/matcher/src/expect/prints_matcher.dart.js 464:22 fail
packages/matcher/src/expect/prints_matcher.dart.js 461:12 _expect
packages/matcher/src/expect/prints_matcher.dart.js 396:12 expect$
packages/flutter_test/src/test_text_input_key_handler.dart.js 10436:12 expect$
eval_utils.dart.js 89:23 testEval
dart-sdk/lib/_internal/js_dev_runtime/patch/async_patch.dart 45:50
dart-sdk/lib/async/zone.dart 1407:47 _rootRunUnary
dart-sdk/lib/async/zone.dart 1308:19 runUnary
dart-sdk/lib/async/future_impl.dart 162:18 handleValue
dart-sdk/lib/async/future_impl.dart 838:44 handleValueCallback
dart-sdk/lib/async/future_impl.dart 867:13 _propagateToListeners
dart-sdk/lib/async/future_impl.dart 643:5 [_completeWithValue]
dart-sdk/lib/async/future_impl.dart 713:7
dart-sdk/lib/async/zone.dart 1399:13 _rootRun
dart-sdk/lib/async/zone.dart 1301:19 run
dart-sdk/lib/async/zone.dart 1209:7 runGuarded
dart-sdk/lib/async/zone.dart 1249:23 callback
dart-sdk/lib/async/schedule_microtask.dart 40:11 _microtaskLoop
dart-sdk/lib/async/schedule_microtask.dart 49:5 _startMicrotaskLoop
dart-sdk/lib/_internal/js_dev_runtime/patch/async_patch.dart 181:7

The test description was:
eval and browse in inspector window
═════════════════════════════════════════════════════════════════

#0 IntegrationTestRunner.run.runTest (package:devtools_shared/src/test/integration_test_runner.dart:138:11)

#1 IntegrationTestRunner.run (package:devtools_shared/src/test/integration_test_runner.dart:143:5)

#2 runFlutterIntegrationTest (file:///Users/runner/work/devtools/devtools/packages/devtools_app/integration_test/test_infra/run/run_test.dart:66:5)

#3 _runTest (file:///Users/runner/work/devtools/devtools/packages/devtools_app/integration_test/run_tests.dart:61:3)

#4 runOneOrManyTests (package:devtools_shared/src/test/integration_test_runner.dart:330:9)

#5 main (file:///Users/runner/work/devtools/devtools/packages/devtools_app/integration_test/run_tests.dart:37:3)

Error: Process completed with exit code 255.
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.