flutter / flutter/devtools

DevTools integration test flakes

Open
#9,474 2 comments 0 reactions 0 assignees View on GitHub
integration tests P2
Dominant language
Dart
Stars
1.7k
Forks
404
Avg merge
6d 17h
Merged PRs (30d)
18

Description

The DevTools integration tests have been getting flaky.

devtools_extensions_test.dart

https://github.com/flutter/devtools/actions/runs/18566177330/job/52927948189

```
FlutterDriveProcess - Launching integration_test/test/live_connection/devtools_extensions_test.dart on Web Server in debug mode...
FlutterDriveProcess - Waiting for connection from debug service on Web Server... 67.1s
FlutterDriveProcess - integration_test/test/live_connection/devtools_extensions_test.dart is being served at http://localhost:49514
FlutterDriveProcess - The web-server device requires the Dart Debug Chrome extension for debugging. Consider using the Chrome or Edge devices for an improved development workflow.
ChromeDriver - [1760628606.940][SEVERE]: Timed out receiving message from renderer: 298.218
ChromeDriver - [1760628606.968][SEVERE]: Timed out receiving message from renderer: 298.218
FlutterDriveProcess - Unhandled exception:
FlutterDriveProcess - TimeoutException (500): timeout: Timed out receiving message from renderer: 298.218
FlutterDriveProcess - (Session info: chrome=140.0.7339.207)
FlutterDriveProcess - #0 parseW3cResponse (package:webdriver/src/handler/w3c/utils.dart:87:9)
FlutterDriveProcess - #1 W3cNavigationHandler.parseNavigateToResponse (package:webdriver/src/handler/w3c/navigation.dart:26:5)
FlutterDriveProcess - #2 AsyncRequestClient.send (package:webdriver/src/common/request_client.dart:96:32)
FlutterDriveProcess -
FlutterDriveProcess - #3 FlutterWebConnection.connect (package:flutter_driver/src/driver/web_driver.dart:301:5)
FlutterDriveProcess -
FlutterDriveProcess - #4 WebFlutterDriver.connectWeb (package:flutter_driver/src/driver/web_driver.dart:99:45)
FlutterDriveProcess -
FlutterDriveProcess - #5 main (file:///Users/runner/work/devtools/devtools/packages/devtools_app/test_driver/integration_test.dart:21:18)
FlutterDriveProcess -
FlutterDriveProcess - Application finished.
Unhandled exception:
Exception: Unhandled exception:
TimeoutException (500): timeout: Timed out receiving message from renderer: 298.218
(Session info: chrome=140.0.7339.207)
#0 parseW3cResponse (package:webdriver/src/handler/w3c/utils.dart:87:9)
#1 W3cNavigationHandler.parseNavigateToResponse (package:webdriver/src/handler/w3c/navigation.dart:26:5)
#2 AsyncRequestClient.send (package:webdriver/src/common/request_client.dart:96:32)

#3 FlutterWebConnection.connect (package:flutter_driver/src/driver/web_driver.dart:301:5)

#4 WebFlutterDriver.connectWeb (package:flutter_driver/src/driver/web_driver.dart:99:45)

#5 main (file:///Users/runner/work/devtools/devtools/packages/devtools_app/test_driver/integration_test.dart:21:18)

#0 IntegrationTestRunner.run.runTest (package:devtools_shared/src/test/integration_test_runner.dart:154:11)

#1 IntegrationTestRunner.run (package:devtools_shared/src/test/integration_test_runner.dart:160:5)

#2 runFlutterIntegrationTest (file:///Users/runner/work/devtools/devtools/packages/devtools_app/integration_test/test_infra/run/run_test.dart:121:5)

#3 _runTest (file:///Users/runner/work/devtools/devtools/packages/devtools_app/integration_test/run_tests.dart:84:3)

#4 runOneOrManyTests (package:devtools_shared/src/test/integration_test_runner.dart:354:9)

#5 main (file:///Users/runner/work/devtools/devtools/packages/devtools_app/integration_test/run_tests.dart:53:3)

Error: Process completed with exit code 255.
```

network_screen_test.dart

https://github.com/flutter/devtools/actions/runs/18568989429/job/52937864918

```
FlutterDriveProcess - Launching integration_test/test/live_connection/network_screen_test.dart on Web Server in debug mode...
FlutterDriveProcess - Waiting for connection from debug service on Web Server... 50.8s
FlutterDriveProcess - integration_test/test/live_connection/network_screen_test.dart is being served at http://localhost:49478
FlutterDriveProcess - The web-server device requires the Dart Debug Chrome extension for debugging. Consider using the Chrome or Edge devices for an improved development workflow.
FlutterDriveProcess - Debug service listening on ws://127.0.0.1:44456/vNmLRKxvmGU=
FlutterDriveProcess - A Dart VM Service on Web Server is available at: http://127.0.0.1:44456/vNmLRKxvmGU=
ChromeDriver - [1760634965.294][SEVERE]: Timed out receiving message from renderer: 30.000
FlutterDriveProcess - Unhandled exception:
FlutterDriveProcess - DriverError: Driver extension has not been initialized correctly.
FlutterDriveProcess - If the test uses a custom VM service extension, make sure it conforms to the protocol used by package:integration_test and package:flutter_driver.
FlutterDriveProcess - If the test uses VM service extensions provided by the Flutter SDK, then this error is likely caused by a bug in Flutter. Please report it by filing a bug on GitHub:
FlutterDriveProcess - https://github.com/flutter/flutter/issues/new?template=02_bug.yml
FlutterDriveProcess - Original error: ScriptTimeoutException (500): script timeout
FlutterDriveProcess - (Session info: chrome=140.0.7339.207)
FlutterDriveProcess - Original stack trace:
FlutterDriveProcess - #0 parseW3cResponse (package:webdriver/src/handler/w3c/utils.dart:78:9)
FlutterDriveProcess - #1 W3cCoreHandler.parseExecuteResponse (package:webdriver/src/handler/w3c/core.dart:59:19)
FlutterDriveProcess - #2 WebDriver.execute. (package:webdriver/src/async/web_driver.dart:264:37)
FlutterDriveProcess - #3 AsyncRequestClient.send (package:webdriver/src/common/request_client.dart:96:32)
FlutterDriveProcess -
FlutterDriveProcess - #4 FlutterWebConnection.sendCommand (package:flutter_driver/src/driver/web_driver.dart:316:7)
FlutterDriveProcess -
FlutterDriveProcess - #5 WebFlutterDriver.sendCommand (package:flutter_driver/src/driver/web_driver.dart:126:14)
FlutterDriveProcess -
FlutterDriveProcess - #6 FlutterDriver.requestData (package:flutter_driver/src/driver/driver.dart:573:7)
FlutterDriveProcess -
FlutterDriveProcess - #7 integrationDriver (package:integration_test/integration_test_driver_extended.dart:87:25)
FlutterDriveProcess -
FlutterDriveProcess - #8 main (file:///Users/runner/work/devtools/devtools/packages/devtools_app/test_driver/integration_test.dart:22:3)
FlutterDriveProcess -
FlutterDriveProcess -
FlutterDriveProcess -
FlutterDriveProcess - #0 FlutterWebConnection.sendCommand (package:flutter_driver/src/driver/web_driver.dart:318:7)
FlutterDriveProcess -
FlutterDriveProcess - #1 WebFlutterDriver.sendCommand (package:flutter_driver/src/driver/web_driver.dart:126:14)
FlutterDriveProcess -
FlutterDriveProcess - #2 FlutterDriver.requestData (package:flutter_driver/src/driver/driver.dart:573:7)
FlutterDriveProcess -
FlutterDriveProcess - #3 integrationDriver (package:integration_test/integration_test_driver_extended.dart:87:25)
FlutterDriveProcess -
FlutterDriveProcess - #4 main (file:///Users/runner/work/devtools/devtools/packages/devtools_app/test_driver/integration_test.dart:22:3)
FlutterDriveProcess -
FlutterDriveProcess - Application finished.
Unhandled exception:
Exception: Unhandled exception:
DriverError: Driver extension has not been initialized correctly.
If the test uses a custom VM service extension, make sure it conforms to the protocol used by package:integration_test and package:flutter_driver.
If the test uses VM service extensions provided by the Flutter SDK, then this error is likely caused by a bug in Flutter. Please report it by filing a bug on GitHub:
https://github.com/flutter/flutter/issues/new?template=02_bug.yml
Original error: ScriptTimeoutException (500): script timeout
(Session info: chrome=140.0.7339.207)
Original stack trace:
#0 parseW3cResponse (package:webdriver/src/handler/w3c/utils.dart:78:9)
#1 W3cCoreHandler.parseExecuteResponse (package:webdriver/src/handler/w3c/core.dart:59:19)
#2 WebDriver.execute. (package:webdriver/src/async/web_driver.dart:264:37)
#3 AsyncRequestClient.send (package:webdriver/src/common/request_client.dart:96:32)

#4 FlutterWebConnection.sendCommand (package:flutter_driver/src/driver/web_driver.dart:316:7)

#5 WebFlutterDriver.sendCommand (package:flutter_driver/src/driver/web_driver.dart:126:14)

#6 FlutterDriver.requestData (package:flutter_driver/src/driver/driver.dart:573:7)

#7 integrationDriver (package:integration_test/integration_test_driver_extended.dart:87:25)

#8 main (file:///Users/runner/work/devtools/devtools/packages/devtools_app/test_driver/integration_test.dart:22:3)

#0 FlutterWebConnection.sendCommand (package:flutter_driver/src/driver/web_driver.dart:318:7)

#1 WebFlutterDriver.sendCommand (package:flutter_driver/src/driver/web_driver.dart:126:14)

#2 FlutterDriver.requestData (package:flutter_driver/src/driver/driver.dart:573:7)

#3 integrationDriver (package:integration_test/integration_test_driver_extended.dart:87:25)

#4 main (file:///Users/runner/work/devtools/devtools/packages/devtools_app/test_driver/integration_test.dart:22:3)

#0 IntegrationTestRunner.run.runTest (package:devtools_shared/src/test/integration_test_runner.dart:154:11)

#1 IntegrationTestRunner.run (package:devtools_shared/src/test/integration_test_runner.dart:160:5)

#2 runFlutterIntegrationTest (file:///Users/runner/work/devtools/devtools/packages/devtools_app/integration_test/test_infra/run/run_test.dart:121:5)

#3 _runTest (file:///Users/runner/work/devtools/devtools/packages/devtools_app/integration_test/run_tests.dart:84:3)

#4 runOneOrManyTests (package:devtools_shared/src/test/integration_test_runner.dart:354:9)

#5 main (file:///Users/runner/work/devtools/devtools/packages/devtools_app/integration_test/run_tests.dart:53:3)

Error: Process completed with exit code 255.
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.