flutter / flutter/flutter

`flutter drive` should emit a specific exit code (at minimum) if an emulator or device goes offline

Open
#153,603 0 comments 1 reaction 0 assignees View on GitHub
a: tests c: tech-debt f: integration_test P2 t: flutter driver team-tool tool tool-still-valid triaged-tool
Dominant language
Dart
Stars
179k
Forks
31.1k
PR merge metrics
PR metrics pending

Description

In https://github.com/flutter/flutter/issues/153445 we investigated that Android 35 (the upcoming release) is too unstable on emulators to be used on pre or post-submit infrastructure (see our [CI changes here (https://github.com/flutter/flutter/issues/153445)). In an informal conversation between @zanderso @reidbaker @johnmccutchan and @matanlurey we talked about our inability to determine root causes.

@johnmccutchan wrote:

> I guess I'm wondering why we can't have finer grained signals of test failures?
>
> Like if an emulator is crashing I'd like that to be flagged as something distinct than a flakey test. Because it wasn't the test that flaked actually, it was our environment.

@zanderso wrote:

> Yeah, I was thinking like `flutter drive` would have different error exit codes for a test failure vs. failures for other reasons like the emulator/device disappearing.

So, this issue is tracking _doing this_.

I admit I am not sure how easy this will be, in particular to do across every platform we support.

If we limit the scope (i.e. _Android_ devices), or file sub-issues for every platform, it might be viable to do _some_ of these.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.