flutter / flutter/flutter

Testing Color Contrast in AlertDialog

Open
#148,657 6 comments 0 reactions 0 assignees View on GitHub
a: tests found in release: 3.22 framework has reproducible steps P3 team-framework triaged-framework
Dominant language
Dart
Stars
179k
Forks
31.1k
PR merge metrics
PR metrics pending

Description

### Steps to reproduce

I'm building a complex app with asynchronous code, providers, external libraries, and so on. It works well and passes tests in most cases, except for this one. I created a minimal code example to demonstrate the issue. In my complex app, for some reason, I need to run two pumps for the AlertDialog test to pass. However, only the color contrast test fails. I'm unsure why it fails, as it shouldn't fail regardless of the number of pumps I run.

### Expected results

Test should pass

### Actual results

Test fails

### Code sample

lib/main.dart

```dart
import 'package:flutter/material.dart';

void main() => runApp(const AlertDialogExampleApp());

class AlertDialogExampleApp extends StatelessWidget {
const AlertDialogExampleApp({super.key});

@override
Widget build(BuildContext context) {
return const MaterialApp(
home: Scaffold(
body: Center(
child: DialogExample(),
),
),
);
}
}

class DialogExample extends StatelessWidget {
const DialogExample({super.key});

@override
Widget build(BuildContext context) {
return TextButton(
child: const Text('Show Dialog'),
onPressed: () => showDialog(
context: context,
builder: (BuildContext context) => AlertDialog(
title: const Text('Title'),
content: const Text('Alert dialog description'),
actions: [
TextButton(
onPressed: () => Navigator.pop(context, 'OK'),
child: const Text('OK'),
),
],
),
),
);
}
}

```

test/widget_test.dart

```dart
import 'package:flutter/material.dart';
import 'package:flutter_test/flutter_test.dart';

import 'package:flutter_dialog_color_contrast/main.dart';

void main() {
testWidgets('Test AlertDialog Color Contrast', (tester) async {
final SemanticsHandle semanticsHandle = tester.ensureSemantics();
await tester.pumpWidget(const AlertDialogExampleApp());

// Before showing dialog. Test passes
expect(find.text('Show Dialog'), findsOneWidget);
expect(find.byType(AlertDialog), findsNothing);
expect(find.text('Alert dialog description'), findsNothing);
expect(find.text('OK'), findsNothing);

// Tap to show AlertDialog
await tester.tap(find.text('Show Dialog'));

// In order for my complex app to pass tests, it requires two pumps as below
await tester.pump();
// This 2nd pump causes the color contrast test failure. Test will pass if commented
await tester.pump(Durations.short1);

// After showing dialog. Test passes
expect(find.byType(AlertDialog), findsOneWidget);
expect(find.text('Alert dialog description'), findsOneWidget);
expect(find.text('OK'), findsOneWidget);

// Color contrast test fails
await expectLater(tester, meetsGuideline(textContrastGuideline));

semanticsHandle.dispose();
});
}

```

### Screenshots or Video

_No response_

### Logs

Logs

```console
══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞════════════════════════════════════════════════════
The following TestFailure was thrown running a test:
Expected: Text contrast should follow WCAG guidelines
Actual:
Which: SemanticsNode#10(Rect.fromLTRB(24.0, 72.0, 366.0, 92.0), label: "Alert dialog
description", textDirection: ltr):
Expected contrast ratio of at least 4.5 but found 3.57 for a font size of 14.0.
The computed colors was:
light - Color(0xffcecad0), dark - Color(0xff676568)
See also: https://www.w3.org/TR/UNDERSTANDING-WCAG20/visual-audio-contrast-contrast.html
SemanticsNode#11(Rect.fromLTRB(302.0, 116.0, 366.0, 164.0), actions: [tap], flags:
[isButton, hasEnabledState, isEnabled, isFocusable], label: "OK", textDirection: ltr):
Expected contrast ratio of at least 4.5 but found 2.27 for a font size of 14.0.
The computed colors was:
light - Color(0xffcecad0), dark - Color(0xff8c7fa9)
See also: https://www.w3.org/TR/UNDERSTANDING-WCAG20/visual-audio-contrast-contrast.html

When the exception was thrown, this was the stack:
#0 fail (package:matcher/src/expect/expect.dart:149:31)
#1 _expect. (package:matcher/src/expect/expect.dart:125:9)

#8 expectLater. (package:flutter_test/src/widget_tester.dart:512:24)

#9 main. (file:///Users/anasqaderi/Coding/play/flutter_dialog_color_contrast/test/widget_test.dart:31:5)

#10 testWidgets.. (package:flutter_test/src/widget_tester.dart:183:15)

#11 TestWidgetsFlutterBinding._runTestBody (package:flutter_test/src/binding.dart:1017:5)

(elided 7 frames from dart:async and package:stack_trace)

The test description was:
Test AlertDialog Color Contrast
════════════════════════════════════════════════════════════════════════════════════════════════════
```

### Flutter Doctor output

Doctor output

```console
[✓] Flutter (Channel stable, 3.19.6, on macOS 14.4.1 23E224 darwin-arm64, locale en-AU)
• Flutter version 3.19.6 on channel stable at /Users/anasqaderi/Coding/exe/flutter
• Upstream repository https://github.com/flutter/flutter.git
• Framework revision 54e66469a9 (5 weeks ago), 2024-04-17 13:08:03 -0700
• Engine revision c4cd48e186
• Dart version 3.3.4
• DevTools version 2.31.1

[✓] Android toolchain - develop for Android devices (Android SDK version 34.0.0)
• Android SDK at /Users/anasqaderi/Library/Android/sdk
• Platform android-34, build-tools 34.0.0
• Java binary at: /Applications/Android Studio.app/Contents/jbr/Contents/Home/bin/java
• Java version OpenJDK Runtime Environment (build 17.0.6+0-17.0.6b829.9-10027231)
• All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 15.4)
• Xcode at /Applications/Xcode.app/Contents/Developer
• Build 15F31d
• CocoaPods version 1.14.2

[✓] Chrome - develop for the web
• Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2022.3)
• Android Studio at /Applications/Android Studio.app/Contents
• Flutter plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/9212-flutter
• Dart plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/6351-dart
• Java version OpenJDK Runtime Environment (build 17.0.6+0-17.0.6b829.9-10027231)

[✓] VS Code (version 1.89.1)
• VS Code at /Applications/Visual Studio Code.app/Contents
• Flutter extension version 3.88.0

[✓] Connected device (4 available)
• Anas iPhone 15 Pro Max (mobile) • 00008130-001079A10A98001C • ios • iOS 17.4.1 21E236
• Anas iPhone XS Max (mobile) • 00008020-000E6CCA1104002E • ios • iOS 17.5 21F79
• macOS (desktop) • macos • darwin-arm64 • macOS 14.4.1 23E224 darwin-arm64
• Chrome (web) • chrome • web-javascript • Google Chrome 124.0.6367.209

[✓] Network resources
• All expected network resources are available.

• No issues found!
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.