darekkay / darekkay/evaluatory

Different Playwright option, per "device"?

Open
#7 2 comments 0 reactions 0 assignees View on GitHub
Type: Enhancement
Dominant language
JavaScript
Stars
98
Forks
7
PR merge metrics
No merged PRs in 30d

Description

The attention to detail you've baked into Evaluatory is inspiring. Thank you for building (and maintaining) this!

I'm interested in weaving the configuration between Playwright and the screenshot module a bit. Can you see a path to defining "devices" so that a given page's report contains both `viewport`-based screenshots and `forcedColors` + `colorScheme` screenshots that approximate high contrast preferences?

I can accomplish a dark text on light background with the following:
```JSON
"playwrightOptions": {
"forcedColors": "active",
"colorScheme": "light"
}
```

And a light text on dark background with:
```JSON
"playwrightOptions": {
"forcedColors": "active",
"colorScheme": "dark"
}
```

Those settings remain for the entirety of the Playwright session. Pulling those options into the "device" definition would require some Playwright resetting (and potentially additional execution time). I'm unsure how feasible that is, though.

In my mind, I see device configuration along these lines:
```JSON
"name": "Narrow, high contrast (dark)",
"options": {
"viewport": {
"width": 320,
"height": 2000
},
"forcedColors": "active",
"colorScheme": "dark"
}
```

Ideally, I would love to corral all the screenshots together — showing various viewports and user preference outcomes per URL.

As it stands, I can run a few passes with Evaluatory, each using a different configuration to accomplish this.

Contributor guide

No contributing guide indexed for this repository

Research direction

The issue names no files or tests. Start by tracing Evaluatory’s Playwright configuration and screenshot-generation entry points, then assess whether device options can be applied per screenshot set while retaining viewport and preference combinations. Done means a working configuration that groups the requested screenshots per URL, with execution-time trade-offs addressed.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, playwright
Domain
accessibility
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.