Improve visual parity tests by adding a threshold
Nobody has claimed this yet.
- Dominant language
- C#
- Stars
- 7.8k
- Forks
- 1.2k
- Avg merge
- 2d 17h
- Merged PRs (30d)
- 49
Description
Is your feature request related to a problem? Please describe.
When there's subtle differences between platforms or graphics cards there may be subtle differences in generated images. I had a test fail because there was 1 pixel difference in the images (reference vs screenshot).
Describe the solution you'd like
Add a threshold parameter in ImageTester.CompareImage. The simplest thing would be to just calculate a percentage of different pixels divided by all pixels in the image. This way we can allow 1-2% changes for different hardware before it needs manual inspection for differences.
I don't think there's much point in implementing a more complex comparison algorithm, but in case people are interested in the topic here's an interesting SO post
The threshold should be configurable per end user test class.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating ImageTester.CompareImage and the end user test classes that call it, then inspect how image comparison failures are reported. The work is done when a configurable percentage threshold allows small pixel differences while still flagging larger mismatches, with tests covering the threshold behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp
- Domain
- computer-graphics, testing-qa
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100