SixLabors / SixLabors/ImageSharp
JpegEncoderTests are not reliable
Nobody has claimed this yet.
- Dominant language
- C#
- Stars
- 8k
- Forks
- 899
- Avg merge
- 1d 5h
- Merged PRs (30d)
- 7
Description
Prerequisites
- I have written a descriptive issue title
- I have verified that I am running the latest version of ImageSharp
- I have verified if the problem exist in both
DEBUGandRELEASEmode - I have searched open and closed issues to ensure it has not already been reported
Description
Our current validation logic in JpegEncoderTests is too tolerant, which caused #1549 to go through the quality gate completely unnoticed. The method for calculating the tolerance percentage is result of trial-and error experiments (dealing with platform differences if memory serves well):
The method above returns a comparer of 15% tolerance, for subsample=420, quality=100, which doesn't really make sense. The image in https://github.com/SixLabors/ImageSharp/issues/1549#issuecomment-778615704 has ~4% difference compared to the image before encoding.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in tests/ImageSharp.Tests/Formats/Jpg/JpegEncoderTests.cs, especially lines 107-128, and review how tolerance is calculated for subsample=420 and quality=100. Compare the validation with the roughly 4% difference reported in #1549 while accounting for the noted platform differences. Done means the JPEG encoder tests catch meaningful regressions without becoming unreliable across platforms.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp
- Domain
- testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100