Perceptual Image Hash

Thanks and congrats on pytest-mpl. It's really lovely 👍

Over on SciTools/iris we've been using a very similar framework to perform graphical testing, but rather than using RMS or SHA-256 hashes we've opted to use perceptual image hashes.

Historically, we found using RMS/SHA-256 quite fragile and too sensitive to change, and that sometimes involved quite an overhead on devs to manually understand whether graphical test failures were significant or not; typically, it really depended on the nature of the image change involved, and that generally took time and effort to figure out.

However, for the last few years now we've adopted perceptual image hashing as a more robust and stable approach to graphical image testing, and we'd love to be able to use pytest-mpl as a general framework (rather than reinvent the wheel) as we're keen to use perceptual image hashing elsewhere in other packages other than iris.

If you want to know more about perceptual image hashing:

• here's a gist notebook detailing outcomes from our initial investigation of perceptual image hashing as an option for SciTools/iris
• here's a scientific paper (of which there are many) performing an analysis of perceptual image hashing algorithms

In the end, we opted to leverage the https://github.com/JohannesBuchner/imagehash package, which is available on both conda-forge and pypi, to perform the actual perceptual image hashing.

At this point, I guess my question to the pytest-mpl core developers are:

1. Have you considered using perceptual image hashing as an approach and decided not to use it? If so, why? That would be really great to know.
2. If you're not against the concept, would you be open to us making an additive change to pytest-mpl where the user can choose to opt-in to an alternative hashing algorithm, that they configure, to perform their graphical testing?

I think that option [2] would be possible and a valuable feature for the pytest-mpl community.

However, you guys have the vision of where you want pytest-mpl go and the use cases that you want to address. It would be really great to know whether you're open to offering configurable hashing kernels as a feature. If you are, then we'd be more than willing to do the work and contribute to pytest-mpl under your guidance and advice.

Thanks 😃

Bắt đầu bằng việc xem xét hành vi hiện có của pytest-mpl trong việc so sánh hình ảnh đồ họa và hashing, sau đó kiểm tra cách gói imagehash có thể hỗ trợ một phương án thay thế opt-in. Được xem là hoàn thành khi đã đưa ra quyết định ở cấp dự án và, nếu được chấp nhận, xác định một tính năng perceptual hashing có thể cấu hình, kèm theo độ bao phủ cho hành vi so sánh hình ảnh của tính năng đó.