matplotlib / matplotlib/pytest-mpl

Perceptual Image Hash

Ouverte
#146 12 commentaires 15 réactions 0 personnes assignées Voir sur GitHub

Personne n'a encore pris cette issue.

enhancement help wanted
Langage dominant
Python
Étoiles
272
Forks
53
Métriques de merge des PR
Aucune PR mergée en 30 j

Description

Thanks and congrats on `pytest-mpl`. It's really lovely :+1:

Over on [SciTools/iris](https://github.com/SciTools/iris) we've been using a very similar framework to perform graphical testing, but rather than using RMS or SHA-256 hashes we've opted to use perceptual image hashes.

Historically, we found using RMS/SHA-256 quite fragile and too sensitive to change, and that sometimes involved quite an overhead on devs to manually understand whether graphical test failures were significant or not; typically, it really depended on the nature of the image change involved, and that generally took time and effort to figure out.

However, for the last few years now we've adopted perceptual image hashing as a more robust and stable approach to graphical image testing, and we'd love to be able to use `pytest-mpl` as a general framework (rather than reinvent the wheel) as we're keen to use perceptual image hashing elsewhere in other packages other than `iris`.

If you want to know more about perceptual image hashing:
- here's a [gist notebook](https://gist.github.com/bjlittle/6440844ea509276edf4bfbea18c8d41a) detailing outcomes from our initial investigation of perceptual image hashing as an option for `SciTools/iris`
- here's a [scientific paper](https://www.sciencedirect.com/science/article/pii/S1877050921011030) (of which there are many) performing an analysis of perceptual image hashing algorithms

In the end, we opted to leverage the https://github.com/JohannesBuchner/imagehash package, which is available on both `conda-forge` and `pypi`, to perform the actual perceptual image hashing.

At this point, I guess my question to the `pytest-mpl` core developers are:
1. Have you considered using perceptual image hashing as an approach and decided not to use it? If so, why? That would be really great to know.
2. If you're not against the concept, would you be open to us making an additive change to `pytest-mpl` where the user can choose to opt-in to an alternative hashing algorithm, that they configure, to perform their graphical testing?

I think that option [2] would be possible and a valuable feature for the `pytest-mpl` community.

However, you guys have the vision of where you want `pytest-mpl` go and the use cases that you want to address. It would be really great to know whether you're open to offering configurable hashing kernels as a feature. If you are, then we'd be more than willing to do the work and contribute to `pytest-mpl` under your guidance and advice.

Thanks :smiley:

Guide de contribution

Aucun guide de contribution indexé pour ce dépôt

Par où commencer

  1. Lisez l'issue en entier, puis le guide de contribution du projet.
  2. Signalez en commentaire que vous la prenez — cela évite que deux personnes fassent le même travail.
  3. Forkez le dépôt et travaillez sur une branche.
  4. Ouvrez une pull request qui référence le numéro de l'issue.

Piste de recherche

Commencez par examiner le comportement existant de pytest-mpl en matière de comparaison d’images graphiques et de hachage, puis étudiez comment le paquet imagehash pourrait prendre en charge une alternative opt-in. Le travail sera considéré comme terminé lorsqu’une décision de projet aura été prise et, si elle est acceptée, qu’une fonctionnalité configurable de hachage perceptuel aura été définie, avec une couverture de son comportement de comparaison d’images.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Évaluation

Stack technique
python
Domaine
testing-qa
Type d'issue
Fonctionnalité
Difficulté
5/5
Temps estimé
Plus d'une semaine
Activité
À l'abandon
Clarté
À clarifier
Accessibilité débutants
35/100

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.