scverse / scverse/scanpy

Improve ergonomics of image comparison fixture

Open
#2,649 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Area - Plotting 🌺 Development Process 🚀
Dominant language
Python
Stars
2.6k
Forks
779
Avg merge
1d 4h
Merged PRs (30d)
27

Description

What kind of feature would you like to request?

Other?

Please describe your wishes

As a continuation of #1772, we should simplify how this is done.

  • Decide on fixture api
  • Decide how generated outputs are handled (.gitignore-d, deleted each time that test is run, both?)

My proposal for the first TODO is as follows.

In order to test the image files

  • ./_images/some_plot/expected.png
  • ./_images/some_plot_x_context/expected.png

… against their actual.png counterparts, we currently do

# per module setup
ROOT = HERE / '_images'

# individual test
def test_some_plot(image_comparer):
    save_and_compare_images = partial(image_comparer, ROOT, tol=15)
    ...
    save_and_compare_images('some_plot')
    ...  # and maybe
    save_and_compare_images('some_plot_x_context')

I propose we arrive here (with --strict-markers on):

# per module setup
@pytest.fixture(scope='module')
def compare_images_root():
    """Set image root for save_and_compare_images."""
    return HERE / '_images'

# individual test
@pytest.mark.compare_images_tol(15)
def test_some_plot(save_and_compare_images):
    ...
    save_and_compare_images()
    ...  # and maybe
    save_and_compare_images('x_context')

for that we can make save_and_compare_images

  1. use the request fixture to retrieve tol and test name
  2. use the test name for the png directory name used.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing #1772 and locating the existing image-comparison fixture and its callers; no implementation files or tests are named. Define the fixture API and generated-output policy, then verify the proposed marker, test-name paths, and strict-marker behavior against the relevant image-comparison tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing-qa
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.