Roadmap for `v1.0.0`

I reviewed all the configuration options which are used in the code and found quite a few inconsistencies. I have listed proposed fixes for them, separated into breaking and non-breaking changes. We can add warning for the breaking changes now, and then change in v1.0.0.

Let's discuss these lists. Are there any items which should be added or removed? For the breaking changes, how long of a warning period should we have? For any of them, should we only start warning in v1.0.0 and then change in v1.1.0?

Review of configuration options

Notes

1. N/A are options which we don't think should exist.
2. Empty boxes are options which we do think should exist.
3. ? are options which could exist but would be of limited use.
4. Due to how paths are computed by Python, the baseline directory and hash library paths can be absolute. In this case --mpl-baseline-relative and what directory the paths are interpreted to be relative to does not have any effect.
5. Path specified in kwargs should be relative to the test file, and paths specified in CLI and ini options should be relative to where pytest was run.

Non-breaking changes

• Document that backend can be specified. #199
• Document the option to use the full test name. #199
• Document that mpl-use-full-test-name ini overrides the filename kwarg. #199
• Add mpl-baseline-path ini option (relative to where pytest was run). #181
• Expand mpl-use-full-test-name ini option to CLI (and maybe kwarg). #181
• Add mpl-hash-library ini option (relative to where pytest was run). #181
• Add --mpl-default-tolerance CLI and mpl-default-tolerance ini options. #181
• Add CLI and ini options for deterministic. #197
• Add --mpl-default-style CLI and mpl-default-style ini options. #181
• Add CLI and ini options for remove_text.
• Add CLI and ini options for backend. #181
• Add generate-summary ini option for generating test summaries. #181

Breaking changes

• --mpl-hash-library should be relative to where pytest was run (to match --mpl-baseline-path).
• Local hash_library kwarg should take precedence over global --mpl-hash-library CLI option. #154
• Deprecate the use of multiple hash libraries in the same pytest run (not necessary because files are small and include the full module path anyway). #154
• Deprecate the use of multiple testing modes in the same pytest run (e.g. warn if one test only has hash comparison while the others only have image comparison)
• Use Matplotlib's default style instead of 'classic'.
• Change the default RMS tolerance to 0.
• Enable deterministic PNG files by default.
• Deprecate remove_text kwarg and replace with better named remove_ticks_and_titles
• (???) Add a remove_text kwarg that actually removes all text.
• (???) Rename baseline_dir/--mpl-baseline-path to have consistent name.

Beginne mit der Prüfung der Konfigurationsoptionen und der in diesem Issue aufgeführten ungelösten Breaking- und Non-Breaking-Änderungen und untersuche anschließend pytest_mpl/plugin.py am genannten remove_text-Einstiegspunkt. Das Issue erfordert Entscheidungen der Maintainer zu Umfang, Warnzeiträumen und Standardwerten, bevor mit der Implementierung begonnen werden kann; als abgeschlossen würde es einen abgestimmten Fahrplan und entsprechende Änderungen erfordern.