pydata / pydata/xarray

DatasetView class breaks Liskov's rule.

Open
#8,855 5 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

topic-DataTree topic-typing
Dominant language
Python
Stars
4.2k
Forks
1.4k
Avg merge
2d 15h
Merged PRs (30d)
14

Description

What is your issue?

Working on migrating the datatree.py module into xarray/core revealed that the DatasetView class, which implements Dataset while disabling methods to mutate the object, breaks Liskov's substitution principle. The type for one of the overloads of DatasetView.__getitem__ is more general than the corresponding Dataset.__getitem__ signature (due to the use of Self in the Dataset signature).

# In Dataset:
class Dataset(...):
    ...
    @overload
    def __getitem__(self, key: Iterable[Hashable]) -> Self: ...

The use of Self means that signature inherited from the superclass has a return type of DatasetView, but the DatasetView signature is overridden to have a return type of Dataset (the more generalised parent).

To avoid this, a couple of implementations were attempted:

  • A class that tries to intercept the methods that mutate the Dataset using getattr. This does not catch the __setitem__ method, as it is a Magic Method, and those aren't affected by getattr.
  • A Metaclass that can intercept Magic Methods, too. Implementation was inspired from here. I didn't get it to fully work, and eventually realised this was getting too complicated given the scope of the original problem.
  • A mix-in for the mutating methods. I couldn't get this to work in the timescale agreed upon.
  • Resorting back to ignoring the mypy errors for now, so we can proceed with the migration (given that there isn't a significant implementation concern identified from these type issues).

Also note, there is a tangentially-related known mypy error when a property setter accepts an argument of a different type to the property itself (https://github.com/python/mypy/issues/3004). This affects the assignment of Dataset objects to the DataTree.ds property. (Separate issue, but related)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading xarray/core/dataset.py around the Dataset.getitem signature and the datatree.py migration linked in the issue. Compare the DatasetView overrides and the attempted interception, metaclass, and mix-in approaches. Done requires a decided design that preserves DatasetView's mutation restrictions while resolving the reported mypy incompatibility; no specific test is named.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data, tooling
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.