pydata / pydata/xarray

Describe a DataTree: adding visualization and summarization capabilities

Open
#9,271 0 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Python
Stars
4.2k
Forks
1.4k
Avg merge
2d 15h
Merged PRs (30d)
14

Description

Is your feature request related to a problem?

The string and rich representations of Dataset and DataArrays are standardized. However, the DataTrees can become pretty complex structurally (unlike DataArrays and Dataset). Hence, adding visualizations capabilities to summarize would be helpful, like the pandas describe method, that summarizes a DataFrame.

Describing the DataTree can be a lite-weight string representation focusing on the structure of the tree only without the contents.

Describe the solution you'd like

Here is some code I used to achieve that goal:

# Example taken from
# https://xarray-datatree.readthedocs.io/en/latest/hierarchical-data.html#ancestry-in-an-evolutionary-tree
vertebrates = dt.DataTree.from_dict(
    name="Vertebrae",
    d={
        "/Sharks": None,
        "/Bony Skeleton/Ray-finned Fish": None,
        "/Bony Skeleton/Four Limbs/Amphibians": None,
        "/Bony Skeleton/Four Limbs/Amniotic Egg/Hair/Primates": None,
        "/Bony Skeleton/Four Limbs/Amniotic Egg/Hair/Rodents & Rabbits": xr.Dataset(
            {f"variable_{k}": None for k in "abc"}
        ),
        "/Bony Skeleton/Four Limbs/Amniotic Egg/Two Fenestrae/Dinosaurs": xr.Dataset(
            {f"variable_{k}": None for k in "abc"}
        ),
        "/Bony Skeleton/Four Limbs/Amniotic Egg/Two Fenestrae/Birds": xr.Dataset(
            {f"variable_{k}": None for k in "abc"}
        ),
    },
)


def repr_datatree(
    xdt: dt.DataTree,
    *,
    tabsize: int = 4,
    offset: int = 1,
    with_full_path: bool = False,
    with_group_variables: bool = False,
):
    lines = []
    for node in xdt.subtree:
        path = PurePosixPath(node.path)
        tabs = len(path.parts)
        lines.append(f'{" " * ((tabs - offset) * tabsize)}{(node.name) if not with_full_path else path}')
        if with_group_variables:
            for varname in node.ds.data_vars:
                lines.append(f'{" " * (tabs * tabsize)}{path / varname if with_full_path else varname}')
    return "\n".join(lines)
print(repr_datatree(vertebrates, with_full_path=False, with_group_variables=False))
Vertebrae
    Sharks
    Bony Skeleton
        Ray-finned Fish
        Four Limbs
            Amphibians
            Amniotic Egg
                Hair
                    Primates
                    Rodents & Rabbits
                Two Fenestrae
                    Dinosaurs
                    Birds
print(repr_datatree(vertebrates, with_full_path=False, with_group_variables=True))
Vertebrae
    Sharks
    Bony Skeleton
        Ray-finned Fish
        Four Limbs
            Amphibians
            Amniotic Egg
                Hair
                    Primates
                    Rodents & Rabbits
                        variable_a
                        variable_b
                        variable_c
                Two Fenestrae
                    Dinosaurs
                        variable_a
                        variable_b
                        variable_c
                    Birds
                        variable_a
                        variable_b
                        variable_c
print(repr_datatree(vertebrates, with_full_path=True, with_group_variables=False))
/
    /Sharks
    /Bony Skeleton
        /Bony Skeleton/Ray-finned Fish
        /Bony Skeleton/Four Limbs
            /Bony Skeleton/Four Limbs/Amphibians
            /Bony Skeleton/Four Limbs/Amniotic Egg
                /Bony Skeleton/Four Limbs/Amniotic Egg/Hair
                    /Bony Skeleton/Four Limbs/Amniotic Egg/Hair/Primates
                    /Bony Skeleton/Four Limbs/Amniotic Egg/Hair/Rodents & Rabbits
                /Bony Skeleton/Four Limbs/Amniotic Egg/Two Fenestrae
                    /Bony Skeleton/Four Limbs/Amniotic Egg/Two Fenestrae/Dinosaurs
                    /Bony Skeleton/Four Limbs/Amniotic Egg/Two Fenestrae/Birds
print(repr_datatree(vertebrates, with_full_path=True, with_group_variables=True))
/
    /Sharks
    /Bony Skeleton
        /Bony Skeleton/Ray-finned Fish
        /Bony Skeleton/Four Limbs
            /Bony Skeleton/Four Limbs/Amphibians
            /Bony Skeleton/Four Limbs/Amniotic Egg
                /Bony Skeleton/Four Limbs/Amniotic Egg/Hair
                    /Bony Skeleton/Four Limbs/Amniotic Egg/Hair/Primates
                    /Bony Skeleton/Four Limbs/Amniotic Egg/Hair/Rodents & Rabbits
                        /Bony Skeleton/Four Limbs/Amniotic Egg/Hair/Rodents & Rabbits/variable_a
                        /Bony Skeleton/Four Limbs/Amniotic Egg/Hair/Rodents & Rabbits/variable_b
                        /Bony Skeleton/Four Limbs/Amniotic Egg/Hair/Rodents & Rabbits/variable_c
                /Bony Skeleton/Four Limbs/Amniotic Egg/Two Fenestrae
                    /Bony Skeleton/Four Limbs/Amniotic Egg/Two Fenestrae/Dinosaurs
                        /Bony Skeleton/Four Limbs/Amniotic Egg/Two Fenestrae/Dinosaurs/variable_a
                        /Bony Skeleton/Four Limbs/Amniotic Egg/Two Fenestrae/Dinosaurs/variable_b
                        /Bony Skeleton/Four Limbs/Amniotic Egg/Two Fenestrae/Dinosaurs/variable_c
                    /Bony Skeleton/Four Limbs/Amniotic Egg/Two Fenestrae/Birds
                        /Bony Skeleton/Four Limbs/Amniotic Egg/Two Fenestrae/Birds/variable_a
                        /Bony Skeleton/Four Limbs/Amniotic Egg/Two Fenestrae/Birds/variable_b
                        /Bony Skeleton/Four Limbs/Amniotic Egg/Two Fenestrae/Birds/variable_c
def empty_datatree_structure(xdt: dt.DataTree) -> dt.DataTree:
    # Print an empty datatree from a full structure (like a tree command)
    return dt.DataTree.from_dict({k.path: None for k in xdt.leaves})


empty_datatree_structure(loaded_xdt)
Describe alternatives you've considered

No response

Additional context

Issue content taken from https://github.com/xarray-contrib/datatree/issues/334

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No repository file or test is named. Start by reviewing the DataTree representation and traversal examples in the issue, then inspect the existing DataTree entry points for string representations; done should provide a lightweight structure-focused summary with the requested options for paths and group variables, supported by appropriate tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data, data-visualization
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.