pydata / pydata/xarray

Hook for better control over copy behavior with duck-arrays?

Open
#7,057 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Python
Stars
4.2k
Forks
1.4k
Avg merge
2d 15h
Merged PRs (30d)
14

Description

Context

By using copy(deep=False) custom operations may avoid copying large amounts of data but can modify, e.g., coord dicts:

tmp = da.copy(deep=False)
del tmp.coords['abc']
# Use tmp

When da wraps a duck-array with substructure the current implementation is insufficient:

tmp = da.copy(deep=False)
# Imagine a duck-array similar to numpy.ma but with a dict of masks
del tmp.data.masks['abc']  # Bad: breaks `da`
# Use tmp
Describe the solution you'd like

Currently there does not appear a solution to this, unless we know details about the duck array. Therefore, I wonder if we need an additional "hook" that duck-arrays may provide, which could be called by Xarray to make a non-deep copy?

class MyDuckArray:
  def _copy_shallow_(self):  # TODO: better name
    """Copy everything except buffers"""
# in xarray.Variable
def copy(self, deep=True):
  if deep:
    data = copy.deepcopy(data)
  elif hasattr(data, '_copy_shallow_'):
   data = data._copy_shallow_()
Additional context

This is the current implementation for Variable. There is no operation for deep=False:
https://github.com/pydata/xarray/blob/716973e41060184beebd64935afe196a805ef481/xarray/core/variable.py#L956-L957

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in xarray/core/variable.py around lines 956-957 and inspect how Variable.copy handles deep=False for duck-array data. Review related copy behavior and determine the API requirements for a shallow-copy hook; done means the hook design and its behavior for nested duck-array state are agreed and covered by tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.