A broadcasting sum for xarray.Dataset
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 4.2k
- Forks
- 1.4k
- Avg merge
- 2d 15h
- Merged PRs (30d)
- 14
Description
I've found it useful to have a version of Dataset.sum which sums variables in a way that's consistent with what would happen if they were broadcast to the full Dataset dimensions.
The difference is in what it does with variables that don't contain some of the dimensions it's asked to sum over: standard sum just ignores the summation over these dimensions for these variables, whereas a broadcasting_sum will multiply the variable by the product of sizes the missing dimensions, like so:
def broadcast_sum(dataset, dims):
def broadcast_sum_var(var):
present_sum_dims = [dim for dim in dims if dim in var.dims]
non_present_sum_dims = [dim for dim in dims if dim not in var.dims]
return var.sum(present_sum_dims) * np.prod([dataset.sizes[dim] for dim in non_present_sum_dims])
return dataset.map(broadcast_sum_var)
This is consistent with mathematical sum notation, where the sum doesn't become a no-op just because the summand doesn't reference the index being summed over. E.g.:
$\sum_{n=1}^N x = N x$
I've found it useful when you need to do some broadcasting operations across different variables after the sum, and you want the summation done in a way that's consistent with the broadcasting logic that will be applied later.
Would you be open to adding this, and if so any preference how? (A separate method, an option to .sum ?)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the existing Dataset.sum entry point and comparing it with the proposed broadcast_sum example in this issue. Determine whether the behavior belongs in a separate method or as an option to sum, then define coverage for variables missing requested dimensions and verify that the result matches broadcasting semantics.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- numpy, python
- Domain
- data
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100