pydata / pydata/xarray

apply_ufunc + dask="parallelized" + no core dimensions should raise a nicer error about core dimensions being absent

Open
#6,372 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
4.2k
Forks
1.4k
Avg merge
2d 15h
Merged PRs (30d)
14

Description

What happened?

From https://github.com/pydata/xarray/discussions/6370

Calling apply_ufunc(..., dask="parallelized") with no core dimensions and dask input "works" but raises an error on compute (ValueError: axes don't match array from np.transpose).

xr.apply_ufunc(
    lambda x: np.mean(x),
    dt,
    dask="parallelized"
)
What did you expect to happen?

With numpy data the apply_ufunc call does raise an error:

xr.apply_ufunc(
    lambda x: np.mean(x),
    dt.compute(),
    dask="parallelized"
)
ValueError: applied function returned data with unexpected number of dimensions. Received 0 dimension(s) but expected 1 dimensions with names: ('x',)
Minimal Complete Verifiable Example
import xarray as xr

dt = xr.Dataset(
    data_vars=dict(
        value=(["x"], [1,1,2,2,2,3,3,3,3,3]), 
    ),
    coords=dict(
        lon=(["x"], np.linspace(0,1,10)),
    ),
).chunk(chunks={'x': tuple([2,3,5])}) # three chunks of different size

xr.apply_ufunc(
    lambda x: np.mean(x),
    dt,
    dask="parallelized"
)
Relevant log output

No response

Anything else we need to know?

No response

Environment

N/A

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at the apply_ufunc entry point and compare the dask="parallelized" path with the numpy-data behavior shown in the issue. The work is done when inputs with no core dimensions raise a clear ValueError before compute, rather than failing later in np.transpose with "axes don't match array".

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.