pydata / pydata/xarray

Concatenate 3D array with 2D array

Open
#3,954 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

topic-documentation topic-error reporting
Dominant language
Python
Stars
4.2k
Forks
1.4k
Avg merge
2d 15h
Merged PRs (30d)
14

Description

The 3D array has three dims: z, y and x.
The 2D array has two dims: y and x.
When I try to concatenate them by expanding the 2D array with z dim, there's something wrong in _dataset_concat

MCVE Code Sample
import xarray as xr
import numpy as np

x = 2
y = 4
z = 3
data = np.arange(x*y*z).reshape(z, x, y)

# 3d array with coords
a = xr.DataArray(data, dims=['z', 'y', 'x'], coords={'z': np.arange(z)})

# 2d array without coords
b = xr.DataArray(np.arange(x*y).reshape(x, y)*1.5, dims=['y', 'x'])

# expand 2d to 3d
b = b.expand_dims('z')

# concat
comb = xr.concat([a, b], dim='z')
Expected Output

Same as np.concatenate:

concat_array = np.concatenate((a, b))
comb = xr.DataArray(concat_array, dims={'z', 'y', 'x'})
<xarray.DataArray (z: 4, x: 2, y: 4)>
array([[[ 0. ,  1. ,  2. ,  3. ],
        [ 4. ,  5. ,  6. ,  7. ]],

       [[ 8. ,  9. , 10. , 11. ],
        [12. , 13. , 14. , 15. ]],

       [[16. , 17. , 18. , 19. ],
        [20. , 21. , 22. , 23. ]],

       [[ 0. ,  1.5,  3. ,  4.5],
        [ 6. ,  7.5,  9. , 10.5]]])
Dimensions without coordinates: z, x, y
Problem Description
    comb = xr.concat([a, b], dim='z')
  File "E:\miniconda3\envs\satpy\lib\site-packages\xarray\core\concat.py", line 135, in concat
    return f(objs, dim, data_vars, coords, compat, positions, fill_value, join)
  File "E:\miniconda3\envs\satpy\lib\site-packages\xarray\core\concat.py", line 455, in _dataarray_concat
    join=join,
  File "E:\miniconda3\envs\satpy\lib\site-packages\xarray\core\concat.py", line 395, in _dataset_concat
    raise ValueError("%r is not present in all datasets." % k)
ValueError: 'z' is not present in all datasets.

As suggested by @dcherian, assigning the coordinate label by changing b = b.expand_dims('z') to b = b.expand_dims(z=[3]) makes it work.

Versions
Output of `xr.show_versions()`

INSTALLED VERSIONS

commit: None
python: 3.7.6 | packaged by conda-forge | (default, Jan 7 2020, 21:48:41) [MSC v.1916 64 bit (AMD64)]
python-bits: 64
OS: Windows
OS-release: 10
machine: AMD64
processor: Intel64 Family 6 Model 158 Stepping 9, GenuineIntel
byteorder: little
LC_ALL: None
LANG: None
LOCALE: None.None
libhdf5: None
libnetcdf: None

xarray: 0.15.1
pandas: 1.0.3
numpy: 1.18.1
scipy: 1.4.1
netCDF4: None
pydap: None
h5netcdf: None
h5py: None
Nio: None
zarr: None
cftime: 1.1.1.2
nc_time_axis: None
PseudoNetCDF: None
rasterio: 1.1.3
cfgrib: None
iris: None
bottleneck: None
dask: 2.10.1
distributed: 2.14.0
matplotlib: 3.2.1
cartopy: 0.17.0
seaborn: 0.10.0
numbagg: None
setuptools: 46.1.3.post20200325
pip: 20.0.2
conda: None
pytest: None
IPython: 7.13.0
sphinx: 2.4.4

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the MCVE and inspect xarray/core/concat.py, especially _dataset_concat, where the reported ValueError is raised. Confirm the behavior with the 3D and expanded 2D arrays, then ensure concatenation succeeds and matches the expected four-slice result without requiring an explicit z coordinate.

Written by the indexing model from the issue text.

Assessment

Tech stack
numpy, python
Domain
data
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.