pydata / pydata/xarray

combine_first of Datasets changes dtype of variable present only in one Dataset

Open
#4,220 2 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

topic-combine
Dominant language
Python
Stars
4.2k
Forks
1.4k
Avg merge
2d 15h
Merged PRs (30d)
14

Description

What happened: I was combining two Datasets using combine_first and to my surprise the dtype of one of the DataArrays in the merged Dataset was changed (from bool to float64).

What you expected to happen: No change in dtype.

Minimal Complete Verifiable Example:

>>> import xarray as xr
>>> ds = xr.Dataset(coords={'abc': list('abc')})
>>> ds['x'] = ('abc', [1., 2., 3.])
>>> ds['y'] = ('abc', [-1., -2., -3.])
>>> ds['t'] = ('abc', [True, False, True])
>>> ds
<xarray.Dataset>
Dimensions:  (abc: 3)
Coordinates:
  * abc      (abc) <U1 'a' 'b' 'c'
Data variables:
    x        (abc) float64 1.0 2.0 3.0
    y        (abc) float64 -1.0 -2.0 -3.0
    t        (abc) bool True False True
>>> xy4b = ds[['x', 'y']].sel(abc=~ds.t) * 10
>>> xy4b.combine_first(ds)
Out[14]: 
<xarray.Dataset>
Dimensions:  (abc: 3)
Coordinates:
  * abc      (abc) object 'a' 'b' 'c'
Data variables:
    x        (abc) float64 1.0 20.0 3.0
    y        (abc) float64 -1.0 -20.0 -3.0
    t        (abc) float64 1.0 0.0 1.0

Anything else we need to know?: No.

Environment:

Output of xr.show_versions()

commit: None
python: 3.7.8 (default, Jul 5 2020, 21:51:42)
[GCC 9.3.0]
python-bits: 64
OS: Linux
OS-release: 5.4.48-gentoo
machine: x86_64
processor: Intel(R) Core(TM) i7-2620M CPU @ 2.70GHz
byteorder: little
LC_ALL: None
LANG: nl_BE.UTF-8
LOCALE: nl_BE.UTF-8
libhdf5: 1.10.5
libnetcdf: 4.6.1

xarray: 0.12.1
pandas: 1.0.4
numpy: 1.18.5
scipy: 1.4.1
netCDF4: 1.5.3
pydap: None
h5netcdf: None
h5py: 2.10.0
Nio: None
zarr: None
cftime: 1.1.3
nc_time_axis: None
PseudonetCDF: None
rasterio: None
cfgrib: None
iris: None
bottleneck: 1.3.2
dask: 1.2.0
distributed: None
matplotlib: 3.2.1
cartopy: None
seaborn: None
setuptools: 46.4.0
pip: 20.0.2
conda: None
pytest: None
IPython: 7.16.1
sphinx: 3.0.4

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No source file or test is named; reproduce the supplied Dataset.combine_first example first, then trace the combine_first implementation. Done means the result keeps variable t as bool rather than float64, with a regression check for the example.

Written by the indexing model from the issue text.

Assessment

Tech stack
numpy, pandas, python
Domain
data
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.