Cannot store data after `groupby_bins`
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 4.2k
- Forks
- 1.4k
- Avg merge
- 2d 15h
- Merged PRs (30d)
- 14
Description
Hi Xarray!
I really like your Library. But now I am stuck completely.
Code Sample, a copy-pastable example if possible
import numpy as np
import xarray as xr
data = [1,2,3,4,5,6,7,8,9,10]
bins = np.array(range(5)) * 2
xr_data = xr.Dataset({'data': data})
out = xr_data.groupby_bins('data', bins).mean()
out.to_netcdf('/tmp/test')
Problem description
Get Error :
Traceback (most recent call last):
File "/home/volker/workspace/pycharm-community-2018.1.2/helpers/pydev/pydevd.py", line 1664, in <module>
main()
File "/home/volker/workspace/pycharm-community-2018.1.2/helpers/pydev/pydevd.py", line 1658, in main
globals = debugger.run(setup['file'], None, None, is_module)
File "/home/volker/workspace/pycharm-community-2018.1.2/helpers/pydev/pydevd.py", line 1068, in run
pydev_imports.execfile(file, globals, locals) # execute the script
File "/home/volker/workspace/pycharm-community-2018.1.2/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "/home/volker/workspace/eprofile_wind/eprofile/src/eprofile/sandbox/test_xarray.py", line 12, in <module>
out.to_netcdf('/tmp/test')
File "/home/volker/workspace/eprofile_wind-CRxNsezQ/lib/python3.5/site-packages/xarray/core/dataset.py", line 1232, in to_netcdf
compute=compute)
File "/home/volker/workspace/eprofile_wind-CRxNsezQ/lib/python3.5/site-packages/xarray/backends/api.py", line 747, in to_netcdf
unlimited_dims=unlimited_dims)
File "/home/volker/workspace/eprofile_wind-CRxNsezQ/lib/python3.5/site-packages/xarray/backends/api.py", line 790, in dump_to_store
unlimited_dims=unlimited_dims)
File "/home/volker/workspace/eprofile_wind-CRxNsezQ/lib/python3.5/site-packages/xarray/backends/common.py", line 261, in store
variables, attributes = self.encode(variables, attributes)
File "/home/volker/workspace/eprofile_wind-CRxNsezQ/lib/python3.5/site-packages/xarray/backends/common.py", line 347, in encode
variables, attributes = cf_encoder(variables, attributes)
File "/home/volker/workspace/eprofile_wind-CRxNsezQ/lib/python3.5/site-packages/xarray/conventions.py", line 605, in cf_encoder
for k, v in iteritems(variables))
File "/home/volker/workspace/eprofile_wind-CRxNsezQ/lib/python3.5/site-packages/xarray/conventions.py", line 605, in <genexpr>
for k, v in iteritems(variables))
File "/home/volker/workspace/eprofile_wind-CRxNsezQ/lib/python3.5/site-packages/xarray/conventions.py", line 241, in encode_cf_variable
var = ensure_dtype_not_object(var, name=name)
File "/home/volker/workspace/eprofile_wind-CRxNsezQ/lib/python3.5/site-packages/xarray/conventions.py", line 201, in ensure_dtype_not_object
data = _copy_with_dtype(data, dtype=_infer_dtype(data, name))
File "/home/volker/workspace/eprofile_wind-CRxNsezQ/lib/python3.5/site-packages/xarray/conventions.py", line 139, in _infer_dtype
.format(name))
ValueError: unable to infer dtype on variable 'data_bins'; xarray cannot serialize arbitrary Python objects
Expected Output
The Dataset should be written to file in netcdf
Output of xr.show_versions()
INSTALLED VERSIONS
commit: None
python: 3.5.3 (default, Sep 27 2018, 17:25:39)
[GCC 6.3.0 20170516]
python-bits: 64
OS: Linux
OS-release: 4.9.0-8-amd64
machine: x86_64
processor:
byteorder: little
LC_ALL: None
LANG: de_DE.UTF-8
LOCALE: de_DE.UTF-8
libhdf5: 1.10.2
libnetcdf: 4.4.1.1
xarray: 0.11.3
pandas: 0.24.1
numpy: 1.16.1
scipy: None
netCDF4: 1.4.2
pydap: None
h5netcdf: None
h5py: None
Nio: None
zarr: None
cftime: 1.0.2.1
PseudonetCDF: None
rasterio: None
cfgrib: None
iris: None
bottleneck: None
cyordereddict: None
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the groupby_bins('data', bins).mean() result and follow out.to_netcdf() into the serialization path shown in the traceback, especially the data_bins variable. Confirm the failure with the provided example and make the resulting Dataset writable as NetCDF without arbitrary Python objects.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- numpy, python
- Domain
- backend, data
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100