A suggested solution to the `TypeError: Invalid value for attr:` error upon `.to_netcdf`
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 4.2k
- Forks
- 1.4k
- Avg merge
- 2d 15h
- Merged PRs (30d)
- 14
Description
Even though netcdf conventions don't allow some data types in the attributes, it might be usefull to simply serialize those as strings rather than throw an error. Maybe add a force_serialization keyword argument to the .to_netcdf method.
Example:
Setup a DataArray with bad values in the attributes:
import numpy as np
from xarray import DataArray, load_dataset
from pandas import Timestamp
from numbers import Number
valid_types = (str, Number, np.ndarray, np.number, list, tuple)
da = DataArray(
name='bad_values',
attrs=dict(
bool_value=True,
none_value=None,
datetime_value=Timestamp.now()
)
)
ds = da.to_dataset()
ds.bad_values.attrs
Output:
{'bool_value': True,
'none_value': None,
'datetime_value': Timestamp('2020-02-03 10:53:02.350105')}
The code in the except clause can be easily impolemented under _validate_attrs.
try:
ds.to_netcdf('test.nc')
# Fails with TypeError: Invalid value for attr: ...
except TypeError as e:
print(e.__class__.__name__, e)
for variable in ds.variables.values():
for k, v in variable.attrs.items():
if not isinstance(v, valid_types) or isinstance(v, bool):
variable.attrs[k] = str(v)
ds.to_netcdf('test.nc') # Works as expected
ds_from_file = load_dataset('test.nc')
ds_from_file.bad_values.attrs
Output:
TypeError Invalid value for attr: None must be a number, a string, an ndarray or a list/tuple of numbers/strings for serialization to netCDF files
{'bool_value': 'True',
'none_value': 'None',
'datetime_value': '2020-02-03 10:43:38.479866'}
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by running the provided DataArray and to_netcdf example to reproduce the invalid-attribute error. Then read _validate_attrs in xarray/backends/api.py and determine the intended force_serialization behavior, including the round-trip result shown in the issue; the issue does not name a test file.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- numpy, pandas, python
- Domain
- data
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100