Handling of non-string dimension names
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 4.2k
- Forks
- 1.4k
- Avg merge
- 2d 15h
- Merged PRs (30d)
- 14
Description
While working on a pull request (#5149) for #5146 I came across an inconsistency in allowed dimension names. If I try and create a DataArray with a non-string dimension, I get a TypeError:
>>> import xarray as xr
>>> da = xr.DataArray(np.ones((5, 5)), dims=[1, "y"])
...
TypeError: dimension 1 is not a string
But creating it with a string and renaming it works:
>>> da = xr.DataArray(np.ones((5, 5)), dims=["x", "y"]).rename(x=1)
>>> da
<xarray.DataArray (1: 5, y: 5)>
array([[1., 1., 1., 1., 1.],
[1., 1., 1., 1., 1.],
[1., 1., 1., 1., 1.],
[1., 1., 1., 1., 1.],
[1., 1., 1., 1., 1.]])
Dimensions without coordinates: 1, y
I can create a dataset via this renaming, but trying to get the repr value fails as xarray.core.utils.SortedKeysDict tries to sort it and cannot compare the string dimension to the int dimension:
>>> import xarray as xr
>>> ds = xr.Dataset({"test": xr.DataArray(np.ones((5, 5)), dims=["x", "y"]).rename(x=1)})
>>> ds
...
~/software/external/xarray/xarray/core/formatting.py in dataset_repr(ds)
519
520 dims_start = pretty_print("Dimensions:", col_width)
--> 521 summary.append("{}({})".format(dims_start, dim_summary(ds)))
522
523 if ds.coords:
~/software/external/xarray/xarray/core/formatting.py in dim_summary(obj)
422
423 def dim_summary(obj):
--> 424 elements = [f"{k}: {v}" for k, v in obj.sizes.items()]
425 return ", ".join(elements)
426
~/software/external/xarray/xarray/core/formatting.py in <listcomp>(.0)
422
423 def dim_summary(obj):
--> 424 elements = [f"{k}: {v}" for k, v in obj.sizes.items()]
425 return ", ".join(elements)
426
/usr/lib/python3.9/_collections_abc.py in __iter__(self)
847
848 def __iter__(self):
--> 849 for key in self._mapping:
850 yield (key, self._mapping[key])
851
~/software/external/xarray/xarray/core/utils.py in __iter__(self)
437
438 def __iter__(self) -> Iterator[K]:
--> 439 return iter(self.mapping)
440
441 def __len__(self) -> int:
~/software/external/xarray/xarray/core/utils.py in __iter__(self)
504 def __iter__(self) -> Iterator[K]:
505 # see #4571 for the reason of the type ignore
--> 506 return iter(sorted(self.mapping)) # type: ignore[type-var]
507
508 def __len__(self) -> int:
TypeError: '<' not supported between instances of 'str' and 'int'
The same thing happens if I call rename on the dataset rather than the array it is initialised with.
If the initialiser requires the dimension names to be strings, and other code (which includes the HTML formatter I was looking at when I found this) assume that they are, then rename and any other method which can alter dimension names should also enforce the string requirement.
Environment:
Output of xr.show_versions()
INSTALLED VERSIONS
commit: 851d85b9203b49039237b447b3707b270d613db5
python: 3.9.2 (default, Feb 20 2021, 18:40:11)
[GCC 10.2.0]
python-bits: 64
OS: Linux
OS-release: 5.11.13-arch1-1
machine: x86_64
processor:
byteorder: little
LC_ALL: None
LANG: en_NZ.UTF-8
LOCALE: en_NZ.UTF-8
libhdf5: 1.12.0
libnetcdf: 4.7.4
xarray: 0.17.0
pandas: 1.2.3
numpy: 1.20.1
scipy: 1.6.2
netCDF4: 1.5.6
pydap: None
h5netcdf: 0.10.0
h5py: 3.2.1
Nio: None
zarr: None
cftime: 1.4.1
nc_time_axis: None
PseudoNetCDF: None
rasterio: 1.2.2
cfgrib: None
iris: None
bottleneck: 1.3.2
dask: 2021.03.0
distributed: 2021.03.0
matplotlib: 3.4.1
cartopy: 0.18.0
seaborn: 0.11.1
numbagg: None
pint: None
setuptools: 54.2.0
pip: 20.3.1
conda: None
pytest: 6.2.3
IPython: 7.22.0
sphinx: 3.5.4
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with dimension validation in the DataArray initializer and the rename paths. Inspect formatting.py's dim_summary and utils.py's SortedKeysDict, which currently fail when string and integer dimension names are mixed. Done means dimension names are handled consistently: invalid non-string names are rejected by rename paths, and dataset repr no longer fails for this case.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100