set encoding parameters in addition to the original encoding
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 4.2k
- Forks
- 1.4k
- Avg merge
- 2d 15h
- Merged PRs (30d)
- 14
Description
Is your feature request related to a problem?
When writing to disk with to_netcdf, the encoding argument causes existing encoding to be dropped. This is described in the docs.
What is a good approach to add encoding parameters in addition to the original encoding? e.g.
import rioxarray
import xarray as xr
import numpy as np
# make some random dummy netcdf file
data = np.random.rand(4, 4)
lat = np.linspace(10, 20, 4)
lon = np.linspace(10, 20, 4)
ds = xr.Dataset({"dummy": (["lat", "lon"], data)}, coords={"lat": lat, "lon": lon})
ds.rio.set_spatial_dims("lon", "lat", inplace=True)
ds.rio.write_crs("EPSG:4326", inplace=True)
# note the spatial_ref coordinate
print(ds.dummy)
<xarray.DataArray 'dummy' (lat: 4, lon: 4)> Size: 128B
...
Coordinates:
* lat (lat) float64 32B 10.0 13.33 16.67 20.0
* lon (lon) float64 32B 10.0 13.33 16.67 20.0
spatial_ref int64 8B 0
ds.to_netcdf("test.nc", mode="w")
# read it back in - ok
ds2 = xr.open_dataset("test.nc", decode_coords="all")
print(ds2.dummy)
<xarray.DataArray 'dummy' (lat: 4, lon: 4)> Size: 128B
...
Coordinates:
* lat (lat) float64 32B 10.0 13.33 16.67 20.0
* lon (lon) float64 32B 10.0 13.33 16.67 20.0
spatial_ref int64 8B ...
# now compress
ds2.to_netcdf("test_compressed.nc", mode="w", encoding={"dummy": {"compression": "zstd"}})
# read it back in - drops the spatial_ref
ds3 = xr.open_dataset("test_compressed.nc", decode_coords="all")
print(ds3.dummy)
<xarray.DataArray 'dummy' (lat: 4, lon: 4)> Size: 128B
...
Coordinates:
* lat (lat) float64 32B 10.0 13.33 16.67 20.0
* lon (lon) float64 32B 10.0 13.33 16.67 20.0
this is because rioxarray stores "grid_mapping" in the encoding.
so what is a nice generic way to specify encoding in addition to the original encoding?
encoding = ds2.dummy.encoding.copy()
encoding["compression"] = "zstd"
ds2.to_netcdf("test_compressed_2.nc", mode="w", encoding={"dummy": encoding})
ValueError: unexpected encoding parameters for 'netCDF4' backend: ['szip', 'zstd', 'bzip2', 'blosc']. Valid encodings are: ...
It seems not possible to pass the original encoding back in (even unmodified) due to additional checks
Describe the solution you'd like
in to_netcdf() be able to specify encoding in addition to the original encoding
Describe alternatives you've considered
No response
Additional context
No response
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in xarray/backends/api.py at the additional encoding-parameter checks linked in the issue, then reproduce the rioxarray example with to_netcdf. The change is complete when callers can add compression settings without losing the original grid_mapping encoding or triggering invalid-backend-parameter errors.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend-api-design, data
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100