pydata / pydata/xarray

better warning message for xarray/core/dataset.py:2060: SerializationWarning

Open
#6,840 0 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Python
Stars
4.2k
Forks
1.4k
Avg merge
2d 15h
Merged PRs (30d)
14

Description

Is your feature request related to a problem?

I was trying to store a zarr fstore and I got:

/opt/userenvs/ray.bell/autofront/lib/python3.9/site-packages/xarray/core/dataset.py:2060: SerializationWarning: saving variable None with floating point data as an integer dtype without any _FillValue to use for NaNs
  return to_zarr(  # type: ignore

What I was seeing was a time coord which had encoding 'dtype': dtype('int64'). Storing the time this way lost the HH:MM on my datetime array.
I think I am seeing 'units': 'days since 2022-MM-DD 00:00:00' whereas it should be hours for my data.

Describe the solution you'd like

The message captures the variable/coord name instead of None

It could offer a solution such as "to store floating point data you could possibly use encoding={variable/coord: {"dtype": float, "_Fillvalue": np.nan}"

Describe alternatives you've considered

No response

Additional context

I'll try to come back and write a MCVE

WIP:
you can get the warning message by doing:

ds = xr.tutorial.open_dataset("air_temperature")
ds["time"].encoding # {'units': 'hours since 1800-01-01', 'calendar': 'standard', 'dtype': dtype('float32')}
ds.isel(time=0).to_zarr("tmp0.zarr", mode="w")

/Users/ray/miniconda3/envs/main/lib/python3.9/site-packages/xarray/core/dataset.py:2060: SerializationWarning: saving variable None with floating point data as an integer dtype without any _FillValue to use for NaNs
return to_zarr( # type: ignore

My guess if one of lat, lon, time, air is raising that message

Can't find the associated code in the repo that raises this. 2060 is to_zarr AFICT (https://github.com/pydata/xarray/blob/main/xarray/core/dataset.py#L2066)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with xarray/core/dataset.py around line 2060 and trace the to_zarr call to the code that emits SerializationWarning. Run the air_temperature tutorial example in the issue, identify which variable or coordinate produces the warning, and update the warning so it names that variable instead of None; verify the resulting message and proposed encoding guidance.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.