to_zarr writes unexpected NaNs with chunks=-1

What happened?

I'm running into some unexpected behavior with ds.to_zarr when my encoding includes chunks=-1 and ds is a dataset that I created by operating on zarr files opened from disk. When I run the following example code, many of my values are NaN. When I run the same code, but with ds.load() before ds.to_zarr(), the correct, non-NaN values are saved.

What did you expect to happen?

My data would be written the same regardless of whether I explicitly loaded the dataset.
The documentation for xarray.Dataset.load includes the following:

Normally, it should not be necessary to call this method in user code, because all xarray functions should either work on deferred data or load data automatically. However, this method can be necessary when working with many file objects on disk.

I encountered this situation when operating on datasets that had been loaded from disk (.sel, then concat), so this seems like a situation that the second sentence addresses, but I did not expect it to silently fail to write the correct data in the way that it did.

Minimal Complete Verifiable Example

import pandas as pd
import xarray as xr
import numpy as np

def create_dataset(time, site):
  temperature = 15 + 8 * np.random.randn(1, 3)
  precipitation = 10 * np.random.rand(1, 3)

  ds = xr.Dataset(
      data_vars=dict(
          temperature=(["site", "time"], temperature),
          precipitation=(["site", "time"], precipitation),
      ),

      coords=dict(
          site=site,
          time=time,
      ),
      attrs=dict(description="Weather related data."),
  )
  return ds

time_1 = pd.date_range("2014-09-06", periods=3)
time_2 = pd.date_range("2014-09-09", periods=3)

# create and save the first dataset as a zarr
ds_a = create_dataset(time_1, ["site_1"])
fname_a = '/tmp/ds_a.zarr'
ds_a.to_zarr(fname_a, mode='w')
ds_a_from_disk = xr.open_dataset(fname_a, engine='zarr', chunks={})

# create and save the second dataset as a zarr
ds_b = create_dataset(time_2, ["site_1"])
fname_b = '/tmp/ds_b.zarr'
ds_b.to_zarr(fname_b, mode='w')
ds_b_from_disk = xr.open_dataset(fname_b, engine='zarr', chunks={})

# concatenate the datasets
ds = xr.concat([ds_a_from_disk.sel(site="site_1"), ds_b_from_disk.sel(site="site_1")], dim='time')

# save all data in one chunk
encoding = {var: {'chunks': -1} for var in list(ds) + list(ds.coords)}
fname = '/tmp/concated.zarr'

# Uncomment the following line to fix this issue
# ds.load()

# save the dataset
ds.to_zarr(fname, mode='w', encoding=encoding)

ds_from_disk = xr.open_dataset(fname, engine='zarr')
print(ds_from_disk.to_dataframe())

MVCE confirmation

• Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
• Complete example — the example is self-contained, including all data and the text of any traceback.
• Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
• New issue — a search of GitHub Issues suggests this is not a duplicate.

Relevant log output

No response

Anything else we need to know?

Example output without ds.load():

            precipitation    site  temperature
time                                          
2014-09-06            NaN  site_1     9.805297
2014-09-07            NaN  site_1    16.119194
2014-09-08            NaN  site_1     4.226150
2014-09-09       7.275470  site_1          NaN
2014-09-10       2.899134  site_1          NaN
2014-09-11       5.777094  site_1          NaN

Example output with ds.load():

            precipitation    site  temperature
time                                          
2014-09-06       3.445305  site_1    18.144503
2014-09-07       7.708728  site_1    20.289742
2014-09-08       7.358939  site_1    19.996060
2014-09-09       6.211692  site_1     9.748291
2014-09-10       4.981796  site_1    -7.676436
2014-09-11       8.667885  site_1    31.934328

My hunch is that this has to do with a mismatch between Dask chunks in the unloaded dataset and the chunks specified in to_zarr, but if they are incompatible I would expect to see an error surfaced.

Environment

Start by running the provided MVCE and compare ds.to_zarr with and without ds.load(). Inspect the deferred data from xr.open_dataset(..., engine='zarr', chunks={}), xr.concat, and the encoding that sets chunks=-1. Done means reopening the written dataset preserves all original values without requiring ds.load(); the issue names no source files or tests.