Update a small slice of a large netcdf file without overwriting the entire file.

MCVE Code Sample

# Your code here
orig = '/tmp/orig.h5'

ii = 100000

data = xr.Dataset(
    {
        'x':('t',np.random.randn(ii)),
        'y':('t',np.random.randn(ii))
    },
    coords={'t':range(ii)}
)

# function to save the large file usnig chunksizes
def save(ds,path,**kwargs):
    dvars = ds.variables
    chunksize = 100
    var_dic = {}
    for var in dvars:
        var_dic[var]={
                'chunksizes': (chunksize,)
            }
    delayed =ds.to_netcdf(path,encoding=var_dic,**kwargs)

save(data,orig)

data.close()

#open the file, using dask
data_1 = xr.open_mfdataset([orig],chunks={'t':100})

#Change variable x 
data_1['x']=data_1['x']+20
data_1.close()
#update only variable x. This works! 
data_1['x'].to_netcdf(orig,mode='a')


# try the same but now update only a slice of the x variable

#open the file, using dask
data_1 = xr.open_mfdataset(orig,chunks={'t':100})

#Change variable x 
data_1['x']=data_1['x']+20
data_1.close()
#update only variable x. this doesnt work!
data_1['x'][{'t':slice(0,10)}].to_netcdf(orig,mode='a')

Expected Output

Problem Description

Hi,
I have a large dataset that does not fit in memory.
Lets say i only want to update a small portion of it. Is there any way to update this small portion without having to rewrite the entire file.

I was fiddling around and found a way to update one variable at a time, but i want to be able to update only a subsection of this variable

Output of xr.show_versions()

Start with the MCVE's Dataset.to_netcdf calls, especially the indexed DataArray slice written with mode='a', and inspect how open_mfdataset and the netCDF4 backend handle writes. Done means updating only the requested slice of a large netCDF variable without rewriting the entire file, with coverage for the demonstrated case.