Parallel access to DataArray within `with` statement causes `BlockingIOError`

What happened:

My general usage is

1. Read one DataArray from an existing dataset within a with statement, so the file closes at the end
2. Run it through some functions, sometimes in parallel
3. after closing the dataset, append to the dataset in a new DataArray with .to_netcdf(existing_file, engine="h5netcdf").

With the setup below, I get

BlockingIOError: [Errno 11] Unable to create file (unable to lock file, errno = 11, error message = 'Resource temporarily unavailable')

It's entirely possible (and likely) that it's an issue with some other library that I'm using/that xarray is using... but I thought I someone might have an idea why a very similar versions of the same script succeeds, while the first one fails.

What you expected to happen:
No error, which happens for the 2nd version

Minimal Complete Verifiable Example:

For this version, I'm using pymp, which I'd rather not include in the MCVE, but i've had similar issues jsut using the python multiprocessing. I just wanted to post this one first.

import xarray as xr
import numpy as np
import pymp


def dummy_function_parallel(stack):
    out = np.zeros(stack.shape, dtype=np.float32)
    # Also fails:
    # out = pymp.shared.array(stack.shape, dtype=np.float32)
    with pymp.Parallel(4) as p:
        for i in p.range(3):
            out[:, i] = stack[:, i] * 3
    return out

# Example of a fucntion that *doesn't* cause a failure ever
def dummy_function2(stack):
    return 2 * stack


if __name__ == "__main__":
    x, y, z = np.arange(3), np.arange(3), np.arange(3)
    data = np.random.rand(3, 3, 3)
    da = xr.DataArray(data, dims=["z", "y", "x"], coords={"x": x, "y": y, "z": z})
    da.to_dataset(name="testdata").to_netcdf("testdata.nc", engine="h5netcdf")

    with xr.open_dataset("testdata.nc") as ds:
        da = ds["testdata"]
        newstack = dummy_function_parallel(da.values)

        # This function does work without the parallel stuff
        # newstack = dummy_function2(da.values) 

        da_new = xr.DataArray(newstack, coords=da.coords, dims=da.dims)

    da_new.to_dataset(name="new_testdata").to_netcdf("testdata.nc", engine="h5netcdf")

Running this causes the following traceback

Traceback (most recent call last):
  File "/home/scott/miniconda3/envs/mapping/lib/python3.8/site-packages/xarray/backends/file_manager.py", line 199, in _acquire_with_cache_info
    file = self._cache[self._key]
  File "/home/scott/miniconda3/envs/mapping/lib/python3.8/site-packages/xarray/backends/lru_cache.py", line 53, in __getitem__
    value = self._cache[key]
KeyError: [<class 'h5netcdf.core.File'>, ('/data4/scott/path85/stitched/top_strip/igrams/testdata.nc',), 'a', (('decode_vlen_strings', True), ('invalid_netcdf', None))]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/scott/repos/insar/insar/testxr.py", line 46, in <module>
    da_new.to_dataset(name="new_testdata").to_netcdf("testdata.nc", engine="h5netcdf")
  File "/home/scott/miniconda3/envs/mapping/lib/python3.8/site-packages/xarray/core/dataset.py", line 1900, in to_netcdf
    return to_netcdf(
  File "/home/scott/miniconda3/envs/mapping/lib/python3.8/site-packages/xarray/backends/api.py", line 1060, in to_netcdf
    store = store_open(target, mode, format, group, **kwargs)
  File "/home/scott/miniconda3/envs/mapping/lib/python3.8/site-packages/xarray/backends/h5netcdf_.py", line 178, in open
    return cls(manager, group=group, mode=mode, lock=lock, autoclose=autoclose)
  File "/home/scott/miniconda3/envs/mapping/lib/python3.8/site-packages/xarray/backends/h5netcdf_.py", line 123, in __init__
    self._filename = find_root_and_group(self.ds)[0].filename
  File "/home/scott/miniconda3/envs/mapping/lib/python3.8/site-packages/xarray/backends/h5netcdf_.py", line 189, in ds
    return self._acquire()
  File "/home/scott/miniconda3/envs/mapping/lib/python3.8/site-packages/xarray/backends/h5netcdf_.py", line 181, in _acquire
    with self._manager.acquire_context(needs_lock) as root:
  File "/home/scott/miniconda3/envs/mapping/lib/python3.8/contextlib.py", line 113, in __enter__
    return next(self.gen)
  File "/home/scott/miniconda3/envs/mapping/lib/python3.8/site-packages/xarray/backends/file_manager.py", line 187, in acquire_context
    file, cached = self._acquire_with_cache_info(needs_lock)
  File "/home/scott/miniconda3/envs/mapping/lib/python3.8/site-packages/xarray/backends/file_manager.py", line 205, in _acquire_with_cache_info
    file = self._opener(*self._args, **kwargs)
  File "/home/scott/miniconda3/envs/mapping/lib/python3.8/site-packages/h5netcdf/core.py", line 712, in __init__
    self._h5file = h5py.File(path, mode, **kwargs)
  File "/home/scott/miniconda3/envs/mapping/lib/python3.8/site-packages/h5py/_hl/files.py", line 442, in __init__
    fid = make_fid(name, mode, userblock_size,
  File "/home/scott/miniconda3/envs/mapping/lib/python3.8/site-packages/h5py/_hl/files.py", line 201, in make_fid
    fid = h5f.create(name, h5f.ACC_TRUNC, fapl=fapl, fcpl=fcpl)
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "h5py/h5f.pyx", line 116, in h5py.h5f.create
BlockingIOError: [Errno 11] Unable to create file (unable to lock file, errno = 11, error message = 'Resource temporarily unavailable')

Anything else we need to know?:

The weird part to me: If I change the end of the script so that the function runs after the with statement exits (so I'm passing the DataArray reference from a closed dataset), there's never an error. Indeed, that's how I fixed this for my real, longer script.

    with xr.open_dataset("testdata.nc") as ds:
        da = ds["testdata"]
    # Now after it closed, this parallel function doesn't cause the hangup
    newstack = dummy_function_parallel(da.values)
    da_new = xr.DataArray(newstack, coords=da.coords, dims=da.dims)
    da_new.to_dataset(name="new_testdata").to_netcdf("testdata.nc", engine="h5netcdf")

Environment:

Reproduce the supplied script with the parallel function inside and outside the with statement, then trace Dataset.to_netcdf through xarray/backends/h5netcdf_.py and backends/file_manager.py. Compare the file lifetime and locking behavior in both cases; done means the reported BlockingIOError is explained and the failing scenario has a verified fix or regression test.