open_mfdataset parallel=True failing with netcdf4 >= 1.6.1
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 4.2k
- Forks
- 1.4k
- Avg merge
- 2d 15h
- Merged PRs (30d)
- 14
Description
What happened?
When using the parallel=True key, open_mfdataset fails with NetCDF: Unknown file format. Running the same command again (with try+except), or with parallel=False executes as expected.
works:
xr.open_mfdataset(dirpath +'\\*.nc', parallel=False)
works:
try:
xr.open_mfdataset(dirpath +'\\*.nc', parallel=True)
except:
xr.open_mfdataset(dirpath +'\\*.nc', parallel=True)
fails:
xr.open_mfdataset(dirpath +'\\*.nc', parallel=True)
[Errno -51] NetCDF: Unknown file format
all with engine='netcdf4'
any help is highly appreciated as I'm a bit lost how to investigate this further.
What did you expect to happen?
No response
Minimal Complete Verifiable Example
No response
MVCE confirmation
- Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
- Complete example — the example is self-contained, including all data and the text of any traceback.
- Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
- New issue — a search of GitHub Issues suggests this is not a duplicate.
Relevant log output
No response
Anything else we need to know?
No response
Environment
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the failure with xarray.open_mfdataset using parallel=True and engine='netcdf4', then compare it with parallel=False and the retry behavior described in the issue. The issue provides no file, test, traceback, or environment details, so done would require identifying the cause and adding a regression test that covers the failing parallel case.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100