Allow us to create space and later write to an existing NetCDF file object
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 4.2k
- Forks
- 1.4k
- Avg merge
- 2d 15h
- Merged PRs (30d)
- 14
Description
What is your issue?
I'm writing an MPI enabled Xarray accessor and one of the caveats with HDF5/NetCDF in parallel is that the metadata and space inside the file needs to be created up front and collectively by all ranks in the communicator. After creation those ranks can then do non-collective write operations to disk.
Does xarray have this capability somewhere already? i.e. something like the following.
f = netcdf_file_object # open the netcdf file
# x is a Dataarray or Dataset
x.create_netcdf(f)
x.write_netcdf(f)
Cheers!
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing xarray's existing NetCDF writing API and the MPI, HDF5, and NetCDF workflow described in the issue. Determine whether the current API accepts an existing file object and separates collective metadata or space creation from later writes; done should be a documented, tested capability if the design is supported.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100