Allow access to netCDF-4 files in S3 object stores
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 150
- Forks
- 23
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 2
Description
Currently netCDF files are only opened and read by the netCDF4 library. However, netCDF4 can't read files in S3 object stores.
h5netcdf, on the other hand can, access S3 netCDF-4 files (but not netCDF-3).
A new default behaviour for an arbitrary file should be to try netCDF4, and fall back on h5netcdf if that doesn't work. There should also be the option to cf.read to specifically choose one or other of the libraries.
Access to an S3 object store may require credentials, which will need to be passed in via a new cf.read argument.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing the cf.read entry point and how it currently selects the netCDF4 library. Define the behavior for automatic fallback, explicit library selection, and credentials passed to S3 access. Done means netCDF-4 files in S3 can be read through h5netcdf while existing netCDF-3 access remains supported.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- cloud, data
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100