Make NetCDF output conform to spec
- Dominant language
- HTML
- Stars
- 1
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Description
## Summary
* The shared InMAP NetCDF content (e.g. [ca_isrm.ncf]) appears to be invalid
* `ncvalidator` states that it fails to conform with classic CDF-2 file format spec
* It contains 5 large fixed-size variables, while the CDF-2 format allows only one
* Fresh NCO install on OS X won't read it (issue #4)
* Fixing this would be nice
## Details
Although `ncdump` gives up:
```sh
dholstius$ NC_PATH="/Users/dholstius/GitHub/BAAQMD/InMAP-SFAB/Data/GDrive/ca_isrm.nc"
dholstius$ ncdump -k $NC_PATH
ncdump: /Users/dholstius/GitHub/BAAQMD/InMAP-SFAB/Data/GDrive/ca_isrm.nc: NetCDF: One or more variable sizes violate format constraints
```
... we can fall back to reading the first four bytes:
```sh
dholstius$ od -An -c -N4 $NC_PATH
C D F 002
```
Those indicate CDF-2 (64-bit offset) format.
After `brew install pnetcdf`:
```sh
dholstius$ ncvalidator $NC_PATH
Error:
Input file contains 5 large fixed-size variables
CDF-2 format allows only one large fixed-size variable
The 1st large fixed-size variable is PrimaryPM25
The 2nd large fixed-size variable is SOA
See: http://www.unidata.ucar.edu/software/netcdf/docs/file_structure_and_performance.html#offset_format_limitations
File "/Users/dholstius/GitHub/BAAQMD/InMAP-SFAB/Data/GDrive/ca_isrm.nc" fails to conform with classic CDF-2 file format specifications
```
## Related Issues
- #4
[ca_isrm.ncf]: https://drive.google.com/drive/folders/1WmLRz7iWo2MjtSikgHEig7M0NvK2sOns?usp=sharing
Contributor guide
No contributing guide indexed for this repository
Research direction
Obtain the shared ca_isrm.nc file and run ncvalidator and ncdump to reproduce the CDF-2 errors described in the issue. Read the CDF-2 format limitations and related issue #4 first. Done means the NetCDF output conforms to the CDF-2 specification and can be read by a fresh NCO installation on OS X.
Written by the indexing model from the issue text.
Assessment
- Domain
- data
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100