MetOffice / MetOffice/lfric_apps

Add a demonstration of NetCDF integer packing to lfric_apps rose-stem

Open
#514 0 comments 0 reactions 0 assignees Claimed by @harry-shepherd View on GitHub
enhancement
Dominant language
Fortran
Stars
31
Forks
118
Avg merge
2d 4h
Merged PRs (30d)
8

Description

### Are there any linked Issues or Pull Requests?

_No response_

### Brief description

Integer packing appears to be a performant and effective way of reducing data volume used by the LFRic model.

A full description can be found in MetOffice/tcd-IO-performance/issues/55

**We would like to add the testing of integer packing to the LFRic apps rose-stem**

### Further details of the issue.

NetCDF allows the packing of floating point fields into integer fields. By NetCDF convention these integer fields have two double precision parameters:

- `scale_factor` (_S_)
- `offset` ($\Delta$)

We pack the floating point data $D_\text{float}$ by

$$
D_\text{packed} = \frac{D_\text{float}}{\Delta} - S
$$

and casting to either a 8 bit or 16 bit integer. To reconstruct the data, we do the reverse process.

XIOS allows us to access the packing functionality, by specifying `prec=2` (16-bit integer), and the variables `add_offset` and `scale_factor` within field references. For example

```

```
may become
```

```

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading MetOffice/tcd-IO-performance issue 55 and the open pull request #542, then inspect the lfric_apps rose-stem configuration where XIOS field references are defined. Verify how the supplied add_offset, scale_factor, prec="2", and conversion_by_netcdf settings are exercised. Done means the rose-stem testing demonstrates NetCDF integer packing for the relevant floating-point fields.

Written by the indexing model from the issue text.

Assessment

Tech stack
fortran
Domain
data, testing-qa
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.