google-deepmind / google-deepmind/weathernext

Constructing datasets from CDS products

Open
#22 6 comments 7 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
7.7k
Forks
986
PR merge metrics
No merged PRs in 30d

Description

My understanding is that each input data file is constructed from a couple of CDS data products for 'atmospheric' and 'single-level' variables. For example, the two queries below pull the required data (for a specific date), as I understand it:

```python
import cdsapi

c = cdsapi.Client()

c.retrieve(
'reanalysis-era5-pressure-levels',
{
'product_type': 'reanalysis',
'format': 'netcdf',
'variable': [
'geopotential', 'specific_humidity', 'temperature',
'u_component_of_wind', 'v_component_of_wind', 'vertical_velocity',
],
'year': '2023',
'month': '11',
'day': '01',
'time': [
'00:00', '06:00',
],
'pressure_level': [
'1', '2', '3',
'5', '7', '10',
'20', '30', '50',
'70', '100', '125',
'150', '175', '200',
'225', '250', '300',
'350', '400', '450',
'500', '550', '600',
'650', '700', '750',
'775', '800', '825',
'850', '875', '900',
'925', '950', '975',
'1000',
],
},
'atmospheric.nc')

c.retrieve(
'reanalysis-era5-single-levels',
{
'product_type': 'reanalysis',
'variable': [
'10m_u_component_of_wind', '10m_v_component_of_wind', '2m_temperature',
'geopotential', 'land_sea_mask', 'mean_sea_level_pressure',
'toa_incident_solar_radiation', 'total_precipitation',
],
'year': '2023',
'month': '11',
'day': '01',
'time': [
'00:00', '06:00',
],
'format': 'netcdf',
},
'single-level.nc')
```

Presumably, these files are then spliced together to form an input file. While I'm able to combine the files easily enough, there are subtleties that I'm clearly messing up, like constructing the `datetime` and `batch` coordinates, which seem to be mandatory (e.g., `ValueError: 'datetime' must be in data coordinates.`)

Do you plan to publish a script or instructions for constructing input files from these data products in the format expected by the model? It would be a fantastic help for applying the model!

Many thanks in advance,
Dan

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.