HydrologicEngineeringCenter / HydrologicEngineeringCenter/Vortex

Vortex is importing some precip grids but not others

Open
#104 10 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Java
Stars
37
Forks
7
PR merge metrics
No merged PRs in 30d

Description

Hello,

I have been successfully using vortex (version 0.11.10-rc.1) to import ECWMF ERA5 reanalysis precip data (in both netcdf and grib format) and applying it in HECRAS rain on grid models with some nice results (big thanks to the vortex developers). I have also successfully used MRMS QPE data. However, vortex does not seem to want to import some other precip datasets that I would like to use in HECRAS, namely:

  • It will not import this precip data from the UK’s national centre for atmospheric science (NCAS):
<xarray.Dataset>
Dimensions:            (ny: 1200, nx: 1500, pal01_colors: 256, pal_RGB: 3, pal02_colors: 256)
Coordinates:
    lat                (ny, nx) float32 ...
    lon                (ny, nx) float32 ...
  * ny                 (ny) float32 1.527e+06 1.524e+06 ... -2.067e+06 -2.07e+06
  * nx                 (nx) float32 3.961e+05 3.991e+05 ... 4.891e+06 4.894e+06
Dimensions without coordinates: pal01_colors, pal_RGB, pal02_colors
Data variables:
    crr                (ny, nx) float32 ...
    crr_intensity      (ny, nx) float32 ...
    crr_accum          (ny, nx) float32 ...
    crr_status_flag    (ny, nx) float32 ...
    crr_conditions     (ny, nx) float32 ...
    crr_quality        (ny, nx) float32 ...
    crr_pal            (pal01_colors, pal_RGB) uint8 ...
    crr_intensity_pal  (pal02_colors, pal_RGB) uint8 ...
Attributes: (12/46)
    Conventions:                  CF-1.6
    title:                        NWC GEO Convective Rainfall Rate Product
    history:                      2021-08-11T00:03:04Z ncas-sat Product Creat...
    institution:                  National Centre For Atmospheric Science
    source:                       NWC/GEO version v2018
    comment:                      Copyright 2021, EUMETSAT, All Rights Reserved
    ...                           ...
    product_quality:              100.0
    product_completeness:         100.0
    geospatial_lat_max:           15.47179
    geospatial_lat_min:           -21.608143
    geospatial_lon_max:           67.83071
    geospatial_lon_min:           3.561473
  • it will not import this precip data from ECWMF’s HRES forecast model:
<xarray.Dataset>
Dimensions:              (isobaricInhPa: 9, latitude: 451, longitude: 900)
Coordinates:
    time                 datetime64[ns] ...
    step                 timedelta64[ns] ...
  * isobaricInhPa        (isobaricInhPa) float64 1e+03 925.0 ... 200.0 50.0
  * latitude             (latitude) float64 90.0 89.6 89.2 ... -89.2 -89.6 -90.0
  * longitude            (longitude) float64 -180.0 -179.6 ... 179.2 179.6
    valid_time           datetime64[ns] ...
    depthBelowLandLayer  float64 ...
    surface              float64 ...
    meanSea              float64 ...
    heightAboveGround    float64 ...
    entireAtmosphere     float64 ...
Data variables: (12/18)
    t                    (isobaricInhPa, latitude, longitude) float32 ...
    st                   (latitude, longitude) float32 ...
    v                    (isobaricInhPa, latitude, longitude) float32 ...
    u                    (isobaricInhPa, latitude, longitude) float32 ...
    tp                   (latitude, longitude) float32 ...
    gh                   (isobaricInhPa, latitude, longitude) float32 ...
    ...                   ...
    tcwv                 (latitude, longitude) float32 ...
    q                    (isobaricInhPa, latitude, longitude) float32 ...
    lsm                  (latitude, longitude) float32 ...
    d                    (isobaricInhPa, latitude, longitude) float32 ...
    vo                   (isobaricInhPa, latitude, longitude) float32 ...
    ro                   (latitude, longitude) float32 ...
Attributes:
    GRIB_edition:            2
    GRIB_centre:             ecmf
    GRIB_centreDescription:  European Centre for Medium-Range Weather Forecasts
    GRIB_subCentre:          0
    Conventions:             CF-1.7
    institution:             European Centre for Medium-Range Weather Forecasts
    history:                 2023-11-23T17:50 GRIB to CDM+CF via cfgrib-0.9.1...

Vortex is either creating an empty output file or gets stuck during the import process. I am quite new to gridded climate datasets and cannot see anything that might be causing vortex an issue. Do you have any ideas as to why vortex is not handling these two datasets, and how you might process them to make them compatible? Below are screenshots of the parameters that are visible when first loading the datasets into vortex.

ECMWF HRES variables in vortex NCAS variables in vortex

Many thanks,
Andy

ps. for info, this is the ERA5 reanalysis data that vortex will work with:

Dimensions:    (longitude: 5, latitude: 5, time: 35064)
Coordinates:
  * longitude  (longitude) float32 38.5 38.75 39.0 39.25 39.5
  * latitude   (latitude) float32 9.5 9.25 9.0 8.75 8.5
  * time       (time) datetime64[ns] 2017-01-01 ... 2020-12-31T23:00:00
Data variables:
    e          (time, latitude, longitude) float32 ...
    tp         (time, latitude, longitude) float32 ...
Attributes:
    Conventions:  CF-1.6
    history:      2022-01-21 20:00:46 GMT by grib_to_netcdf-2.23.0: /opt/ecmw...

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No source files, tests, or entry points are named. Start by reproducing the import with the NCAS and ECMWF HRES datasets, compare them with the working ERA5 and MRMS inputs, and define done as a completed import that produces a non-empty output.

Written by the indexing model from the issue text.

Assessment

Domain
data
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.