awslabs / awslabs/open-data-docs

Partial https downloading using egrep produces different results to NCEP servers

Open
#16 0 comments 0 reactions 1 assignee Claimed by @cstner View on GitHub
Dominant language
Jupyter Notebook
Stars
180
Forks
32
PR merge metrics
No merged PRs in 30d

Description

Hi,

When making an egep call using with get_inv.pl and get_grib.pl the AWS GEFS returns different data to NOMADS.

For example:

:(TMP:850 mb|HGT:500 mb|PRMSL|CSNOW.(surface)|APCP|TMAX.(2 m above ground)|TMIN.(2 m above ground)):

Will download the following variables:

APCPsfc 0,1,0 0,1,8,1 ** surface Total Precipitation [kg/m^2]
CAPE180_0mb 0,108,18000,0 0,7,6 ** 180-0 mb above ground Convective Available Potential Energy [J/kg]
CFRZRsfc 0,1,0 0,1,193,0 ** surface Categorical Freezing Rain [-]
CICEPsfc 0,1,0 0,1,194,0 ** surface Categorical Ice Pellets [-]
CIN180_0mb 0,108,18000,0 0,7,7 ** 180-0 mb above ground Convective Inhibition [J/kg]
CRAINsfc 0,1,0 0,1,192,0 ** surface Categorical Rain [-]
CSNOWsfc 0,1,0 0,1,195,0 ** surface Categorical Snow [-]
DLWRFsfc 0,1,0 0,5,192,0 ** surface Downward Long-Wave Rad. Flux [W/m^2]
DSWRFsfc 0,1,0 0,4,192,0 ** surface Downward Short-Wave Radiation Flux [W/m^2]
HGTprs 12,100 0,3,5 ** (1000 925 850 700 500.. 250 200 100 50 10) Geopotential Height [gpm]
ICETKsfc 0,1,0 10,2,1 ** surface Ice Thickness [m]
LHTFLsfc 0,1,0 0,0,10,0 ** surface Latent Heat Net Flux [W/m^2]
PRESsfc 0,1,0 0,3,0 ** surface Pressure [Pa]
PRMSLmsl 0,101,0 0,3,1 ** mean sea level Pressure Reduced to MSL [Pa]
PWATclm 0,200,0 0,1,3 ** entire atmosphere (considered as a single layer) Precipitable Water [kg/m^2]
RHprs 12,100 0,1,1 ** (1000 925 850 700 500.. 250 200 100 50 10) Relative Humidity [%]
RH2m 0,103,2 0,1,1 ** 2 m above ground Relative Humidity [%]
SHTFLsfc 0,1,0 0,0,11,0 ** surface Sensible Heat Net Flux [W/m^2]
SNODsfc 0,1,0 0,1,11 ** surface Snow Depth [m]
SOILW0_10cm 0,106,0,0.1 2,0,192 ** 0-0.1 m below ground Volumetric Soil Moisture Content [Fraction]
TCDCclm 0,10,0 0,6,1,0 ** entire atmosphere Total Cloud Cover [%]
TMAX2m 0,103,2 0,0,4,2 ** 2 m above ground Maximum Temperature [K]
TMIN2m 0,103,2 0,0,5,3 ** 2 m above ground Minimum Temperature [K]
TMPprs 12,100 0,0,0 ** (1000 925 850 700 500.. 250 200 100 50 10) Temperature [K]
TMP2m 0,103,2 0,0,0 ** 2 m above ground Temperature [K]
TSOIL0_10cm 0,106,0,0.1 2,0,2 ** 0-0.1 m below ground Soil Temperature [K]
UGRDprs 12,100 0,2,2 ** (1000 925 850 700 500.. 250 200 100 50 10) U-Component of Wind [m/s]
UGRD10m 0,103,10 0,2,2 ** 10 m above ground U-Component of Wind [m/s]
ULWRFsfc 0,1,0 0,5,193,0 ** surface Upward Long-Wave Rad. Flux [W/m^2]
ULWRFtoa 0,8,0 0,5,193,0 ** top of atmosphere Upward Long-Wave Rad. Flux [W/m^2]
USWRFsfc 0,1,0 0,4,193,0 ** surface Upward Short-Wave Radiation Flux [W/m^2]
VGRDprs 12,100 0,2,3 ** (1000 925 850 700 500.. 250 200 100 50 10) V-Component of Wind [m/s]
VGRD10m 0,103,10 0,2,3 ** 10 m above ground V-Component of Wind [m/s]
VVEL850mb 0,100,85000 0,2,8 ** 850 mb Vertical Velocity (Pressure) [Pa/s]
WEASDsfc 0,1,0 0,1,13 ** surface Water Equivalent of Accumulated Snow Depth [kg/m^2]

On the NOMADS server the same request will return a much smaller and (I think correct) subset of variables:

APCPsfc 0,1,0 0,1,8,1 ** surface Total Precipitation [kg/m^2]
CSNOWsfc 0,1,0 0,1,195,0 ** surface Categorical Snow [-]
HGT500mb 0,100,50000 0,3,5 ** 500 mb Geopotential Height [gpm]
PRMSLmsl 0,101,0 0,3,1 ** mean sea level Pressure Reduced to MSL [Pa]
TMAX2m 0,103,2 0,0,4,2 ** 2 m above ground Maximum Temperature [K]
TMIN2m 0,103,2 0,0,5,3 ** 2 m above ground Minimum Temperature [K]
TMP850mb 0,100,85000 0,0,0 ** 850 mb Temperature [K]

It seems that egrep regex is not working correctly on the AWS servers.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.