PyPSA / PyPSA/atlite

ERA5 weather variables have incorrect units and names

Open
#509 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

type: bug
Dominant language
Python
Stars
402
Forks
134
PR merge metrics
No merged PRs in 30d

Description

Version Checks (indicate both or one)
  • I have confirmed this bug exists on the lastest release of atlite.

  • I have confirmed this bug exists on the current master branch of atlite.

Issue Description

Units and names for multiple variables in era5 are incorrect.

Here are the units and long_names extracted from an example cutout:

Variable Reported units Correct units Reported long_name Correct long_name
height m**2 s**-2 m Geopotential Height
wnd100m m s**-1 m s**-1 100 metre wind speed
wnd_shear_exp `` (empty) `` (empty) wind shear exponent
wnd_azimuth m s**-1 rad 100 metre U wind component 100 metre wind azimuth
roughness m m Forecast surface roughness
influx_toa W m**-2 W m**-2 TOA incident short-wave ...
influx_direct W m**-2 W m**-2 Surface direct short-wave ...
influx_diffuse W m**-2 W m**-2 Surface diffuse solar ...
albedo (0 - 1) (0 - 1) Albedo
solar_altitude rad rad latitude solar altitude
solar_azimuth rad rad latitude solar azimuth
temperature K K 2 metre temperature
soil temperature K K Soil temperature level 4
dewpoint temperature K K 2 metre dewpoint temperature
runoff m m Runoff

I see 4 issues:

  1. height is still listed in geopotential units m**2 s**-2, even though it is converted to m.
  2. wnd_azimuth is in velocity units m s**-1 even though it's an angle and should be in rad
  3. solar_altitude and solar_azimuth inherit the latitude long name from the locational variables that are used to calculate them
  4. Capitalisation of long_name's is inconsistent, some start with a capital and some don't.
Reproducible Example
import atlite

# should download pretty fast, same as conftest
cutout = atlite.Cutout(
    path="mre_era5.nc",
    module="era5",
    bounds=(-4, 56, 1.5, 62),
    time="2013-01-01",
)
cutout.prepare()

print(f"{'variable':22s} {'units':12s} long_name")
print("-" * 70)
for v in cutout.data.data_vars:
    attrs = cutout.data[v].attrs
    print(f"{v:22s} {str(attrs.get('units', '')):12s} {attrs.get('long_name', '')}")
Expected Behavior

units are physically correct

Installed Versions

argcomplete==3.1.4
attrs==23.2.0
Babel==2.10.3
bcc==0.29.1
bcrypt==3.2.2
beautifulsoup4==4.12.3
blinker==1.7.0
Brlapi==0.8.5
Brotli==1.1.0
certifi==2023.11.17
chardet==5.2.0
click==8.1.6
cloud-init==25.3
colorama==0.4.6
command-not-found==0.3
configobj==5.0.8
cryptography==41.0.7
cssselect==1.2.0
cupshelpers==1.0
dbus-python==1.3.2
defer==1.0.6
defusedxml==0.7.1
distro==1.9.0
distro-info==1.7+build1
duplicity==2.1.4
fasteners==0.18
html5lib==1.1
httplib2==0.20.4
idna==3.6
Jinja2==3.1.2
jsonpatch==1.32
jsonpointer==2.0
jsonschema==4.10.3
language-selector==0.1
launchpadlib==1.11.0
lazr.restfulclient==0.14.6
lazr.uri==1.0.6
libevdev==0.5
louis==3.29.0
lxml==5.2.1
Mako==1.3.2.dev0
Markdown==3.5.2
markdown-it-py==3.0.0
MarkupSafe==2.1.5
mdurl==0.1.2
meson==1.3.2
monotonic==1.6
netaddr==0.8.0
netifaces==0.11.0
numpy==1.26.4
oauthlib==3.2.2
olefile==0.46
openshot-qt==3.5.1
packaging==24.0
paramiko==2.12.0
pexpect==4.9.0
pillow==10.2.0
pipx==1.4.3
platformdirs==4.2.0
psutil==5.9.8
ptyprocess==0.7.0
py==1.11.0
pycairo==1.25.1
pycups==2.0.1
Pygments==2.17.2
PyGObject==3.48.2
PyJWT==2.7.0
PyNaCl==1.5.0
pyparsing==3.1.1
PyQt5==5.15.10
PyQt5-sip==12.13.0
PyQtWebEngine==5.15.6
pyrsistent==0.20.0
pyserial==3.5
python-apt==2.7.7+ubuntu5.2
python-dateutil==2.8.2
python-debian==0.1.49+ubuntu2
pytz==2024.1
pyudev==0.24.0
pyxdg==0.28
PyYAML==6.0.1
pyzmq==24.0.1
requests==2.31.0
rich==13.7.1
rofication==1.2.2
scour==0.38.2
sentry-sdk==1.39.2
setuptools==68.1.2
six==1.16.0
soupsieve==2.5
systemd-python==235
typing_extensions==4.10.0
ubuntu-drivers-common==0.0.0
ubuntu-pro-client==8001
ufw==0.36.2
unattended-upgrades==0.1
urllib3==2.0.7
usb-creator==0.3.16
userpath==1.9.1
wadllib==1.3.6
webencodings==0.5.1
wheel==0.42.0
xdg==5
xkit==0.0.0

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the era5 module and the Cutout preparation path, using the issue's reproducible example and the existing conftest setup to inspect generated variable attributes. Update the metadata for the four reported issues and verify that the affected units and long_name values are physically correct and consistently capitalized.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
54/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.