google / google/weather-tools

Deprecated Apache Beam Version Causing Error in weather-dl tool.

Open
#362 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
252
Forks
60
Avg merge
8d 4h
Merged PRs (30d)
3

Description

I encountered an issue while using the following command to download data from `cds` using `weather-dl` tool. Since this is an urgent task, I would greatly appreciate any assistance in resolving this error.

command used : `python "weather_dl/weather-dl" weather-tools/config_files/*.cfg --runner DataflowRunner --project $PROJECT --region $REGION --temp_location "gs://$BUCKET/tmp/" --disk_size_gb 260 --sdk_container_image "gcr.io/*****/miniconda3-beam:weather-tools-with-aria2" --experiment use_runner_v2`

I performed the above command in a fresh environment of `weather-tools`, and within the `config_files` directory, there is only one file with the following content:
```
[parameters]
client=cds
dataset=reanalysis-era5-complete
target_path=gs://gcp-public-data-arco-era5/raw/ERA5GRIB/HRES/Daily/{date:%%Y/%%Y%%m%%d}_hres_dve.grb2
partition_keys=
date

[parameters.sahil]
api_url=https://cds.climate.copernicus.eu/api/v2
api_key=****:****-****-****

[selection]
class=ea
stream=oper
expver=1
type=an
levtype=ml
levelist=1/to/137
date=2023-04-01/to/2023-04-30
time=00/to/23
param=138/155
```

while researching the above issue I discovered that the version of Apache Beam used in tool is [deprecated](https://cloud.google.com/dataflow/docs/support/sdk-version-support-status#python).

The error I got is ::
![e1](https://github.com/google/weather-tools/assets/123355381/fc77558c-e78e-4084-b110-1d56ea2925ba)
![e2](https://github.com/google/weather-tools/assets/123355381/96ac8c35-eb40-410c-9ae4-45bdde787f2a)
![e3](https://github.com/google/weather-tools/assets/123355381/dd755dd5-9340-4c90-8e6f-066369e2cff3)
![e4](https://github.com/google/weather-tools/assets/123355381/bfdba8ee-fe08-4ae5-af0a-c84a6992b180)
![e5](https://github.com/google/weather-tools/assets/123355381/7b0b244f-28b6-4457-a090-9816af1901c5)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.