bcgov / bcgov/wps

Innovation: Raw data collection tool

Open
#1,521 0 comments 0 reactions 2 assignees View on GitHub

@andrea-williams is already working on this.

Since Dec 2, 2021.

User Requested User Story
Dominant language
Python
Stars
65
Forks
11
Avg merge
1d 2h
Merged PRs (30d)
70

Description

As a forecaster I want a thing that can download all kinds of data for me So That I don't have to waste time clicking around on the internet and copying files around manually

Additional Context

  • Forecaster collect all kinds of data (e.g. weather soundings for Tephigrams) in a slow and labour intensive manner. Ideally, at the click of a button, the data they need should be download for them.
  • In the past, forecasters have cobbled together their own solutions.
  • Potential solution: 0) Write code in Python 1) Use Python Fire to generate CLI 2) use Gooey to slap a UI on it 3) freeze code and distribute to end user.
  • Not an innovation? Could be something we just work on from the side of our desk during learning time, etc.
  • Brand name "Edna" (Esoteric data nabber automated)
    • It's a boring thing that just goes and fetches data from a bunch of places a bunch of times. How boring and tedious (and old-fashioned). Hence the old-fashioned name Edna.
  • from discussions with JE & MM, desired solution is script that runs locally (on Windows machines) that will download data to local user-specified folder. MM wants script to run when button is clicked. JE wants to be able to create a cron job so script runs automatically.
  • files should be downloaded as CSVs (one file for each station) to import into RAOB
  • data sources used (that we know of):
    - EC's FTP site contains augmented model data (WFS pays for it, authenticates using fire centre-specific IDIRs). Andrea has site address & credentials
    - EC also has data mart with freely available data that's sometimes used (https://dd.weather.gc.ca/vertical_profile/)
    - University of Wyoming for actual soundings (http://weather.uwyo.edu/upperair/sounding.html)
    - also for actuals (http://www.rap.ucar.edu/weather/upper/current.rawins)
    - NOAA: https://www.ready.noaa.gov/READYcmet.php. Set lat/long, pick Sounding > HRRR model (3km, 18h, 1 hrly, CONUS, pressure), select forecast cycle (00Z, 06Z, 12Z, 18Z)

Question
Does this happen for more than just Tephigrams?

Acceptance Criteria

  • Given I need some data - at the click of a button it's downloaded from it's source and placed in the correct location on my local computer.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.