Innovation: Raw data collection tool
Open
@andrea-williams is already working on this.
Since Dec 2, 2021.
User Requested
User Story
- Dominant language
- Python
- Stars
- 65
- Forks
- 11
- Avg merge
- 1d 2h
- Merged PRs (30d)
- 70
Description
As a forecaster I want a thing that can download all kinds of data for me So That I don't have to waste time clicking around on the internet and copying files around manually
Additional Context
- Forecaster collect all kinds of data (e.g. weather soundings for Tephigrams) in a slow and labour intensive manner. Ideally, at the click of a button, the data they need should be download for them.
- In the past, forecasters have cobbled together their own solutions.
- Potential solution: 0) Write code in Python 1) Use Python Fire to generate CLI 2) use Gooey to slap a UI on it 3) freeze code and distribute to end user.
- Not an innovation? Could be something we just work on from the side of our desk during learning time, etc.
- Brand name "Edna" (Esoteric data nabber automated)
- It's a boring thing that just goes and fetches data from a bunch of places a bunch of times. How boring and tedious (and old-fashioned). Hence the old-fashioned name Edna.
- from discussions with JE & MM, desired solution is script that runs locally (on Windows machines) that will download data to local user-specified folder. MM wants script to run when button is clicked. JE wants to be able to create a cron job so script runs automatically.
- files should be downloaded as CSVs (one file for each station) to import into RAOB
- data sources used (that we know of):
- EC's FTP site contains augmented model data (WFS pays for it, authenticates using fire centre-specific IDIRs). Andrea has site address & credentials
- EC also has data mart with freely available data that's sometimes used (https://dd.weather.gc.ca/vertical_profile/)
- University of Wyoming for actual soundings (http://weather.uwyo.edu/upperair/sounding.html)
- also for actuals (http://www.rap.ucar.edu/weather/upper/current.rawins)
- NOAA: https://www.ready.noaa.gov/READYcmet.php. Set lat/long, pick Sounding > HRRR model (3km, 18h, 1 hrly, CONUS, pressure), select forecast cycle (00Z, 06Z, 12Z, 18Z)
Question
Does this happen for more than just Tephigrams?
Acceptance Criteria
- Given I need some data - at the click of a button it's downloaded from it's source and placed in the correct location on my local computer.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.