Parallelize tasks that can be run in parallel
Open
Nobody has claimed this yet.
component: NWP components
priority: medium
type: enhancement
- Dominant language
- Python
- Stars
- 18
- Forks
- 6
- PR merge metrics
- No merged PRs in 30d
Description
Describe the Enhancement
- Pulling GFS data: each forecast hour can be pulled in parallel
- ungrib.exe: each file can be "ungribbed" in parallel
- metgrid.exe: each metgrid file can be processed as soon as ungrib.exe completes for that hour
- UPP/graphics: each forecast hour can be processed as soon as the WRF output is written to disk
Time Estimate
1 day per sub-issue
Sub-Issues
Consider breaking the enhancement down into sub-issues.
- Pull GFS data for each forecast hour in parallel
- ungrib each GFS input file in parallel
- process metgrid files once ungrib completes for a given hour
Relevant Deadlines
NONE
Define the Metadata
Assignee
- Select engineer(s) or no engineer required
- Select scientist(s) or no scientist required
Labels
- Select component(s)
- Select priority
Projects and Milestone
- Select Project
- Select Milestone as the next official version or Backlog of Development Ideas
Enhancement Checklist
- Complete the issue definition above, including the Time Estimate and Funding Source.
- Fork this repository or create a branch of develop.
Branch name:feature_<Issue Number>/<Description> - Complete the development and test your changes.
- Add/update log messages for easier debugging.
- Add/update tests.
- Add/update documentation.
- Push local changes to GitHub.
- Submit a pull request to merge into develop.
Pull request:feature <Issue Number> <Description> - Define the pull request metadata, as permissions allow.
Select: Reviewer(s), Project, and Development issue
Select: Milestone as the next official version - Iterate until the reviewer(s) accept and merge your changes.
- Delete your fork or branch.
- Close this issue.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue names no files, tests, or entry points; start by locating the workflow that pulls GFS data and runs ungrib.exe, metgrid.exe, and UPP/graphics. Break the work into the listed sub-issues, then define completion as independent forecast hours progressing concurrently with each downstream step starting after its corresponding input is ready.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100