NCAR / NCAR/wrfcloud

Parallelize tasks that can be run in parallel

Open
#78 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

component: NWP components priority: medium type: enhancement
Dominant language
Python
Stars
18
Forks
6
PR merge metrics
No merged PRs in 30d

Description

Describe the Enhancement

  • Pulling GFS data: each forecast hour can be pulled in parallel
  • ungrib.exe: each file can be "ungribbed" in parallel
  • metgrid.exe: each metgrid file can be processed as soon as ungrib.exe completes for that hour
  • UPP/graphics: each forecast hour can be processed as soon as the WRF output is written to disk
Time Estimate

1 day per sub-issue

Sub-Issues

Consider breaking the enhancement down into sub-issues.

  • Pull GFS data for each forecast hour in parallel
  • ungrib each GFS input file in parallel
  • process metgrid files once ungrib completes for a given hour
Relevant Deadlines

NONE

Define the Metadata

Assignee
  • Select engineer(s) or no engineer required
  • Select scientist(s) or no scientist required
Labels
  • Select component(s)
  • Select priority
Projects and Milestone
  • Select Project
  • Select Milestone as the next official version or Backlog of Development Ideas

Enhancement Checklist

  • Complete the issue definition above, including the Time Estimate and Funding Source.
  • Fork this repository or create a branch of develop.
    Branch name: feature_<Issue Number>/<Description>
  • Complete the development and test your changes.
  • Add/update log messages for easier debugging.
  • Add/update tests.
  • Add/update documentation.
  • Push local changes to GitHub.
  • Submit a pull request to merge into develop.
    Pull request: feature <Issue Number> <Description>
  • Define the pull request metadata, as permissions allow.
    Select: Reviewer(s), Project, and Development issue
    Select: Milestone as the next official version
  • Iterate until the reviewer(s) accept and merge your changes.
  • Delete your fork or branch.
  • Close this issue.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The issue names no files, tests, or entry points; start by locating the workflow that pulls GFS data and runs ungrib.exe, metgrid.exe, and UPP/graphics. Break the work into the listed sub-issues, then define completion as independent forecast hours progressing concurrently with each downstream step starting after its corresponding input is ready.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.