CIROH-UA / CIROH-UA/datastreamcli

integrate the preprocessor for forcings

Open
#115 0 comments 0 reactions 0 assignees View on GitHub
enhancement P2 speed
Dominant language
Python
Stars
3
Forks
6
Avg merge
1d 1h
Merged PRs (30d)
4

Description

Which NextGen forcings processing algorithm is the quickest can depend on the size of the requested processing job in terms of number of catchments and number of time steps. The currently integrated forcingprocessor is optimized for NRDS forecasting configurations of VPU scale and relatively few time steps (<250). The preprocessor is better suited for retrospective configurations of ~100 catchments and thousands of time steps.

Datastreamcli could contain logic to decide which integrated tool to use. The preprocessor is now planned to be integrated as a subsetting and bmi config provider, so it'll be available for forcings as well. The cleanest path to this could be to make the choice inside of configure_datastream.py. and have the datastream script run forcingprocessor if that config file is present, or the preprocessor if not.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start in configure_datastream.py and inspect how the datastream script currently selects and runs forcingprocessor. Check how the preprocessor will be exposed as a subsetting and BMI config provider, then implement and verify the forcings selection between forcingprocessor and the preprocessor based on the available configuration.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.