NCAR / NCAR/MPAS-Workflow

Test "OSDF" and verify the performance and result

Open
#423 13 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
25
Forks
19
Avg merge
1d 49m
Merged PRs (30d)
2

Description

With a realistic DA case, test "OSDF" (internal obs data structure) and verify the performance (runtime, memory usage) and result (from obs feedback file).

See the example from Steve Vahl.
To answer your question you asked during the quarterly meeting, here is an example yaml block that shows what you need to add in order to test using the new OSDF (data frame) container for Obs Spaces. Obviously, the last line is the critical one. Hope this helps.

- obs space:
    name: "Radiosonde_1var"
    simulated variables:
    - airTemperature
    obsdatain:
      engine:
        type: H5File
        obsfile: Data/testinput_tier_1/sondes_obs_2018041500_m.nc4
    obsdataout: 
      engine:
        type: H5File
        obsfile: testoutput/sondes_obs_2018041500_m_osdf_frame_cols.nc4
    use data frame container: true

In the current develop code, this is set as 'false' : https://github.com/JCSDA-internal/ioda/blob/develop/src/ObsSpaceParameters.h#L95

After doing some extensive test, this option would be a default option in future.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the YAML example in the issue and the current use data frame container setting in src/ObsSpaceParameters.h. Run the realistic Radiosonde_1var case using the referenced input and output files, then compare runtime, memory usage, and the observation-feedback result. Done means the OSDF behavior is verified and there is enough evidence to decide whether it should become the default.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, yaml
Domain
data, performance, testing
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.