NCAR / NCAR/DART

Performance issue? obs_def gpsro key vs. obs_seq key

Open
#512 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

performance psi
Dominant language
Fortran
Stars
263
Forks
182
Avg merge
11d 12h
Merged PRs (30d)
7

Description

Describe the bug

Anderson found this when running obs_sequence_tool.

  1. List the steps someone needs to take to reproduce:
  • take an old obs_seq.out and run it through obs_sequence_tool to get the same data, but with an updated header (TYPES consistent with compiled dart: obs_sequence.processed. The goal of this was to make a obs_seq file for easy comparison when testing read/write obs_sequence.
  • run your (Anderson's) read & write obs_sequence code.
  1. What was the expected outcome?
    input obs_sequence.processed === output obs_sequence.processed

  2. What actually happened?
    The gpsroref keys are different, and gpsref keys ininput obs_sequence.processed do not start from 1.

when you call initialize_module for obs_def_gps_mod
the keycount is set to 0.
https://github.com/NCAR/DART/blob/1b76f3afa5978469d6a119710bb638c1c077ae20/observations/forward_operators/obs_def_gps_mod.f90#L134-L135

what gets passed in to write_obs_def is the key from the obs_sequence. Which is not the same as the gpsroref key.

obs_sequence_tool -> write_obs -> write_obs_def
https://github.com/NCAR/DART/blob/1b76f3afa5978469d6a119710bb638c1c077ae20/assimilation_code/modules/observations/obs_sequence_mod.f90#L2509

/glade/scratch/hkershaw/DART/Bugs/obs_sequence_tool
grep gps obs_seq.out | sort -k2 -n |head  -n 1
gpsroref  646965

grep gps obs_seq.processed | sort -k2 -n | head  -n 1
gpsroref   81405
Error Message

No error, I think using the key means you allocate more memory than you need to in obs_def_gps_mod (and obs_def_rttov_mod which also used module data accessed by key).
Also, tripped up Anderson when trying to compare two obs_sequences which is fair enough.
I believe (not checked with Anderson's code) that a simple read/write you get the gpsro starting at 1.

sanity checked by changing GPSREF value (they are all zero all the time) that the obs keeps the gpsref data
/glade/scratch/hkershaw/DART/Bugs/obs_sequence_tool

diff obs_seq.out.X obs_seq.out
10815868c10815868
<   0.000000000000000E+000  0.000000000000000E+000  0.000000000000000E+000  0.000000000000000E+000  0.000000000000000E+000  1.230000000000000E+000 GPSREF
---
>   0.000000000000000E+000  0.000000000000000E+000  0.000000000000000E+000  0.000000000000000E+000  0.000000000000000E+000  0.000000000000000E+000 GPSREF
Which model(s) are you working with?

its a obs sequence file from a cam-fv experiment.

Version of DART

Which version of DART are you using?
v10.8.0

Have you modified the DART code?

No

Build information

Please describe:

  1. Cheyenne
  2. intel

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with observations/forward_operators/obs_def_gps_mod.f90, especially initialize_module, and assimilation_code/modules/observations/obs_sequence_mod.f90 at write_obs_def. Reproduce the case with obs_sequence_tool using the described obs_seq.out input, then compare the processed output keys and GPSREF values. Done means read/write output matches the input data and does not introduce inconsistent gpsroref keys or unnecessary key-based allocation.

Written by the indexing model from the issue text.

Assessment

Tech stack
fortran
Domain
data
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.