NCAR / NCAR/DART

Documentation: include more visible information for running high-res / large state assimilations

Open
#999 0 comments 0 reactions 1 assignee View on GitHub

@mjs2369 is already working on this.

Since Mar 24, 2026.

Documentation performance
Dominant language
Fortran
Stars
263
Forks
182
Avg merge
11d 12h
Merged PRs (30d)
7

Description

What's the issue?

📚 Please describe the problem you’ve noticed in the documentation.

The documentation needs more information for running high-res / large model size assimilations, and it needs to be more visible in the docs.

The buffer_state_io option can be found in the docs in the "IO - reading and writing of the model state" section of the "Data management in DART" page, but this page is deep into the documentation, and I feel many users do not end up reading. https://docs.dart.ucar.edu/en/latest/guide/data-management-issues.html#data-management-in-dart

This nml option greatly improves the performance of filter with large states, especially when the state vector does not fit into memory on a single node.

Additionally, nowhere in the documentation does it talk about how running on scratch will astronomically improve the IO speed on Derecho. For reference, in a ROMS_Rutgers run with ~650 million elements in the state, running on scratch causes the state space output to be completed in a fraction of the time.

12 nodes and 300 mpiprocs on work (30 mins):

Before state space output TIME: 2025/10/30 09:36:00
After  state space output TIME: 2025/10/30 10:06:10

vs 12 nodes and 300 mpiprocs on scratch (2 mins):

Before state space output TIME: 2025/11/06 23:24:23
After  state space output TIME: 2025/11/06 23:26:20

This is due to the fact that scratch uses a Lustre file system, which is a different file system from both work and home (https://ncar-hpc-docs-arc-iframe.readthedocs.io/storage-systems/glade/lustre/).

Finally, the options in the &ensemble_manager_nml (namely layout and tasks_per_node) could also be more visible. They are detailed in both the "Data management in DART" and "MODULE ensemble_manager_mod" doc pages, but I think it should also be included in with this new section of the documentation.

What needs to be fixed?

Share what’s incorrect, unclear, missing, or outdated.

Information on these performance enhancing tactics is either missing or very difficult to find in the documentation

Suggestions for improvement

If you have any ideas to fix or generally improve this documentation, please share them.

This information could be added as a small section to the quickstart guide, which would both make the information more visible and also promote our capabilities to run with large states.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.