ccpp-capgen v1: a unified successor to ccpp-prebuild and ccpp-capgen v0

Description

Note. ccpp-capgen v1 was previously called ccpp-capgen-ng. Some of the supporting documents attached below (until 2026-06-28) use this terminology.

ccpp-capgen v1 is a next-generation code generator for the CCPP framework. It is not a criticism of the existing tools — both ccpp-prebuild and ccpp-capgen v0 have served their respective communities well and embody years of careful design. This project attempts to synthesize the strengths of both: the performance of ccpp-prebuild and the metadata richness and feature set of ccpp-capgen v0, while also taking advantage of relaxed code-generator requirements that have emerged over time and lessons learned from operating both tools in development, research, and production. It also aims to resolve several outstanding issues that have been blocking progress toward a long-discussed unification of the two code generators.

Supporting documents — including a redesign analysis, the LLM prompt used to bootstrap development, developer and user documentation, and a migration guide — will be provided as part of the draft pull request.

The code was developed with the assistance of an LLM (Claude, by Anthropic) as a coding partner. Where possible, existing code was reused and repackaged.

Testing

Python docstring tests, unit tests

Significantly expanded test coverage with Python docstring tests and 1500-ish unit tests. Run in CI, all pass.

End-to-end tests

The following tests and features are currently supported by ccpp-capgen v1. They run successfullly in CI and on my development boxes.

• Former ccpp-capgen v0 tests:
  • advection
  • advection_auto_clone (support legacy constituents auto-cloning from metadata)
  • capgen_ng
  • ddthost
  • nested_suite
  • var_compat
• Former ccpp-prebuild tests:
  • chunked_data
  • blocked-data not supported in capgen or capgen-ng
  • opt_arg
• Newly added features/tests:
  • multi-instances
  • multi-instances + constituents
  • suite_allocate (exercises suite-owned variables with various types of runtime-known dimensions)
  • constituents_dim (suite- and host-owned variables with dimensions from constituents objects)

Host model integration

• CCPP-SCM all suites compile and all tests run, results are b4b identical for all except a few tests that are also problematic in develop
  • https://github.com/NCAR/ccpp-scm/pull/695 and PRs listed in there
  • https://github.com/NCAR/ccpp-framework/pull/762
  • https://github.com/NCAR/ccpp-physics/pull/1221
  • https://github.com/NCAR/TEMPO/pull/31
  • https://github.com/ufs-community/c3/pull/3
• CAM-SIMA Python unit tests pass, system tests pass with Intel and GNU
  • https://github.com/climbfuji/CAM-SIMA/tree/feature/capgen-ng
  • https://github.com/climbfuji/CAM-SIMA/pull/1
  • https://github.com/climbfuji/atmospheric_physics/pull/1
  • All regression tests (test/system) pass with both GNU and Intel, and with b4b-identical results except for two tests that also fail on development (SMS_Ln9.ne3pg3_ne3pg3_mg37.FKESSLER.derecho_intel.cam-outfrq_se_cslam_multitape, SMS_Ln9.ne3pg3_ne3pg3_mg37.FADIAB.derecho_gnu.cam-outfrq_se_cslam)
• UFS all regression tests pass, answers are bit-for-bit identical
  • https://github.com/ufs-community/ufs-weather-model/pull/3288
  • https://github.com/NOAA-EMC/ufsatm/pull/1125
  • https://github.com/NCAR/ccpp-framework/pull/762
  • https://github.com/ufs-community/ccpp-physics/pull/378
  • https://github.com/NOAA-GFDL/GFDL_atmos_cubed_sphere/pull/419
  • https://github.com/NOAA-EMC/CMEPS/pull/173
• NEPTUNE complete
  • Pull requests merged in NRL Enterprise GitHub
  • All 300+ tests pass with the three mandatory compilers (Intel LLVM, GCC, LLVM native)
  • Regular, mid- and high-altitude physics covered (including proprietary NRL physics code)

Issues

A list of issues that will be addressed with this issue will be provided in the associated pull request.

Addressing this issue will also address/close the following issues if/when the participating host models switch to capgen-ng:

• https://github.com/NCAR/ccpp-framework/issues/760
• https://github.com/NCAR/ccpp-framework/issues/757
• https://github.com/NCAR/ccpp-framework/issues/755
• https://github.com/NCAR/ccpp-framework/issues/754
• https://github.com/NCAR/ccpp-framework/issues/753
• https://github.com/NCAR/ccpp-framework/issues/751
• https://github.com/NCAR/ccpp-framework/issues/746
• https://github.com/NCAR/ccpp-framework/issues/744
• https://github.com/NCAR/ccpp-framework/issues/690
• https://github.com/NCAR/ccpp-framework/issues/688
• https://github.com/NCAR/ccpp-framework/issues/686
• https://github.com/NCAR/ccpp-framework/issues/682
• https://github.com/NCAR/ccpp-framework/issues/680
• https://github.com/NCAR/ccpp-framework/issues/679
• https://github.com/NCAR/ccpp-framework/issues/595
• https://github.com/NCAR/ccpp-framework/issues/588
• https://github.com/NCAR/ccpp-framework/issues/561
• https://github.com/NCAR/ccpp-framework/issues/516
• https://github.com/NCAR/ccpp-framework/issues/360
• https://github.com/NCAR/ccpp-framework/issues/380

To check/may not be implemented yet:

• https://github.com/NCAR/ccpp-framework/issues/749
• https://github.com/NCAR/ccpp-framework/issues/748
• https://github.com/NCAR/ccpp-framework/issues/747
• https://github.com/NCAR/ccpp-framework/issues/681

Working towards:

• https://github.com/NCAR/ccpp-framework/issues/737

Supporting Materials

See doc/ folder in the feature/capgen-v1 - documents evolved over time (commit history).

The documents were produced by Claude during development and most have them have not been formally reviewed. They are shared for transparency and to give context for the design decisions made. They should be treated as starting points rather than authoritative references.

An update of the CCPP technical documentation is made in https://github.com/NCAR/ccpp-doc/pull/81.