ESMCI / ESMCI/cime

Allow config_machines to support inheritance

Open
#4,988 0 comments 0 reactions 0 assignees View on GitHub
ty: Discussion ty: enhancement
Dominant language
Python
Stars
174
Forks
225
Avg merge
3d 10h
Merged PRs (30d)
18

Description

In E3SM, config_machines.xml is a whopping 5700 lines long. Yet, a lot of options are common across several machines (e.g., b/c they all use the same job scheduler).

I am envisioning an XML structure that allows to "extent" a base node. Something like this:

```xml

Linux
$CIME_OUTPUT_ROOT/archive/$CASE
$ENV{HOME}/projects/acme/scratch
none
[other generic options]

slurm
[define slurm commands/flags for running]

[define openmpi commands/flags for running]

[this overrides the base class(es) option]
$ENV{PSCRATCH}/e3sm_scratch/pm-cpu
[other pm-cpu options, but no need to specify slurm stuff]

```
Here, the `abstract="true"` specifier could be omitted, but I figured it would be nice to prevent users from using these "base" machines for create newcase/test (and get an error cause the machine is not "concrete"). One could envision a handful of very small, topic-oriented abstract machines, which take care of defining options only for one choice of a particular item (I could only think of slurm-vs-openmpi/mpich, or batch system specs, but there may be others), and a "default" mach settings (e.g. declaring stuff like SUPPORTED_BY, OS, RUNDIR, EXEROOT, TPUT/MEMLEAK tolerances, etc.). It may not reduce the file by a factor 10, but if we can remove more lines from the screen, and fit more useful info (so we scroll less), it may be still worth it.

I don't have a solid understanding of CIME to take on this task, though I can try to explain it to an AI bot and see what it comes up with. We do use this paradigm in E3SM's C++ atm component, inside the namelist defaults xml file, and it does help reducing the lines of code (highlighting only what makes each node diff from others).

I wonder what everyone's thoughts are on this matter.

Contributor guide

Open the contributing guide

Research direction

Start by tracing how config_machines.xml is parsed, validated, and used for create newcase/test. Compare that flow with the inheritance approach already used in E3SM's namelist defaults XML. Done should include a concrete inheritance design, handling for abstract machines, and confirmation that inherited and overridden machine options behave as intended.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.