ESMCI / ESMCI/cime

Refactor Slice 3A: SRCROOT standardization (remove `config_files.xml`)

Open
#4,998 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
174
Forks
225
Avg merge
1d 16h
Merged PRs (30d)
14

Description

Part of #4950. Slice 3A of 5. Estimated 4 weeks. Depends on #4996.

This is a **new feature** with breaking-change potential — done with feature
flag and 4-stage rollout.

## Goal

Remove `config_files.xml` indirection and standardize SRCROOT resolution for
submodule and standalone usage. Resolves #4956 and addresses #3923.

## Resolution priority for SRCROOT

1. Environment variable `SRCROOT`
2. User config `~/.cime/config`
3. Command-line flag `--srcroot`
4. Step up one directory from CIMEROOT (typical submodule layout)
5. CIMEROOT itself (standalone mode)

## Expected directory structure

```
$SRCROOT/
├── cime_config/ # Model config (validated)
│ ├── config_grids.xml
│ ├── config_compsets.xml
│ ├── config_machines.xml
│ └── config_tests.xml
└── cime/ # CIME submodule
```

## What's removed

- `CIME/data/config/{e3sm,cesm,ufs}/config_files.xml`
- `MODEL_CONFIG_FILES` indirection
- Model-specific `get_config_path()` logic

## What's added

- `CIME/core/config/srcroot.py` — `SRCROOTResolver` (constructor injection,
no globals, fully testable).
- `CIME/core/config/loader.py` — `ConfigFileLoader` for direct config XML
loading.
- `--srcroot` flag on CLI tools.
- Explicit standalone mode for unit tests.

## Tasks

### Implementation
- [ ] `CIME/core/config/srcroot.py` with `SRCROOTResolver` class.
- [ ] `CIME/core/config/loader.py` with `ConfigFileLoader` class.
- [ ] Factory functions providing production defaults, mockable for tests.
- [ ] Add `--srcroot` flag to all CLI tools.
- [ ] Refactor `CIME/XML/files.py` to use new loader (gated by feature flag).

### Move from `utils.py` → `core/config/`
- [ ] `get_cime_root`, `get_src_root`, `get_model`, `set_model`,
`get_cime_config`, `get_config_path`, `get_schema_path`,
`get_template_path`, `get_tools_path`, `get_cime_default_driver`,
`get_all_cime_models`, `get_model_config_location_within_cime`,
`get_scripts_root`, `get_model_config_root`, `get_htmlroot`,
`get_urlroot`.

### Migration (4-stage rollout)
1. **Opt-in** (weeks 1–2): feature flag `CIME_USE_NEW_CONFIG_LOADER=true`.
2. **Opt-out** (weeks 3–4): new system default; old available via flag.
3. **Deprecation**: warnings on old code paths.
4. **Removal**: delete `config_files.xml` system.

## Validation

- [ ] E3SM case-creation workflow.
- [ ] CESM case-creation workflow.
- [ ] NorESM case-creation workflow.
- [ ] Standalone mode for unit tests.

## Definition of done

- [ ] Resolution priority implemented and unit-tested with all 5 paths.
- [ ] `config_files.xml` no longer referenced after stage 4.
- [ ] All 3 host models pass case creation.
- [ ] `--srcroot` flag works on every CLI tool.
- [ ] Resolves #4956 and addresses #3923.

## Related

- Resolves #4956 (submodule conventions)
- Addresses #3923 (standalone-checkout detection)
- Possibly resolves #4637 (special chars in SRCROOT) — verify
- Depends on #4996 (bootstrap, exception hierarchy)
- Blocks #4999 (build refactor must come after stable config loading)

Contributor guide

Open the contributing guide

Research direction

Start by reading the dependency on #4996, then inspect CIME/core/config/srcroot.py, CIME/core/config/loader.py, and CIME/XML/files.py as named in the task. Trace the existing utilities and CLI entry points before implementing the resolver and loader. Done requires all five resolution paths to be unit-tested, the CLI flag to work across tools, and the listed host-model workflows to pass.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
build-system, cli, tooling
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.