Refactor Slice 3A: SRCROOT standardization (remove `config_files.xml`)
- Dominant language
- Python
- Stars
- 174
- Forks
- 225
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 14
Description
Part of #4950. Slice 3A of 5. Estimated 4 weeks. Depends on #4996.
This is a **new feature** with breaking-change potential — done with feature
flag and 4-stage rollout.
## Goal
Remove `config_files.xml` indirection and standardize SRCROOT resolution for
submodule and standalone usage. Resolves #4956 and addresses #3923.
## Resolution priority for SRCROOT
1. Environment variable `SRCROOT`
2. User config `~/.cime/config`
3. Command-line flag `--srcroot`
4. Step up one directory from CIMEROOT (typical submodule layout)
5. CIMEROOT itself (standalone mode)
## Expected directory structure
```
$SRCROOT/
├── cime_config/ # Model config (validated)
│ ├── config_grids.xml
│ ├── config_compsets.xml
│ ├── config_machines.xml
│ └── config_tests.xml
└── cime/ # CIME submodule
```
## What's removed
- `CIME/data/config/{e3sm,cesm,ufs}/config_files.xml`
- `MODEL_CONFIG_FILES` indirection
- Model-specific `get_config_path()` logic
## What's added
- `CIME/core/config/srcroot.py` — `SRCROOTResolver` (constructor injection,
no globals, fully testable).
- `CIME/core/config/loader.py` — `ConfigFileLoader` for direct config XML
loading.
- `--srcroot` flag on CLI tools.
- Explicit standalone mode for unit tests.
## Tasks
### Implementation
- [ ] `CIME/core/config/srcroot.py` with `SRCROOTResolver` class.
- [ ] `CIME/core/config/loader.py` with `ConfigFileLoader` class.
- [ ] Factory functions providing production defaults, mockable for tests.
- [ ] Add `--srcroot` flag to all CLI tools.
- [ ] Refactor `CIME/XML/files.py` to use new loader (gated by feature flag).
### Move from `utils.py` → `core/config/`
- [ ] `get_cime_root`, `get_src_root`, `get_model`, `set_model`,
`get_cime_config`, `get_config_path`, `get_schema_path`,
`get_template_path`, `get_tools_path`, `get_cime_default_driver`,
`get_all_cime_models`, `get_model_config_location_within_cime`,
`get_scripts_root`, `get_model_config_root`, `get_htmlroot`,
`get_urlroot`.
### Migration (4-stage rollout)
1. **Opt-in** (weeks 1–2): feature flag `CIME_USE_NEW_CONFIG_LOADER=true`.
2. **Opt-out** (weeks 3–4): new system default; old available via flag.
3. **Deprecation**: warnings on old code paths.
4. **Removal**: delete `config_files.xml` system.
## Validation
- [ ] E3SM case-creation workflow.
- [ ] CESM case-creation workflow.
- [ ] NorESM case-creation workflow.
- [ ] Standalone mode for unit tests.
## Definition of done
- [ ] Resolution priority implemented and unit-tested with all 5 paths.
- [ ] `config_files.xml` no longer referenced after stage 4.
- [ ] All 3 host models pass case creation.
- [ ] `--srcroot` flag works on every CLI tool.
- [ ] Resolves #4956 and addresses #3923.
## Related
- Resolves #4956 (submodule conventions)
- Addresses #3923 (standalone-checkout detection)
- Possibly resolves #4637 (special chars in SRCROOT) — verify
- Depends on #4996 (bootstrap, exception hierarchy)
- Blocks #4999 (build refactor must come after stable config loading)
Contributor guide
Research direction
Start by reading the dependency on #4996, then inspect CIME/core/config/srcroot.py, CIME/core/config/loader.py, and CIME/XML/files.py as named in the task. Trace the existing utilities and CLI entry points before implementing the resolver and loader. Done requires all five resolution paths to be unit-tested, the CLI flag to work across tools, and the listed host-model workflows to pass.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- build-system, cli, tooling
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100