Refactor Slice 2: Move batch/scheduler logic into `CIME/core/batch/`
- Dominant language
- Python
- Stars
- 174
- Forks
- 225
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 14
Description
Part of #4950. Slice 2 of 5. Estimated 4 weeks. Depends on #4996.
## Goal
Move batch logic from scattered locations (`XML/env_batch.py`,
`case/case_submit.py`, `utils.py`) into a coherent `CIME/core/batch/` package.
Existing entrypoints become thin wrappers.
## Why
`EnvBatch` (`XML/env_batch.py`, 1590 lines, 43 methods) is the second-largest
class in CIME. It handles batch config parsing, job submission, queue
management, and directive generation all in one class. Submission logic is
also smeared into `case_submit.py` and `utils.py`.
## Tasks
### Decompose `EnvBatch`
- [ ] `CIME/core/batch/config.py` — batch config / queue parsing.
- [ ] `CIME/core/batch/submit.py` — job submission logic (consumes from
`EnvBatch` and `case_submit.py`).
- [ ] `CIME/core/batch/directives.py` — directive generation.
- [ ] `EnvBatch` becomes a thinner wrapper delegating to the above.
### Scheduler protocol
- [ ] Define `Scheduler` protocol in `CIME/core/batch/scheduler.py`.
PBS, Slurm, LSF are genuine polymorphic backends — DI is warranted here.
- [ ] Concrete implementations: `PBSScheduler`, `SlurmScheduler`, `LSFScheduler`.
### Move from `utils.py` → `core/batch/`
- [ ] `batch_jobid`, `get_batch_script_for_job`, `get_project`,
`get_charge_account`, `add_mail_type_args`, `resolve_mail_type_args`,
`transform_vars`.
### Move from `utils.py` → `core/shell.py`
- [ ] `run_cmd`, `run_cmd_no_fail` (general utility, pulled here because batch
is the first heavy consumer).
### Consolidation
- [ ] `transform_vars` (`utils.py:2118`) takes `case` and calls
`case.get_value()` repeatedly — consolidate as a `Case` method, with
core template logic in `CIME/core/batch/`.
- [ ] `batch_jobid` (`utils.py:2360`) calls `case.get_job_id()` — move into
the batch subsystem.
### Imports & re-exports
- [ ] Update internal CIME imports to point at `core/batch/`.
- [ ] Leave re-exports in `utils.py`, `case/case_submit.py` for external
consumers.
## Definition of done
- [ ] All batch tests pass.
- [ ] External models (E3SM, CESM, NorESM) submit jobs unchanged.
- [ ] Batch logic lives in `core/batch/`, not in `XML/`, `case/`, `utils.py`.
- [ ] `EnvBatch` is a thin wrapper.
- [ ] `Scheduler` protocol with PBS/Slurm/LSF implementations.
- [ ] 80%+ coverage on `core/batch/`.
## Related
- Depends on #4996 (foundation, exception hierarchy)
- Blocks #4999 (build orchestration may depend on batch interface)
Contributor guide
Research direction
Start by reading XML/env_batch.py, case/case_submit.py, and the listed utilities, then review dependency #4996 and the existing batch tests. Map current submission, queue, directive, and scheduler entry points before dividing the work across CIME/core/batch/. Done means all batch tests pass, external models submit unchanged, EnvBatch is a thin wrapper, and the stated coverage and backend requirements are met.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- infrastructure
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100