CliMA / CliMA/ParameterEstimocean.jl
Hierarchical, "scheduled" calibration
- Dominant language
- Julia
- Stars
- 22
- Forks
- 6
- PR merge metrics
- No merged PRs in 30d
Description
We're developing an algorithm for calibrating unknown parameters using Ensemble Kalman Inversion in the context of realistic, challenging, expensive problems. Two interrelated challenges we face are
* Getting a "general idea" of where parameter vectors need to point for a model to be at all reasonable (let alone optimal)
* Finding globally optimal parameters rather than local optima.
### A nascent strategy
1) Perform an initial "global search" for parameters by setting up very short, cheap calibration experiment that permits huge ensembles and very short time-steps
2) Refine the search systematically by
- Tightening priors around "reasonable" parameter values (possibly in an "open minded" way that still permits _rare_ exotic parameter choices). Crucially, this means we can take longer time-steps without destroying our ensembles (some particle failure are ok, but if 100% of our particles fail we are out of luck).
- Increasing the time-span / _relevance_ of the physical problem used for calibration. For time-dependent scenarios, a nice strategy involves performing the global search over a short time span at the _end_ of the scenario, and moving the _start time_ backwards during subsequent `stage`s as our priors are refined and our time-steps can be increased.
What are other important strategeries?
### Features we need
I think we can adapt our `EnsembleKalmanInversion` object to achieve these features. We'll need
* The concept of a calibration stages that co-vary key parameters of the calibration:
- Prior tightness (eg `ScaledLogitNormal.bounds`, `Lognormal.std`, or `Normal.sigma`)
- Duration of the observations / model-data comparison
- Simulation time-step
- Ensemble size?
Maybe a starting point is a new type that captures the essential variables that we need to vary:
```julia
struct CalibrationStage
Nensemble
simulation_dt
observation_times
priors
end
```
Then we can design a `CalibrationSchedule` that stores multiple `CalibrationStage`s, with methods for updating priors as we move along.
* Adaptive time-stepping to carefully control the rate of convergence of an iteration. We want slow convergence during the global search, faster convergence later on, and we'll need to carefully control convergence in order to use the results of the previous calibration in a "priors update".
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reading the existing EnsembleKalmanInversion object and the proposed CalibrationStage and CalibrationSchedule shapes in the issue. Clarify how priors, observation duration, simulation time-step, ensemble size, and adaptive convergence should vary between stages. Done means an agreed design and implementation plan for hierarchical scheduled calibration, including how prior updates consume previous results.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- julia
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100