CliMA / CliMA/CalibrateEmulateSample.jl

[audit 2026-07-09] General bugs for emulator interface and emulator-specific issues

Open
#451 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Julia
Stars
90
Forks
16
PR merge metrics
No merged PRs in 30d

Description

Issues found by `math-auditor`: M - "major", m - "minor", h - "hygeine"

### Forward-map wrapper
- M2 — predict ignores `add_obs_noise_cov` entirely; always returns `Γ` as the uncertainty of a deterministic map.
- M3 — predict crashes (DimensionMismatch) for any truncating output encoder.

### General emulator interface
- M1 — with default `noise_learn=true`, `add_obs_noise_cov=false` still returns noise-inflated covariance and `=true` adds only the `1e-3` jitter — the flag's meaning is inverted relative to its docs
- m9 — passing an already-built MLT into a new Emulator only warns `"already built. skipping..."`, but the constructor still fits a new encoder schedule to the new data, silently pairing old-encoder-trained models with new encoders.
- h10 — `show.jl` labels model count "(one per output dimension)" when it should say encoded output dimension; contradicts the adjacent Emulator display under truncating encoders.

### GaussianProcess emulator
- m7 — `add_obs_noise_cov=true` adds `alg_reg_noise·diag(Γ)`, silently dropping off-diagonal noise correlations; scale also wrong unless `noise_learn=false`.
- m10 — AGPJL ArgumentError message tells users the wrong parameter index (`model_params[end-2] instead of [end-1]`) and omits a value for `log_std_noise`.
- m11 — WhiteKernel semantics differ across backends at training-point coincidence (sklearn always zero cross-covariance; GPJL/AGPJL give σ²) — same hyperparameters, different predictions.
- h11 — `_predict` allocates Float64 buffers regardless of input eltype (silent `Float32` demotion; blocks Dual numbers for GPJL/SKLJL), plus assorted dead variables/branches.

### RandomFeature emulator (Scalar/Vector)
- M5 — documented optimizer option "prior" is silently ignored; the rebuild branch is provably dead.
- m1 — NICE correlation-noise level formula wrong by a scaling factor.
- m2 — latent data race (Threads.@threads shared read-modify-write) in `mean_of_covs`; currently unreachable but would silently corrupt `\Gamma` if the commented-out `approx_σ2` code is re-enabled.
- m3 — coefficient-penalty observable drops the regularization factor λ relative to stated theory (matches only at the `λ=1` default).
- m4 — predict docstrings claim noise-inclusive covariance, but default is `add_obs_noise_cov=false` (same trap direction as M1).
- m5 — scalar interface silently keeps only the diagonal of the output structure matrix while the vector interface uses the full matrix on identical input; no warning.
- h1 — `tullio_threading = "tullio"` hardcoded as a String where the API implies Bool; latent break on dependency update.
- h2 — docstrings document `"prior_in_scale"/"prior_out_scale"` options that are never read.
- h3 — objective RNG streams differ across multithread modes; seeded results change with JULIA_NUM_THREADS.
- h5 — `cov_sample_multiplier`'s documented minimum (0.0) permits `n_cov_samples ∈ {0,1}`, producing NaN covariance downstream.

Contributor guide

No contributing guide indexed for this repository

Research direction

This is a broad audit covering Forward-map, General emulator, GaussianProcess, and RandomFeature emulator behavior, with `show.jl` and `_predict` explicitly mentioned. Start by selecting one finding and tracing its named interface or code path; done means the selected documented behavior is corrected without taking on the unrelated audit items.

Written by the indexing model from the issue text.

Assessment

Tech stack
julia
Domain
machine-learning
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.