jejjohnson / jejjohnson/xrtoolz
Epic V5: Phenomena-based evaluation — event detection, matching, object metrics
- Dominant language
- Python
- Stars
- 1
- Forks
- 0
- Avg merge
- 13d 21h
- Merged PRs (30d)
- 3
Description
## Summary
Implement the fifth validation view from [validation.md §5](docs/design/validation.md): identify and verify scientifically meaningful phenomena (eddies, fronts, marine heatwaves, upwelling) rather than only continuous fields. A model can have low grid-level error while smoothing, displacing, delaying, weakening, or missing events — object-based verification scores those failure modes directly.
## User story
> As an applied scientist, I want to evaluate specific events such as eddies, fronts, marine heatwaves, upwelling events, storms, or plumes, so that the model is judged by its ability to reproduce scientifically meaningful phenomena.
## Lands in
- `xr_toolz.phenomena` (**new top-level module**) — `EventDefinition`, generic detection / labelling / matching, domain detectors (`DetectMarineHeatwaves`, `DetectEddies`, `DetectFronts`)
- `xr_toolz.metrics.object` (stub from F1.1, names pre-created by F1.3) — fills the pre-stubbed `ProbabilityOfDetection`, `FalseAlarmRatio`, `CriticalSuccessIndex`, `IntersectionOverUnion`, `DurationError`, `IntensityBias`, `CentroidDistance`
- `xr_toolz.viz.validation` — match-overlay panel (defer if V6 hasn't shipped)
## Sub-issues
- V5.1 — `EventDefinition` + generic detection / labelling / matching
- V5.2 — Domain detectors — `DetectMarineHeatwaves`, `DetectEddies`, `DetectFronts`
- V5.3 — `metrics.object` operator bodies (the F1.3 pre-stubbed names)
- V5.4 — `object_properties` + `contingency_table` plumbing
- V5.5 — Demo notebook (MHW detection on OSTIA SST vs ODYSSEA L3S)
## Out of scope
- `xr_toolz.extremes` (deferred to `xtremax` per validation.md §5).
- Tropical-cyclone tracking, storm detection — separate domain epic.
- Probabilistic event verification — overlaps V2.2; revisit if needed.
## Acceptance criteria
- [ ] V5.1–V5.5 closed.
- [ ] `EventDefinition` is reusable: same definition produces consistent detections on prediction *and* reference (per validation.md §5 note).
- [ ] Object outputs preserve event IDs, time bounds, geometry summaries, matched-pair metadata as xarray-native.
- [ ] No top-level `regionprops` / `scikit-image` import — lazy where used.
- [ ] mkdocs strict build is net-zero new warnings.
## References
- [docs/design/validation.md §5 Phenomena-Based Evaluation](docs/design/validation.md)
- [docs/design/validation-decisions.md §D14 — phenomena vs object metrics separation](docs/design/validation-decisions.md)
Contributor guide
Research direction
Start with docs/design/validation.md §5 and docs/design/validation-decisions.md §D14, then review the V5.1–V5.5 scope and the listed xr_toolz.phenomena, xr_toolz.metrics.object, and xr_toolz.viz.validation entry points. Done means all sub-issues are closed, object outputs retain the specified metadata, imports remain lazy where required, and the mkdocs strict build adds no warnings.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100