jejjohnson / jejjohnson/xrtoolz

Epic V5: Phenomena-based evaluation — event detection, matching, object metrics

Open
#63 0 comments 0 reactions 0 assignees View on GitHub
area:code enhancement epic validation-framework
Dominant language
Python
Stars
1
Forks
0
Avg merge
13d 21h
Merged PRs (30d)
3

Description

## Summary
Implement the fifth validation view from [validation.md §5](docs/design/validation.md): identify and verify scientifically meaningful phenomena (eddies, fronts, marine heatwaves, upwelling) rather than only continuous fields. A model can have low grid-level error while smoothing, displacing, delaying, weakening, or missing events — object-based verification scores those failure modes directly.

## User story
> As an applied scientist, I want to evaluate specific events such as eddies, fronts, marine heatwaves, upwelling events, storms, or plumes, so that the model is judged by its ability to reproduce scientifically meaningful phenomena.

## Lands in
- `xr_toolz.phenomena` (**new top-level module**) — `EventDefinition`, generic detection / labelling / matching, domain detectors (`DetectMarineHeatwaves`, `DetectEddies`, `DetectFronts`)
- `xr_toolz.metrics.object` (stub from F1.1, names pre-created by F1.3) — fills the pre-stubbed `ProbabilityOfDetection`, `FalseAlarmRatio`, `CriticalSuccessIndex`, `IntersectionOverUnion`, `DurationError`, `IntensityBias`, `CentroidDistance`
- `xr_toolz.viz.validation` — match-overlay panel (defer if V6 hasn't shipped)

## Sub-issues
- V5.1 — `EventDefinition` + generic detection / labelling / matching
- V5.2 — Domain detectors — `DetectMarineHeatwaves`, `DetectEddies`, `DetectFronts`
- V5.3 — `metrics.object` operator bodies (the F1.3 pre-stubbed names)
- V5.4 — `object_properties` + `contingency_table` plumbing
- V5.5 — Demo notebook (MHW detection on OSTIA SST vs ODYSSEA L3S)

## Out of scope
- `xr_toolz.extremes` (deferred to `xtremax` per validation.md §5).
- Tropical-cyclone tracking, storm detection — separate domain epic.
- Probabilistic event verification — overlaps V2.2; revisit if needed.

## Acceptance criteria
- [ ] V5.1–V5.5 closed.
- [ ] `EventDefinition` is reusable: same definition produces consistent detections on prediction *and* reference (per validation.md §5 note).
- [ ] Object outputs preserve event IDs, time bounds, geometry summaries, matched-pair metadata as xarray-native.
- [ ] No top-level `regionprops` / `scikit-image` import — lazy where used.
- [ ] mkdocs strict build is net-zero new warnings.

## References
- [docs/design/validation.md §5 Phenomena-Based Evaluation](docs/design/validation.md)
- [docs/design/validation-decisions.md §D14 — phenomena vs object metrics separation](docs/design/validation-decisions.md)

Contributor guide

Open the contributing guide

Research direction

Start with docs/design/validation.md §5 and docs/design/validation-decisions.md §D14, then review the V5.1–V5.5 scope and the listed xr_toolz.phenomena, xr_toolz.metrics.object, and xr_toolz.viz.validation entry points. Done means all sub-issues are closed, object outputs retain the specified metadata, imports remain lazy where required, and the mkdocs strict build adds no warnings.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.