MeteoSwiss / MeteoSwiss/evalml

Evaluate ensemble forecasts

Open
#78 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
11
Forks
0
Avg merge
1d 21h
Merged PRs (30d)
5

Description

Extend evalml to also work for ensemble forecasts.

### Changes required
* realization as an additional dimension in forecasts and baselines
* additional metrics for ensemble forecasts
* configurable or automatically determined set of metrics and scores for deterministic / ensemble forecasts

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reviewing evalml's existing deterministic forecast evaluation pipeline, including how forecasts, baselines, metrics, and scores are represented. Identify the entry points and tests covering those components before deciding how realization dimensions and ensemble-specific metrics should fit. Done means ensemble forecasts are evaluated with an appropriate, configurable or automatically selected metric set alongside deterministic forecasts.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.