climate-analytics-lab / climate-analytics-lab/ClimateBench2

Infrastructure: Tier I/II/III aggregated scoring and weighting

Open
#100 1 comment 0 reactions 0 assignees View on GitHub
infrastructure
Dominant language
Shell
Stars
1
Forks
1
PR merge metrics
No merged PRs in 30d

Description

## Description
Design and implement the overall scoring scheme that combines results across tiers.

## Open questions
- How to weight individual metrics within each tier
- How to combine Tier I pass/fail with Tier II continuous scores
- Treatment of extended vs core metrics in overall score
- Presentation format (leaderboard, radar plots, etc.)
- Whether Tier I failures should gate Tier II/III scoring entirely

## Notes
- This is a protocol-level design question that needs community input
- Start with a simple additive scheme and iterate

Contributor guide

No contributing guide indexed for this repository

Research direction

No files, tests, or entry points are named. Start by reviewing the existing tier metrics and scoring outputs, then gather community input on metric weights, tier combinations, extended versus core metrics, presentation, and gating. Done means an agreed protocol-level scoring scheme is documented and ready to implement.

Written by the indexing model from the issue text.

Assessment

Domain
analytics
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.