cleodora-forecasting / cleodora-forecasting/cleodora

API: Calculate the mean Brier score of a forecast

Open
#513 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
24
Forks
1
PR merge metrics
No merged PRs in 30d

Description

Best would be averaged over time instead of just the mean of all estimates, otherwise making a lot of tiny updates just before resolution would heavily skew the mean.

Contributor guide

Open the contributing guide

Research direction

Start by locating the API entry point that calculates Brier scores and trace how forecast estimates and resolution times are represented. Define how the score should be weighted across time, including updates made shortly before resolution, then add coverage for forecasts with uneven update timing. Done means the API reports the intended time-averaged Brier score.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
api, data
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.