cmu-delphi / cmu-delphi/forecast-eval

Forecasters with multiple forecasts for same target date with same score failing on tsibble key/index selection

Open
#157 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
R
Stars
6
Forks
3
PR merge metrics
No merged PRs in 30d

Description

Forecasters may have multiple forecasts for the same week end date with the same score, and tsibbles need unique keys/indexes. Simply removing dups before setting the tsibble (something like this: `filteredScoreDf = filteredScoreDf[!duplicated(filteredScoreDf[c("Week_End_Date" , "Forecaster")]), ]`) will mess up how the faceted plots work.

Right now this is only happening with the CU-select forecaster. It doesn't show a plot for US deaths bc it contains week end date / score dups.

Tsibble code:
```
# Fill gaps so there are line breaks on weeks without data
filteredScoreDf = filteredScoreDf %>%
as_tsibble(key = c(Forecaster, ahead), index = Week_End_Date) %>%
group_by(Forecaster, Forecast_Date, ahead) %>%
fill_gaps(.full = TRUE)
```

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the shown filteredScoreDf -> as_tsibble(...)->fill_gaps pipeline and reproduce the CU-select case for US deaths. Trace why repeated Week_End_Date/score rows violate the tsibble key/index constraints; done means the plot renders while faceting remains correct.

Written by the indexing model from the issue text.

Assessment

Tech stack
r
Domain
data-visualization
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.