cmu-delphi / cmu-delphi/forecast-eval

retool exclusion criteria for location overlap

Open
#151 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
R
Stars
6
Forks
3
PR merge metrics
No merged PRs in 30d

Description

Seems like we are potentially throwing a lot of possibly viewable data away using the current criteria that requires full location overlap at all times. E.g. LANL submits a basically full set of hospitalization forecasts every week. but maybe 1-2 weeks they only submitted one location and then that would get filtered down to just that one location even if there are another 30 weeks where we have great overlap on 50 locations. Is there a relatively simple way to, say, drop the few weeks that have an anomalously low number of submission locations?
![image](https://user-images.githubusercontent.com/1280767/127516474-6e426319-f748-414a-9ceb-6fe3cad3ea9f.png)
![image](https://user-images.githubusercontent.com/1280767/127516485-6d3313e9-1ff9-407c-8e5d-935f2282e1b0.png)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by tracing the current criteria that requires full location overlap at all times in the forecast-eval code. Review the LANL hospitalization forecast examples and determine how anomalously low-location weeks should be identified. Done means normal weeks retain their available locations while isolated low-submission weeks no longer discard the broader overlapping data.

Written by the indexing model from the issue text.

Assessment

Tech stack
r
Domain
data
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.