cmu-delphi / cmu-delphi/exploration-tooling

Consider branching targets over forecast_date

Open
#182 0 comments 0 reactions 1 assignee Claimed by @dshemetov View on GitHub
Dominant language
HTML
Stars
2
Forks
2
PR merge metrics
No merged PRs in 30d

Description

We do this in the prod pipelines, but not explore. This has a few benefits:

- built-in per date caching, so redoing backtests on a slightly different date range is faster
- removes the caching hack we do with epix_slide_simple (see the bottom of the joined_archive_data target)
- removes the need for epix_slide_simple entirely

I tried extending this to explore and ran into a few issues:

- I made nested tar_map calls, first over forecast_date and then over forecasters and their parameters; this worked fine
- however, I got tangled in the scoring; I now realize that we can simply score each forecaster-date target and then aggregate after

Branching like this should save a lot of time in exploration, since adding a single date of forecasting will not require rerunning a 3 hour pipeline.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.