dssg / dssg/triage

Metric calculation is bogus

Open
#223 5 comments 0 reactions 0 assignees View on GitHub
needs-discussion
Dominant language
Jupyter Notebook
Stars
201
Forks
62
PR merge metrics
No merged PRs in 30d

Description

precision calculation is currently taking predictions for several as of dates, and calculating precision across all of them together, resulting in bogus results. need to look at how to do it for each as of date separately and then aggregate or something more reasonable.

Contributor guide

Open the contributing guide

Research direction

Start by tracing the precision calculation and how predictions from several as-of dates are combined. Determine a reasonable per-date calculation and aggregation approach, then verify that precision is no longer computed by pooling all dates together.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.