Add absolute accuracy
- Dominant language
- Python
- Stars
- 56
- Forks
- 19
- PR merge metrics
- No merged PRs in 30d
Description
Webapp should plot predicted versus true probabilities. Bin predicted probabilities, e.g. all between .4 and .5 and see if the average is ~.45. I'm looking for something like [this](http://d39mdc1iz3oyyf.cloudfront.net/content/erj/39/2/366/F2.large.jpg).
We can also get an AUC score for this, but unlike ROC, the best is 0.5 and the worst is 1.0 or 0.0. We should generate this AUC score for every model and store it in the `results.evaluations` table
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by locating the webapp's model-evaluation flow and the results.evaluations table, then trace how prediction outputs are currently presented. Confirm the requested probability binning and AUC convention from the issue. Done means the webapp shows predicted versus true probabilities and stores the requested score for every model.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data-visualization, databases, machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100