dssg / dssg/triage

Incorrect as-of-date year in Postmodeling plots?

Open
#656 0 comments 0 reactions 0 assignees View on GitHub
postmodeling
Dominant language
Jupyter Notebook
Stars
201
Forks
62
PR merge metrics
No merged PRs in 30d

Description

In `dirtyduck` the last prediction `as-of-date` is `2017-12-01`. The following table is a fragment of the final models from the inspections part of `dirtyduck`:

```| model_group_id | model_type | hyperparameters | feature_groups | models | times | precision@10% |
|--------------+-----------------------------------------+-----------------------------------------------------------------------------------------+----------------------------------------+-------------+------------------------------------+------------------------------|
| 1 | sklearn.dummy.DummyClassifier | {"strategy": "prior"} | {inspections} | {1,4,7} | {2015-12-01,2016-12-01,2017-12-01} | {" 0.295"," 0.242"," 0.251"} |
| 2 | sklearn.dummy.DummyClassifier | {"strategy": "uniform"} | {inspections} | {2,5,8} | {2015-12-01,2016-12-01,2017-12-01} | {" 0.295"," 0.242"," 0.251"} |
| 3 | sklearn.dummy.DummyClassifier | {"strategy": "most_frequent"} | {inspections} | {3,6,9} | {2015-12-01,2016-12-01,2017-12-01} | {" 0.295"," 0.242"," 0.251"} |

```

Creating the plot for performance over time:

```python
audited_models_class.plot_prec_across_time(param_type='rank_pct',
param=10,
baseline=True,
baseline_query=params.baseline_query,
metric='precision@',
figsize=params.figsize)
```

Gives you the following plot:

![4fd47e296418feefce5cad1683930772e3edf253](https://user-images.githubusercontent.com/494528/55018879-bf672080-4fc1-11e9-8811-d4c67c34469d.png)

Note the years.

`Audition` makes a similar plot:

![metric_over_time_precision@10_pct](https://user-images.githubusercontent.com/494528/55019442-e7a34f00-4fc2-11e9-9366-450ab58e0b45.png)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.