Glavin001 / Glavin001/IssueBot
Detect which classes are difficult to predict
- Dominant language
- JavaScript
- Stars
- 14
- Forks
- 5
- PR merge metrics
- No merged PRs in 30d
Description
If you look at the scores you can see that certain classes are very difficult to predict and should be ignored.
### Example
For example, take https://github.com/Glavin001/atom-beautify
#### Before
`We correctly predicted 573 of 835 labelled issues, obtaining a score of 68.62275449101797%!`
```
precision recall f1-score support
add-beautifier 0.97 1.00 0.99 37
add-beautifier-option 1.00 0.93 0.97 30
add-language 1.00 1.00 1.00 144
bug 1.00 0.94 0.97 103
deprecated-atom-api 0.93 0.74 0.82 19
enhancement 1.00 1.00 1.00 134
feature 1.00 1.00 1.00 58
high priority 1.00 0.32 0.48 95
in-progress 1.00 0.36 0.53 11
invalid 1.00 0.56 0.72 66
mac 1.00 0.50 0.67 4
published 1.00 0.15 0.26 164
question 1.00 0.93 0.97 168
quick-todo 0.00 0.00 0.00 12
third-party bug 0.96 0.99 0.98 139
update third-party dependency 1.00 0.31 0.47 13
user-update-atom 0.00 0.00 0.00 4
waiting-for-user-information 0.81 0.53 0.64 32
windows 0.93 1.00 0.96 25
wontfix 0.00 0.00 0.00 7
avg / total 0.97 0.75 0.80 1265
```
#### After
Ignore the following labels to achieve 97%
``` python
ignore_labels = ['quick-todo', 'wontfix', 'user-update-atom', 'in-progress', 'pending-publication', 'published', 'waiting-for-user-information', 'high priority']
```
```
precision recall f1-score support
add-beautifier 0.97 1.00 0.99 37
add-beautifier-option 1.00 0.93 0.97 30
add-language 1.00 1.00 1.00 144
bug 1.00 0.97 0.99 103
deprecated-atom-api 1.00 0.84 0.91 19
enhancement 1.00 1.00 1.00 134
feature 1.00 1.00 1.00 58
invalid 1.00 0.53 0.69 66
mac 1.00 0.50 0.67 4
question 1.00 0.95 0.98 168
third-party bug 0.97 0.99 0.98 139
update third-party dependency 1.00 0.31 0.47 13
windows 0.93 1.00 0.96 25
avg / total 0.99 0.94 0.96 940
```
---
Resources:
- http://stats.stackexchange.com/questions/117654/what-does-the-numbers-in-the-classification-report-of-sklearn-mean
- http://scikit-learn.org/stable/modules/generated/sklearn.metrics.precision_recall_fscore_support.html
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.