EducationalTestingService / EducationalTestingService/rsmtool
Improve confusion matrix and score distributions
- Dominant language
- Python
- Stars
- 71
- Forks
- 21
- PR merge metrics
- No merged PRs in 30d
Description
Right now, the confusion matrix and the score distribution plots rely entirely on the test set human scores. This means that if there are, say, 6s in the training data but none in the test data, the confusion matrix will only be 5x5 and the score distribution plot will only show 5 bars. Perhaps it would be better to take the union of the human scores from both the training and the test sets?
Contributor guide
No contributing guide indexed for this repository
Research direction
Locate the code that builds the confusion matrix and score distribution plots, then trace how human scores are collected from the training and test sets. Update the plots so their score categories use the union of both sets, and verify that scores present only in training still appear in both visualizations.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data-visualization, machine-learning
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100