EducationalTestingService/rsmtool

View on GitHub

A Python package to facilitate research on building and evaluating automated scoring models.

Stars: 71
Forks: 21
Open beginner issues: 0
Indexed issues: 46

Dominant language: Python
License: Apache-2.0
Last GitHub push: Dec 27, 2024
Latest indexed: Sep 15, 2026
Contributing guide: No contributing guide
Code of conduct: No code of conduct
Beginner labels: help wanted good first issue
PR merge metrics: No merged PRs in 30d

Open All

46 issues indexed so far

Save RSMCompare outputs Open

enhancement help wanted

EducationalTestingService/rsmtool#125 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Improve confusion matrix and score distributions Open

enhancement

EducationalTestingService/rsmtool#151 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
More prominent warning for classifiers that do not support expected probabilities Open

enhancement help wanted

EducationalTestingService/rsmtool#196 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Add more analyses to RSMCompare/RSMSummarize Open

enhancement help wanted

EducationalTestingService/rsmtool#260 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Allow `features` to take a list of dictionaries Open

enhancement

EducationalTestingService/rsmtool#298 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Capture warning message from notebook Open

enhancement

EducationalTestingService/rsmtool#320 · 0 comments · 0 reactions · 1 assignee · Indexed Sep 15, 2026
The name of human score column in RSMPredict Open

EducationalTestingService/rsmtool#329 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Match the terminology between RSMTool and RSMCompare Open

EducationalTestingService/rsmtool#330 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Highlight non-zero differences in RSMCompare Open

enhancement help wanted

EducationalTestingService/rsmtool#332 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Improve RSMSummarize and RSMCompare tutorials Open

documentation

EducationalTestingService/rsmtool#333 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Look into parallelization Open

EducationalTestingService/rsmtool#334 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 15, 2026
More flexible treatment of continuous co-variates Open

enhancement

EducationalTestingService/rsmtool#335 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Provide descriptives for all features, not just those in the final model Open

enhancement

EducationalTestingService/rsmtool#336 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Consistently show in the report if the features were standardized Open

EducationalTestingService/rsmtool#337 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Compute metrics for all data once Open

enhancement

EducationalTestingService/rsmtool#338 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Add consistency to RSMSummarize output Open

enhancement good first issue

EducationalTestingService/rsmtool#340 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Generate predictions for responses with non-numeric human scores Open

enhancement

EducationalTestingService/rsmtool#341 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Kappa computation when predicted scores are on a different scale Open

bug

EducationalTestingService/rsmtool#342 · 10 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Resource warning when running tests Open

enhancement

EducationalTestingService/rsmtool#346 · 8 comments · 0 reactions · 1 assignee · Indexed Sep 15, 2026
Try solution for kernel RuntimeErrors Open

enhancement

EducationalTestingService/rsmtool#350 · 2 comments · 0 reactions · 1 assignee · Indexed Sep 15, 2026
Modify tests to fix warnings based on SKLL hyperparameters Open

enhancement help wanted

EducationalTestingService/rsmtool#360 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Capture incorrect input to correlations Open

enhancement

EducationalTestingService/rsmtool#365 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Consistent approach to warnings Open

enhancement help wanted

EducationalTestingService/rsmtool#367 · 5 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Replace all path operations and functions with `pathlib` Open

good first issue help wanted

EducationalTestingService/rsmtool#370 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Exclude unnecessary packages from conda-recipe Open

enhancement

EducationalTestingService/rsmtool#374 · 0 comments · 0 reactions · 1 assignee · Indexed Sep 15, 2026
Investigate whether we can use any visualizers from Yellowbrick Open

enhancement help wanted

EducationalTestingService/rsmtool#383 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Make sure that everything that accepts paths also accepts pathlib.Path Open

enhancement

EducationalTestingService/rsmtool#385 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Output file or directory in rsmpredict Open

enhancement

EducationalTestingService/rsmtool#396 · 3 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Allow config dictionary to contain dataframe rather than path to a file Open

EducationalTestingService/rsmtool#399 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Add suppress_warnings argument to various metrics Open

enhancement

EducationalTestingService/rsmtool#411 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 15, 2026
QWK generates runtime warning Open

EducationalTestingService/rsmtool#440 · 1 comment · 0 reactions · 1 assignee · Indexed Sep 15, 2026
Tests for notebooks Open

EducationalTestingService/rsmtool#470 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Add option for NaNs to be converted to 0s Open

EducationalTestingService/rsmtool#480 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Run nbqa on our notebooks and integrate into workflow Open

enhancement

EducationalTestingService/rsmtool#481 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Rename argument name in `configuration_parser.check_flag_column()` Open

enhancement

EducationalTestingService/rsmtool#485 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Consider refactoring SMD code/changing documentation. Open

EducationalTestingService/rsmtool#489 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Deprecate _modify_eval_columns_to_ensure_version_compatibilty in RSMCompare Open

EducationalTestingService/rsmtool#490 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Rewrite docstrings to match scikit-learn Style Open

documentation enhancement help wanted

EducationalTestingService/rsmtool#491 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Add support for SKLL Voting Learners Open

enhancement

EducationalTestingService/rsmtool#512 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Use raw predictions in tutorial Open

documentation good first issue

EducationalTestingService/rsmtool#524 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Simplify sign in subset_feature_selection Open

enhancement

EducationalTestingService/rsmtool#529 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Streamline the code for flagging warnings in the reports Open

good first issue help wanted

EducationalTestingService/rsmtool#539 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
New method for predicting from coefficients Open

enhancement

EducationalTestingService/rsmtool#545 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Add support for sections in rsmxval Open

EducationalTestingService/rsmtool#557 · 0 comments · 0 reactions · 1 assignee · Indexed Sep 15, 2026
Log additional intermediate files to W&B Open

enhancement

EducationalTestingService/rsmtool#630 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Slim installation option or model inference export option Open

EducationalTestingService/rsmtool#689 · 5 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026

EducationalTestingService/rsmtool

Get new issues in your inbox