EducationalTestingService / EducationalTestingService/rsmtool

Provide descriptives for all features, not just those in the final model

Open
#336 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
71
Forks
21
PR merge metrics
No merged PRs in 30d

Description

Aoife: In a few cases features of interest didn't make it into the final NNLS model. Unfortunately that also meant that they were excluded from all analyses including  feature descriptions. We were able to get feature descriptions by rerunning with Linear Regression, but would it be possible to include feature descriptives and inter-feature correlations for all features listed in the features file, not just the ones that make it into the final model? 

Jeremy: We can address this in the refactored version of RSMTool.

The feature descriptives and inter-feature correlations get calculated by the run_training_analyses() method in the Analyzer class. Currently, this method subsets the data using a selected_features parameter, but we could use something like {{all_features }}instead.

I'm assuming we would want to give users the option to include all features (whether or not they drop out of the model) or selected features in the analysis, correct? If so, we'll have to add a new configuration parameter to that effect.

Aoife: Making it an option sounds reasonable to me

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.