dmarx / dmarx/make_for_datascience

Figure out how to handle experimentation with permutations of collections of nodes

Open
#27 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
HTML
Stars
3
Forks
0
PR merge metrics
No merged PRs in 30d

Description

Let's say we we have P imputation strategies we want to consider, Q feature selection approaches, and R model architectures. Wrapping this 3 step sequence into a "model", we will end up with P*Q*R models. The way things work right now, we'd need to write P*Q*R separate model specs, where each one's train_model function wraps a separate imputation and feature selection operation. E.g.:

# logreg_ImputeKnn_FeatSelLasso.R
train_model <- function(X, Y, ...){
candidate_feats = read.table("task0/data/processed/abt_features.txt", stringsAsFactors=FALSE)[,1]
X_imp = knn_impute(X, candidate_feats)
feats = lass_feat_sel(X, candidate_feats)

rhs = paste(feats, collapse=" + ")
formula = paste0('target ~ ', rhs)

X$target = Y
glm(formula, data=X, family=binomial)
}

It would be cool if there were a way to abstract out building chunks of the pipeline as permutations on sets of nodes like this. This would solve both the issue of writing P*Q*R modeling scripts when really we only have 3 different models, and additionally would solve the issue of the data pre-processing we're experimenting with being disguised from the pipeline inside the train_model function. If possible, every single data processing step should be represented in the pipeline as a stand-alone operation.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.