INRIA / INRIA/scikit-learn-mooc

Explaining train-validation-test split in M3

Open
#439 0 comments 0 reactions 0 assignees View on GitHub
video
Dominant language
Jupyter Notebook
Stars
1.4k
Forks
600
Avg merge
6d 20h
Merged PRs (30d)
2

Description

The first time we use a train-validation-test split for parameter tuning is at the beginning of M3 in [this exercise](https://inria.github.io/scikit-learn-mooc/python_scripts/parameter_tuning_sol_02.html), as we (silently) use CV within the training set and then we score the model in a fixed test set (as in [this figure](https://github.com/ArturoAmorQ/scikit-learn-mooc/blob/main/figures/cross_validation_train_test_diagram.png)).

Nowadays we explain this concept, as well as nested cross-validation, in the [Evaluation and hyperparameter tuning notebook](https://inria.github.io/scikit-learn-mooc/python_scripts/parameter_tuning_nested.html). But it would be better if we had a **video** to dynamically show the swapping folds in the train-validation set, similar to the [Validation of a model video](https://inria.github.io/scikit-learn-mooc/predictive_modeling_pipeline/02_numerical_pipeline_video_cross_validation.html).

See also #755.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.