INRIA / INRIA/scikit-learn-mooc
Explaining train-validation-test split in M3
- Dominant language
- Jupyter Notebook
- Stars
- 1.4k
- Forks
- 600
- Avg merge
- 6d 20h
- Merged PRs (30d)
- 2
Description
The first time we use a train-validation-test split for parameter tuning is at the beginning of M3 in [this exercise](https://inria.github.io/scikit-learn-mooc/python_scripts/parameter_tuning_sol_02.html), as we (silently) use CV within the training set and then we score the model in a fixed test set (as in [this figure](https://github.com/ArturoAmorQ/scikit-learn-mooc/blob/main/figures/cross_validation_train_test_diagram.png)).
Nowadays we explain this concept, as well as nested cross-validation, in the [Evaluation and hyperparameter tuning notebook](https://inria.github.io/scikit-learn-mooc/python_scripts/parameter_tuning_nested.html). But it would be better if we had a **video** to dynamically show the swapping folds in the train-validation set, similar to the [Validation of a model video](https://inria.github.io/scikit-learn-mooc/predictive_modeling_pipeline/02_numerical_pipeline_video_cross_validation.html).
See also #755.
Contributor guide
Assessment
This issue has not been assessed yet.