MetOffice / MetOffice/ml_weather_tutorial
Improvements to hyperparameter tuning tutorial.
@stevehadd is already working on this.
Since Nov 8, 2022.
- Dominant language
- Jupyter Notebook
- Stars
- 4
- Forks
- 2
- PR merge metrics
- No merged PRs in 30d
Description
From review by @HarveySouth
Could: extend hyperparameter tuning tutorial with a test set evaluation from the best model. It would be interesting to discuss, but my own idea of reporting a good estimate of model performance involves finding the best hyperparameters that maximize the desired metric on the validation set, and then applying exactly the same training process to the model, trained on the training and validation data aggregated, tested against the test set since step/epoch are hyperparameters the final model inference metric is reportable. It is of course possible to report just the whole training process (which should be done anyway) and leave interpretation to the user/reader, but not sure what a best practice in this case would look like.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.