MetOffice / MetOffice/data_science_cop
era5 autoenvcoder torch lightning demo
- Dominant language
- Jupyter Notebook
- Stars
- 26
- Forks
- 10
- Avg merge
- 7d 16m
- Merged PRs (30d)
- 1
Description
once era5 5degree autoencoder is working, need to do the following
- use pytorch lighting for training
- guide on translating to lightning [https://lightning.ai/docs/pytorch/stable/starter/introduction.html](https://lightning.ai/docs/pytorch/stable/starter/introduction.html)
- tutorial basic [https://lightning.ai/docs/pytorch/stable/levels/core_skills.html](https://lightning.ai/docs/pytorch/stable/levels/core_skills.html)
- inter [https://lightning.ai/docs/pytorch/stable/levels/intermediate.html](https://lightning.ai/docs/pytorch/stable/levels/intermediate.html)
- adding in mlflow
- create pytorch logger module, with mlflow an extension
- create pyearthtools pipeline with era5 5 degree
- use pyearthtools lightning training
- create keras version of autoencoder
- create pyeartgtools keras trainer
- run training on azureml
Contributor guide
No contributing guide indexed for this repository
Research direction
No source files or tests are named. Start by locating the ERA5 5-degree autoencoder and its existing training flow, then review the linked PyTorch Lightning guidance before separating the requested Lightning, MLflow, pyearthtools, Keras, and AzureML work. Done means the requested training and pipeline variants are implemented and training runs successfully on AzureML.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure, jupyter-notebook, keras, python, pytorch
- Domain
- cloud, data-engineering, machine-learning, tooling
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100