MetOffice / MetOffice/data_science_cop

era5 autoenvcoder torch lightning demo

Open
#65 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
26
Forks
10
Avg merge
7d 16m
Merged PRs (30d)
1

Description

once era5 5degree autoencoder is working, need to do the following
- use pytorch lighting for training
- guide on translating to lightning [https://lightning.ai/docs/pytorch/stable/starter/introduction.html](https://lightning.ai/docs/pytorch/stable/starter/introduction.html)
- tutorial basic [https://lightning.ai/docs/pytorch/stable/levels/core_skills.html](https://lightning.ai/docs/pytorch/stable/levels/core_skills.html)
- inter [https://lightning.ai/docs/pytorch/stable/levels/intermediate.html](https://lightning.ai/docs/pytorch/stable/levels/intermediate.html)
- adding in mlflow
- create pytorch logger module, with mlflow an extension
- create pyearthtools pipeline with era5 5 degree
- use pyearthtools lightning training
- create keras version of autoencoder
- create pyeartgtools keras trainer
- run training on azureml

Contributor guide

No contributing guide indexed for this repository

Research direction

No source files or tests are named. Start by locating the ERA5 5-degree autoencoder and its existing training flow, then review the linked PyTorch Lightning guidance before separating the requested Lightning, MLflow, pyearthtools, Keras, and AzureML work. Done means the requested training and pipeline variants are implemented and training runs successfully on AzureML.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure, jupyter-notebook, keras, python, pytorch
Domain
cloud, data-engineering, machine-learning, tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.