microsoft / microsoft/aurora

Inference results using ERA5 data

Open
#76 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1k
Forks
174
PR merge metrics
No merged PRs in 30d

Description

Hi, thanks for releasing the strong foundation model!

I am trying to get the inference result of the released AuroraSmall checkpoint using ERA5 data.
I download the ERA5 data from Climate Data Store, which has 4 surf-vars and 5 atmos-vars with 8 pressure levels (differenct from the settings in the paper).
After that, I follow the guidance in the document to build the inference pipeline.
Also, I implement a RMSE metric by myself to evaluate the results.

However, I get a pretty bad performance on some of the variables.
I have known that the result is sensitive to different combinations of input variables, and I believe that the optimized inference result need to be generated by HRES-T0 dataset.
I would like to check if there is anything else I miss during my implementation that leads to the bad performance.
Or could you provide the inference result (the RMSE value of each variable, ex. 2t, 10u, t850, u500, ...) using a year (maybe 2020) of ERA5 data as a reference value?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the documented inference pipeline for the AuroraSmall checkpoint and compare the ERA5 input variables and pressure levels with the settings described in the paper. Reproduce the reported RMSE evaluation, then determine whether the variable-specific performance gap is caused by the ERA5 configuration or implementation; a useful resolution would document the cause or provide 2020 ERA5 reference RMSE values.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.