Issue on checkpoints
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1k
- Forks
- 174
- PR merge metrics
- No merged PRs in 30d
Description
Thanks for sharing this project. It is great to see such a powerful foundation model!
I have been testing the open-source model and the corresponding checkpoints provided by your team in ERA5. We used the ERA5 data of May 2024 as input and followed the steps in https://microsoft.github.io/aurora/example_era5.html. We found that RMSE was similar to IFS, but there was a gap between GraphCast. The results from our experiments have not been as promising, and we are unsure if this might be due to the checkpoint we are using or if there are specific aspects of the model or experimental setup that we may have overlooked.
Could you kindly confirm whether the checkpoint we are using corresponds to the model that delivered the best results in your published work? If not, are there other checkpoints or configurations that we should consider to replicate the performance presented in the paper? Additionally, any insights on potential pitfalls or nuances in the implementation would be greatly appreciated.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the ERA5 example at https://microsoft.github.io/aurora/example_era5.html and identify which checkpoint and configuration were used. Compare those details with the checkpoint associated with the published results, then document whether the setup can reproduce the reported GraphCast comparison or whether another checkpoint is required.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100