GenCast 0p25deg Operational performs poorly on HRES/GFS input

We are currently testing the GenCast 0p25deg Operational model using different initial conditions, specifically HRES and GFS. We observe a significant drop in performance when the model is applied to either of these inputs.

To illustrate this, we examined the predicted 2-meter temperature (T2M) at 2022-02-01 12UTC. We found that:
• The GenCast 0p25deg Operational model, when initialized with HRES data, produces predictions with an RMSE approximately 4 times higher than the RMSE of HRES itself compared to ERA5.
• The same issue persists when we apply the model to GFS inputs.
• Surprisingly, even the fine-tuned version of GenCast 0p25deg on HRES shows similarly poor results.

This raises the question:

Why does the Operational model — even after being fine-tuned on HRES — still perform so poorly when initialized with HRES?

We would appreciate any clarification regarding:
• Whether the model was intended to generalize to different operational initial conditions like HRES/GFS,
• If additional pre-processing is needed for compatibility,
• Or if this behavior is expected due to limitations in fine-tuning.

Thank you for your help!

Figure description:
The figure below compares 2-meter temperature (T2M) fields at 2022-02-01 12UTC from various sources:

• Top-left: Prediction generated by GenCast 0p25deg Operational checkpoint, initialized with HRES. This prediction is an ensemble mean of 4 members (ens size = 4)
• Bottom-left: Prediction from the original GenCast 0p25deg checkpoint, also initialized with HRES. This prediction is an ensemble mean of 4 members (ens size = 4).
• Top-right: ERA5 reanalysis data used as the ground truth.
• Bottom-right: HRES forecast at the same time.

These visualizations are intended to highlight the discrepancy in performance between different GenCast configurations and the ERA5 baseline.