performance after fine-tuning

Hello,

This model was pre-trained on Protein, DNA, and RNA data collectively. Therefore, I initially expected that fine-tuning with RNA-only data would improve its performance specifically for RNA structure prediction. However, the results did not align with my expectations.

Despite adjusting parameters such as lowering the learning rate and tuning ema_decay, the more I fine-tuned the model, the worse its performance became on the test RNA dataset. In the end, the pretrained model remained the best-performing version.

Do you have any advice on why this might be happening or how I could improve the fine-tuning process?

The issue names no files, tests, or entry points. Start by locating the fine-tuning and RNA evaluation workflow, then compare the pretrained and fine-tuned runs, including learning rate and ema_decay settings. Done means identifying why fine-tuning degrades RNA test performance or documenting a validated improvement.