Integration between NeMo and NeMo-RL for audio LLMs

Hi, we are training our audio LLMs using the code provided in https://github.com/tiiuae/aiccu-audio-nemo/tree/main/nemo/collections/speechlm . Under the hood these are just MegatronLM models.

Now we'd like to apply RL training to these models and we'd like to use NeMo-RL for that. What's the easiest way to load our models into NeMo-RL? Is this something that's relatively simple to add or do you think it's a really big task?