Questions about the Integration of Off-policy Improvements

Dear Slime Team,

First and foremost, I would like to express my sincere appreciation for your outstanding work on the Slime framework. It has been an invaluable resource for the community.

I am writing to inquire about the potential roadmap for integrating off-policy improvements or experience replay mechanisms into the framework. Specifically, I am interested in whether there are plans to adopt techniques such as the Decoupled PPO proposed in AReal, or the Experience Replay strategy demonstrated in RLEP.

Could you kindly provide some insight into the following:

• Feasibility & Complexity: Based on the current architecture of Slime, how would you assess the technical feasibility and complexity of implementing such experience replay or off-policy algorithms?
• Strategic Perspective: I would greatly value your team's professional perspective on the role of off-policy algorithms within the current landscape of LLM post-training. Do you view these as a priority direction for the framework?

Thank you very much for your time and your continued contributions to the field.

No files, tests, or entry points are identified in the issue. Clarify whether the request is for a roadmap discussion or an implementation, then define the targeted off-policy or experience-replay approach and what completion would require.