Off-policy RL algorithms support

Hi, first of all, great work. This is a very useful library for research on RL and NLP. It will be very helpful if it's possible to add off-policy RL methods like Q-learning, SAC, etc. along with benchmarks.

Also, new offline RL methods applied to NLP like ILQL can be very interesting for human alignment, and support for such methods will further enhance the value of this codebase.