Eclectic-Sheep / Eclectic-Sheep/sheeprl
Pure python training, evaluation and rollout documentation request.
- Dominant language
- Python
- Stars
- 439
- Forks
- 66
- PR merge metrics
- No merged PRs in 30d
Description
Hi everyone,
As a professional who has worked with a few RL frameworks in the past, I can confidently say that this is one of the cleanest, most user-friendly, and advanced RL library I've encountered. In fact, I'm planning to introduce it to my team as our future RL framework, and we're excited to contribute to its development. I especially appreciate the Dreamer implementations and the informative blog posts – amazing work!
Based on my experience with RL framework development, I have a few recommendations that could make this library even more appealing to a wider range of engineers:
**Pure Python Examples:**
While I understand the value of Hydra as a tool for configuration management and rapid experimentation, it can be intimidating for newcomers. To address this barrier and encourage broader adoption, I recommend creating 3-4 pure Python documentation/tutorial examples demonstrating training, evaluation, and rollout using existing Lagos functionalities. This approach has been successful in attracting large-scale users to other RL libraries.
Here are some examples that might be helpful:
Similar to: https://github.com/araffin/rl-tutorial-jnrr19/blob/sb3/1_getting_started.ipynb
A bit more complex (but valuable): https://github.com/araffin/rl-tutorial-jnrr19/blob/sb3/1_getting_started.ipynb
Integrating with Isaac Gym: https://docs.omniverse.nvidia.com/isaacsim/latest/isaac_gym_tutorials/tutorial_advanced_rl_stable_baselines.html
**Tips and Tricks:**
As we all know, RL algorithms are sensitive to hyperparameters and often require specific techniques like action masking, observation normalization, and reward scaling to be successful on new environments. Given the library's advanced capabilities with World Models, sharing insights and best practices on these topics would be incredibly valuable to the community (including myself!). Here are some examples from other libraries:
https://stable-baselines.readthedocs.io/en/master/guide/rl_tips.html
https://maze-rl.readthedocs.io/en/latest/best_practices_and_tutorials/tricks_of_the_trade.html
**Transitioning to Hydra:**
Once users become comfortable with the library's fundamentals, they'll naturally progress towards exploring scalability and advanced experimentation, which is where Hydra shines. Consider creating a separate tutorial or example notebook showcasing how to leverage Hydra and Sheep-RL's train and evaluate functionalities to achieve this transition smoothly.
I hope you find these recommendations helpful. Best of luck to the developers!
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.