🚀 Recent Updates Summary for ROLL Project

Hello everyone!
Thank you for your interest in ROLL.
We continue to iterate and improve the ROLL project. Below is a summary of recent updates, categorized for your reference.

✅ New Features (New Functionality)

- vl process_image function to handle list of images: Support for multi-image input in VL RLRV pipelines.
- support more test case types for code reward worker: Extended test case types for the code reward worker.
- add rlvr 7B 8gpus config: Added a new RLVR 7B configuration for use with 8 GPUs.
- add env action global limiter: Introduced a global rate limiter to control traffic pressure on tool services.
- add processor save support in Megatron strategy: Added support for saving processors within the Megatron strategy.
- add webshop env: Added support for the WebShop environment.
- solve port conflict: Resolved occasional port conflict issues.

🔧 Configuration Optimization (Performance & Config Refactor)

- set default max_length/max_new_tokens to 8192: Set the default maximum generation length to 8192.
- set vllm enforce_eager=False (use cudagraph) by default: Enabled CudaGraph by default to accelerate inference.
- refactor mcore register_config: Refactored the MCore configuration registration logic.
- extract gpu & ray utils: Extracted GPU and Ray utility functions to improve modularity.
- refine rlvr config: Optimized RLVR-related configurations.
- cast logits to float, maybe cause oom: Casted logits to float type to avoid potential OOM issues.
- add create_container.sh: Added a script for building containers to simplify deployment.

🛠️ Bug Fixes

- support model update under deepspeed with vllm v1: Fixed an issue with model updates when using Deepspeed and vLLM v1.
- Qwen2VL RAM leak: Fixed a memory leak issue in Qwen2VL.
- fix metrics cal: Corrected validation metric calculation logic.
- topr loss: Fixed an issue with the TOPR loss calculation.
- fix default envs & remove ACCL_TUNING_LEVEL env: Fixed default environment variables and removed redundant ones.
- fix max_gpu_num: Corrected the logic for determining the maximum number of GPUs.
- fix model load fa setting: Fixed an error in FA settings during model loading.

For the full commit history, please refer to the GitHub Commit History.

Thank you for your attention and support!
If you have any questions, feedback, or suggestions, feel free to leave a comment in the Issues section.