Have an example showing Reward/Critic runs in an independnent rayVirtualWorker
Open
external
x-SBIntuitions
- Dominant language
- Python
- Stars
- 2k
- Forks
- 561
- Avg merge
- 4d 5h
- Merged PRs (30d)
- 145
Description
Trying NeMo-RL and would like to use specific GPUs for reward task.
According to the Design doc, Reward/Critic runs in an independent RayVirtualWorker .
https://docs.nvidia.com/nemo/rl/latest/design-docs/design-and-philosophy.html
But the [actual code](https://github.com/NVIDIA-NeMo/RL/blob/main/examples/run_grpo_math.py#L152-L159) is Environment running directly on ray.remote and without through RayVirtualWorker .
- This is because with simple MathEnv we didn't need lot of resources and that's why its just ray remote.
TO DO - have an example demonstrating more complex environments.
Contributor guide
Assessment
This issue has not been assessed yet.