NVIDIA-NeMo / NVIDIA-NeMo/RL

Have an example showing Reward/Critic runs in an independnent rayVirtualWorker

Open
#853 1 comment 0 reactions 0 assignees View on GitHub
external x-SBIntuitions
Dominant language
Python
Stars
2k
Forks
561
Avg merge
4d 5h
Merged PRs (30d)
145

Description

Trying NeMo-RL and would like to use specific GPUs for reward task.
According to the Design doc, Reward/Critic runs in an independent RayVirtualWorker .
https://docs.nvidia.com/nemo/rl/latest/design-docs/design-and-philosophy.html
But the [actual code](https://github.com/NVIDIA-NeMo/RL/blob/main/examples/run_grpo_math.py#L152-L159) is Environment running directly on ray.remote and without through RayVirtualWorker .

- This is because with simple MathEnv we didn't need lot of resources and that's why its just ray remote.
TO DO - have an example demonstrating more complex environments.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.