CarperAI / CarperAI/DRLX

Reward model inference

Open
#23 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
193
Forks
8
PR merge metrics
No merged PRs in 30d

Description

Need to add reward model inference for when the RM is a sizable model. Currently attempts to have RM on each GPU. This is problematic because there are many cases where RM is too big to fit alongside the denoiser model. Solution in LLM case is often to use Triton inference server or to put RM on one gpu while main model uses rest of GPUs. Should be explored further.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.