huggingface / huggingface/course

Missing comma in the $J_{GRPO}$ equation

Open
#1,174 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
MDX
Stars
4.2k
Forks
1.4k
Avg merge
13m
Merged PRs (30d)
1

Description

In Chapter 12 of LLM course, in a file named `3b.mdx`, where the targent function is explained [here](https://github.com/huggingface/course/blob/main/chapters/en/chapter12/3b.mdx#step-3-policy-update), there is a missing comma after the first $A_i$, which can be misleading as we are calculating the $\min$. Here the correct equation:

$$J_{GRPO}(\theta) = \left[\frac{1}{G} \sum_{i=1}^{G} \min \left( \frac{\pi_{\theta}(o_i|q)}{\pi_{\theta_{old}}(o_i|q)} A_i, \text{clip}\left( \frac{\pi_{\theta}(o_i|q)}{\pi_{\theta_{old}}(o_i|q)}, 1 - \epsilon, 1 + \epsilon \right) A_i \right)\right]- \beta D_{KL}(\pi_{\theta} \|\| \pi_{ref})$$

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.