huggingface / huggingface/deep-rl-class
[HANDS-ON mistake] Unit 4 Policy gradient - mistake in exercise explanation
- Dominant language
- MDX
- Stars
- 5k
- Forks
- 811
- PR merge metrics
- No merged PRs in 30d
Description
On the hands on section of unit 4 (policy gradient), when building the REINFORCE policy, the exercise has an intentional bug causing the error message
`ValueError: The value argument to log_prob must be a Tensor`.
The explanation says that this is error occurs because we should sample the probability distribution rather than argmax it - but that's not the reason this error occurs.
The error occurs because we use `np.argmax(Categorical(probs))` rather than `torch.argmax(probs)`.
Indeed, we would nevertheless like to substitute the argmax with sampling, but for different reasons, so the reasoning in the notebook is somewhat misleading.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.