huggingface / huggingface/deep-rl-class

[HANDS-ON mistake] Unit 4 Policy gradient - mistake in exercise explanation

Open
#582 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
MDX
Stars
5k
Forks
811
PR merge metrics
No merged PRs in 30d

Description

On the hands on section of unit 4 (policy gradient), when building the REINFORCE policy, the exercise has an intentional bug causing the error message
`ValueError: The value argument to log_prob must be a Tensor`.
The explanation says that this is error occurs because we should sample the probability distribution rather than argmax it - but that's not the reason this error occurs.
The error occurs because we use `np.argmax(Categorical(probs))` rather than `torch.argmax(probs)`.
Indeed, we would nevertheless like to substitute the argmax with sampling, but for different reasons, so the reasoning in the notebook is somewhat misleading.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.