huggingface / huggingface/deep-rl-class

[QUESTION] How P(τ;θ) disappeared while estimating the gradients using trajectory samples?

Open
#495 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
MDX
Stars
5k
Forks
811
PR merge metrics
No merged PRs in 30d

Description

I am referring to the gradient derivation [here](https://huggingface.co/learn/deep-rl-course/unit4/pg-theorem#optional-the-policy-gradient-theorem).

The paragraph where the instructor claimed "we can approximate the likelihood ratio policy gradient with sample-based estimate" then term of P(τ;θ) (probability of trajectory τ given the parameters θ) disappeared in the subsequent summation. Why?

I asked the same question on the discord study-group ([here](https://discord.com/channels/879548962464493619/971379033642266654/1209056045222334504)) but got no response.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.