Policy gradients

Hi Aurelien,

Thanks a lot for your book! It has been amazing. I have a question pertaining to policy gradients (Chapter 16 -- reinforcement learning). In the implementation, a policy gradient is calculated in **2** steps: 1) finding the relationship between policy parameters and an action, and 2) finding the relationship between the action and the value, by Monte Carlo sampling.

I was wondering if this is a typical implementation of policy gradients? Naively, would it be easier to directly calculate the policy gradient by Monte Carlo --and find the relationship between the policy parameters and the Value -- in a single step?

Thank you in advance!