LOLA Policy Gradient Target Computation
- Lingua principale
- Jupyter Notebook
- Stelle
- 156
- Fork
- 38
- Metriche di merge delle PR
- Nessuna PR unita negli ultimi 30g
Descrizione
Hello, thank you for open-sourcing the code! :-)
The code is really helpful in understanding the papers deeper.
I am interested in LOLA, especially its policy gradient method ([lola/train_pg.py](https://github.com/alshedivat/lola/blob/master/lola/train_pg.py)).
As mentioned in the paper, this implementation shows the actor-critic method.
However, I could not fully understand the target computation code:
`self.target = self.sample_return + self.next_v` ([code](https://github.com/alshedivat/lola/blob/master/lola/networks.py#L155)).
According to the [reference](http://incompleteideas.net/book/bookdraft2017nov5.pdf) (chapter 13, page 274, one-step actor-critic pseudocode), I wonder whether the target computation should use the step reward (i.e., reward at timestep t) instead of the return.
Thank you for your time and consideration!
Guida per i contributori
Nessuna guida per i contributori indicizzata per questo repository
Valutazione
Questa issue non è ancora stata valutata.