DLR-RM / DLR-RM/stable-baselines3
[Question] why SAC train method updates the target every steps?
- Dominant language
- Python
- Stars
- 13.8k
- Forks
- 2.2k
- Avg merge
- 1h 35m
- Merged PRs (30d)
- 2
Description
### ❓ Question
Hi, I was looking into the code inside the SAC algorithm, and I encountered an odd [line here](https://github.com/DLR-RM/stable-baselines3/blob/2dce430161087c46b9e668f56f3c94c634eebff8/stable_baselines3/sac/sac.py#L290).
When comparing to make the update of the target, should not be using self._n_updates, or something similar instead gradient_step? such variable is always zero, hence, is always entering in the condition?.
There is maybe any specific reason to do it this way?
Thanks!
### Checklist
- [x] I have checked that there is no similar [issue](https://github.com/DLR-RM/stable-baselines3/issues) in the repo
- [x] I have read the [documentation](https://stable-baselines3.readthedocs.io/en/master/)
- [ ] If code there is, it is [minimal and working](https://github.com/DLR-RM/stable-baselines3/issues/982#issuecomment-1197044014)
- [ ] If code there is, it is formatted using the [markdown code blocks](https://help.github.com/en/articles/creating-and-highlighting-code-blocks) for both code and stack traces.
Contributor guide
Assessment
This issue has not been assessed yet.