DLR-RM / DLR-RM/stable-baselines3

[Question] why SAC train method updates the target every steps?

Open
#2,158 2 comments 0 reactions 0 assignees View on GitHub
question
Dominant language
Python
Stars
13.8k
Forks
2.2k
Avg merge
1h 35m
Merged PRs (30d)
2

Description

### ❓ Question

Hi, I was looking into the code inside the SAC algorithm, and I encountered an odd [line here](https://github.com/DLR-RM/stable-baselines3/blob/2dce430161087c46b9e668f56f3c94c634eebff8/stable_baselines3/sac/sac.py#L290).

When comparing to make the update of the target, should not be using self._n_updates, or something similar instead gradient_step? such variable is always zero, hence, is always entering in the condition?.

There is maybe any specific reason to do it this way?

Thanks!

### Checklist

- [x] I have checked that there is no similar [issue](https://github.com/DLR-RM/stable-baselines3/issues) in the repo
- [x] I have read the [documentation](https://stable-baselines3.readthedocs.io/en/master/)
- [ ] If code there is, it is [minimal and working](https://github.com/DLR-RM/stable-baselines3/issues/982#issuecomment-1197044014)
- [ ] If code there is, it is formatted using the [markdown code blocks](https://help.github.com/en/articles/creating-and-highlighting-code-blocks) for both code and stack traces.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.