[Feature Request] DQN Loss importance sampling
Open
@vmoens is already working on this.
Since Mar 8, 2023.
enhancement
- Dominant language
- Python
- Stars
- 3.6k
- Forks
- 484
- Avg merge
- 1d 1h
- Merged PRs (30d)
- 207
Description
Motivation
I believe the current DQN Losses don't apply importance sampling weights. This is almost always applied when using a PER Buffer.
Solution
The PER buffer outputs "_weight" in info, this if in inputTensorDict, could be applied at time of loss calculation
Alternatives
element-wise loss could be returned with any transforms or weighting to be applied later on
Checklist
- I have checked that there is no similar issue in the repo (required)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.