google-deepmind / google-deepmind/mujoco_playground
Return substeps for reward accumulation?
Open
- Dominant language
- Python
- Stars
- 2.2k
- Forks
- 359
- Avg merge
- 7d 3h
- Merged PRs (30d)
- 1
Description
Currently mjx_env.step(state, action, n_substeps) only returns the final state of substepping. However, this makes it so that rewards can only be obtained for that state instead of being accumulated/averaged over the substeps as is normally done when frame skipping. I think this is a pretty simple feature in theory - just have an arg called `return_substeps=False` and return all the states over the substeps. I'm just wondering if theres a more memory efficient way to do so?
Contributor guide
Assessment
This issue has not been assessed yet.