google-deepmind / google-deepmind/reverb
Hang in end_episode
- Dominant language
- C++
- Stars
- 793
- Forks
- 113
- PR merge metrics
- No merged PRs in 30d
Description
Friends,
I'm using dm-reverb 0.14.0, and I'm getting a hang during end_episode() in my SAC implementation. This implementation is, as far as I can tell, copied verbatim from the one in the SAC minotaur demo on the tf-agents website. The only difference is I'm using a different env of my own design.
As far as I can tell, the sequence of events is:
* In `ReverbAddTrajectoryObserver.__call__`, detect a boundary.
* reset(False)
* self._writer is not None, so we call end_episode()
That end_episode call never returns and no feedback is given in the logs as to why this might be.
For what it's worth, I do not have this problem when I run the SAC minitaur example from the tutorial. It seems like I can set up my revert client just fine there.
----
As I'm writing this, I found a workaround that appears to resolve the issue at least with respect to the problem I was having. In `__call__`, I adjusted the trajectory.is_boundary() condition at the end with:
```
else:
self._get_writer().flush(10)
```
After that, I at least no longer hang. I'm still having other problems but at least this one seems to have been resolved.
Contributor guide
Research direction
Start in ReverbAddTrajectoryObserver.__call__ and trace the boundary path through reset(False) to self._writer.end_episode(). Compare that path with the SAC minitaur demo and the reported self._get_writer().flush(10) workaround. Done means reproducing the hang with dm-reverb 0.14.0 and establishing why end_episode() fails to return.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100