google-deepmind / google-deepmind/reverb

Hang in end_episode

Open
#144 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
793
Forks
113
PR merge metrics
No merged PRs in 30d

Description

Friends,

I'm using dm-reverb 0.14.0, and I'm getting a hang during end_episode() in my SAC implementation. This implementation is, as far as I can tell, copied verbatim from the one in the SAC minotaur demo on the tf-agents website. The only difference is I'm using a different env of my own design.

As far as I can tell, the sequence of events is:

* In `ReverbAddTrajectoryObserver.__call__`, detect a boundary.
* reset(False)
* self._writer is not None, so we call end_episode()

That end_episode call never returns and no feedback is given in the logs as to why this might be.

For what it's worth, I do not have this problem when I run the SAC minitaur example from the tutorial. It seems like I can set up my revert client just fine there.

----

As I'm writing this, I found a workaround that appears to resolve the issue at least with respect to the problem I was having. In `__call__`, I adjusted the trajectory.is_boundary() condition at the end with:

```
else:
self._get_writer().flush(10)
```

After that, I at least no longer hang. I'm still having other problems but at least this one seems to have been resolved.

Contributor guide

Open the contributing guide

Research direction

Start in ReverbAddTrajectoryObserver.__call__ and trace the boundary path through reset(False) to self._writer.end_episode(). Compare that path with the SAC minitaur demo and the reported self._get_writer().flush(10) workaround. Done means reproducing the hang with dm-reverb 0.14.0 and establishing why end_episode() fails to return.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.