google-deepmind / google-deepmind/reverb
Maintaining episode-level priority while sampling individual transitions from a replay buffer with MinHeap remover
- Dominant language
- C++
- Stars
- 793
- Forks
- 113
- PR merge metrics
- No merged PRs in 30d
Description
Hi,
I have a table containing episodic trajectories. For each episode, I assign a priority. I only want to keep N episodes with the highest priority. For that, I use a MinHeap selector as remover. However, I want to sample individual transitions from all these trajectories, not full episodes. I have tried adding another table that stores individual transitions and sampling from there, but since it keeps a separate remover, the items in both tables do not match when inserting new data. Any ideas how I could achieve this?
On a related note, is it possible to keep a separate priority for sampling (to achieve something like Prioritized Experience Replay) and for removing for the same item?
Any help would be greatly appreciated!
Contributor guide
Research direction
No source file, test, or entry point is named. Start by tracing how the table, MinHeap remover, and sampling behavior interact, then clarify whether the requested outcome is synchronized episode and transition tables, independent sampling and removal priorities, or both; done requires an agreed supported design and tests for the chosen behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- data, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100