instadeepai / instadeepai/flashbax
Replay Across Experiments
- Dominant language
- Python
- Stars
- 283
- Forks
- 22
- PR merge metrics
- No merged PRs in 30d
Description
### Is your feature request related to a problem? Please describe
My feature does not relate directly to a problem in flashbax.
### Describe the solution you'd like
I would like to propose my implementation of “[replay across experiments](https://arxiv.org/abs/2311.15951)”. The main idea is to be able to load an offline dataset(s) and use it to mix offline and online data when training online. One of the main motivations for this approach is that it allows to recycle old data from experiments. This is particularly useful when training RL policies on real hardware, where each experiment is costly.
### Describe alternatives you've considered
N/A
### Additional context
I have an implementation already, based on Brax's replay buffers. The main features are:
* Support for arbitrary pytrees for observations (including images)
* Loading old buffers from W&B checkpoints
My understanding is that I will need to port the core logic there to flashbax so that their interfaces match.
---
### Misc
- [x] Check for duplicate requests.
I am happy to contribute this, but might need very minor help because my experience so far has been with Brax-style RL.
Contributor guide
Assessment
This issue has not been assessed yet.