instadeepai / instadeepai/flashbax

Replay Across Experiments

Open
#69 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
283
Forks
22
PR merge metrics
No merged PRs in 30d

Description

### Is your feature request related to a problem? Please describe
My feature does not relate directly to a problem in flashbax.

### Describe the solution you'd like
I would like to propose my implementation of “[replay across experiments](https://arxiv.org/abs/2311.15951)”. The main idea is to be able to load an offline dataset(s) and use it to mix offline and online data when training online. One of the main motivations for this approach is that it allows to recycle old data from experiments. This is particularly useful when training RL policies on real hardware, where each experiment is costly.

### Describe alternatives you've considered
N/A

### Additional context
I have an implementation already, based on Brax's replay buffers. The main features are:
* Support for arbitrary pytrees for observations (including images)
* Loading old buffers from W&B checkpoints
My understanding is that I will need to port the core logic there to flashbax so that their interfaces match.

---
### Misc
- [x] Check for duplicate requests.

I am happy to contribute this, but might need very minor help because my experience so far has been with Brax-style RL.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.