This repo is trying to do too much

The future is adopting [skyrl-tx](https://github.com/NovaSky-AI/SkyRL/tree/main/skyrl-tx) and using [verifiers](https://verifiers.readthedocs.io/en/latest/) for handling all the environments code.

This way anytime someone writes code that uses tinker it is able to run on TPUs.

This is all the API someone using your library needs (taken from https://tinker-docs.thinkingmachines.ai/):
```
forward_backward - Perform a forward pass and a backward pass, accumulating the gradient.
optim_step - Update weights based on the accumulated gradient.
sample - Generate tokens for interaction, evaluation, or RL actions.
save_state - Save training progress for resumption.
```

I will point you towards [this integration](https://github.com/thinking-machines-lab/tinker-cookbook/tree/main/tinker_cookbook/recipes/verifiers_rl) that allows one to train RL agents on any env from the Environments Hub (hundreds) using tinker.

skyrl-tx acts as a backend for tinker and allows one to run any tinker client on TPUs.
Right now it looks experimental, but it has a lot of potential.