possibilities for Asynchronous (RL online rollout) evaluation during training

Nobody has claimed this yet.

## 🚀 Feature

I am wondering if it is possible to include the asynchronous evaluation during the training process

### Motivation

For RL projects (or imitation training + online rollout evaluation), evaluation is really a bottleneck during the training process even the environments are vectorized, especially if we want to evaluate lots of long-horizon episodes. Now the training only can continue after eval is done, but it seems not necessary as the evaluation could be done with weights at that timestep, and does not matter the future training.

### Pitch

Option for doing asynchronous evaluation during training

### Alternatives

using separate scripts to do this

### Additional context

No

______________________________________________________________________

#### If you enjoy Lightning, check out our other projects! ⚡

- [**Metrics**](https://github.com/Lightning-AI/metrics): Machine learning metrics for distributed, scalable PyTorch applications.

- [**Lite**](https://pytorch-lightning.readthedocs.io/en/latest/starter/lightning_lite.html): enables pure PyTorch users to scale their existing code on any kind of device while retaining full control over their own loops and optimization logic.

- [**Flash**](https://github.com/Lightning-AI/lightning-flash): The fastest way to get a Lightning baseline! A collection of tasks for fast prototyping, baselining, fine-tuning, and solving problems with deep learning.

- [**Bolts**](https://github.com/Lightning-AI/lightning-bolts): Pretrained SOTA Deep Learning models, callbacks, and more for research and production with PyTorch Lightning and PyTorch.

- [**Lightning Transformers**](https://github.com/Lightning-AI/lightning-transformers): Flexible interface for high-performance research using SOTA Transformers leveraging PyTorch Lightning, Transformers, and Hydra.

cc @borda @awaelchli @rohitgr7

No files, tests, or entry points are named in the issue. Start by tracing the existing training and evaluation flow, then clarify how evaluation should run asynchronously, how weights are selected, and how completion is reported; done requires an agreed design and implementation for an asynchronous evaluation option.

Written by the indexing model from the issue text.