Add Speculative Sampling
Open
feature request
research
- Dominant language
- Python
- Stars
- 4.8k
- Forks
- 487
- PR merge metrics
- No merged PRs in 30d
Description
### 🚀 The feature, motivation, and pitch
https://arxiv.org/abs/2302.01318
This is helpful for generating rollouts 2x faster.
It also presents an interesting way to combine models of different scales.
This can also be seen as "reward model rejection sampling guided generation" if you view the scorer model as a RM.
Contributor guide
Assessment
This issue has not been assessed yet.