huggingface / huggingface/optimum-graphcore
Allow the encoder outputs to be computed on the CPU for generation
Open
enhancement
generation
- Dominant language
- Python
- Stars
- 87
- Forks
- 33
- PR merge metrics
- No merged PRs in 30d
Description
Currently, during generation, the encoder outputs can only be computed after having compiled the encoder separately.
It would be nice to be able to compute the encoder outputs directly from the CPU, and to run the decoding loop on the IPUs.
The `IPUConfig` attribute for that, and some other features have already been added, but the whole thing does not work for now.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.