huggingface / huggingface/optimum-nvidia
How to run whisper after #133
- Dominant language
- Python
- Stars
- 1k
- Forks
- 103
- Avg merge
- 2m
- Merged PRs (30d)
- 1
Description
I see that previously, whisper could be run as follows: [https://github.com/huggingface/optimum-nvidia/blob/whisper-inference/examples/automatic-speech-recognition/whisper.py](https://github.com/huggingface/optimum-nvidia/blob/whisper-inference/examples/automatic-speech-recognition/whisper.py)
But after #133 the code has been significantly refactored. Is there any documentation that shows how to properly run whisper with a tensorRT backend?
```python
from optimum.nvidia.pipelines import pipeline
asr = pipeline("automatic-speech-recognition", model="openai/whisper-base", device=device)
> NotImplementedError: Model type whisper is not currently supported
```
```python
from optimum.nvidia.models.whisper import WhisperForConditionalGeneration
model = WhisperForConditionalGeneration.from_pretrained("openai/whisper-base", torch_dtype=torch_dtype)
> AttributeError: type object 'WhisperForConditionalGeneration' has no attribute 'from_pretrained'
```
Contributor guide
Assessment
This issue has not been assessed yet.