huggingface / huggingface/optimum-nvidia

How to run whisper after #133

Open
#176 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1k
Forks
103
Avg merge
2m
Merged PRs (30d)
1

Description

I see that previously, whisper could be run as follows: [https://github.com/huggingface/optimum-nvidia/blob/whisper-inference/examples/automatic-speech-recognition/whisper.py](https://github.com/huggingface/optimum-nvidia/blob/whisper-inference/examples/automatic-speech-recognition/whisper.py)

But after #133 the code has been significantly refactored. Is there any documentation that shows how to properly run whisper with a tensorRT backend?

```python
from optimum.nvidia.pipelines import pipeline
asr = pipeline("automatic-speech-recognition", model="openai/whisper-base", device=device)
> NotImplementedError: Model type whisper is not currently supported
```

```python
from optimum.nvidia.models.whisper import WhisperForConditionalGeneration
model = WhisperForConditionalGeneration.from_pretrained("openai/whisper-base", torch_dtype=torch_dtype)
> AttributeError: type object 'WhisperForConditionalGeneration' has no attribute 'from_pretrained'
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.