Azure / Azure/azureml-examples
Whisper STT: Expected a torch.device with a specified index or an integer, but got:cuda
- Dominant language
- Jupyter Notebook
- Stars
- 2k
- Forks
- 1.7k
- Avg merge
- 18h 18m
- Merged PRs (30d)
- 2
Description
### Operating System
Linux
### Version Information
Python Version: 3.9.6
azure-ai-ml package version: 1.8.0
### Steps to reproduce
1. `sdk/python/foundation-models/system/inference/automatic-speech-recognition/asr-online-endpoint.ipynb`
2. Change the machine type from `Standard_DS5_v2` to `Standard_NC8as_T4_v3`
3. Deploy the endpoint
4. Run inference with `response = workspace_ml_client.online_endpoints.invoke( ... )`
### Expected behavior
Audio is transcribed
### Actual behavior
```
Exception has occurred: Exception
Expected a torch.device with a specified index or an integer, but got:cuda
StopIteration: 0
During handling of the above exception, another exception occurred:
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
During handling of the above exception, another exception occurred:
File "/Users/danielve/azureml-examples/inference/automatic-speech-recognition/asr-online-endpoint-inference.py", line 28, in
response = workspace_ml_client.online_endpoints.invoke(
Exception: Expected a torch.device with a specified index or an integer, but got:cuda
```
### Addition information
This issue seems to arise when using GPUs. I have enough quota, so that shouldn't be the problem.
Contributor guide
Research direction
Start with sdk/python/foundation-models/system/inference/automatic-speech-recognition/asr-online-endpoint.ipynb and inference/automatic-speech-recognition/asr-online-endpoint-inference.py, reproducing the deployment on Standard_NC8as_T4_v3. Trace the Whisper endpoint's GPU device handling and the invoke response error. Done means the endpoint deploys and transcribes audio successfully on the GPU machine type.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure, python
- Domain
- cloud, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100