CUDA failed with error no CUDA-capable device is detected

Open
#326 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
35/100
Issue type
Bug
Clarity
Needs clarification
Activity status
Stale
Tech stack
docker, python
Domain
ai, infrastructure

Research direction

Start with whisperx/main.py, whisperx/transcribe.py, and whisperx/asr.py, then inspect the reported faster-whisper CUDA initialization failure and the surrounding Docker runtime. Reproduce the failure after extended operation and identify why the container loses access to the CUDA device; done means transcription continues reliably without the no-CUDA-device error.

Written by the indexing model from the issue text.

Description

After some time (around 12h) running, the Scriber Docker loses the CUDA device.
I restart the container, everything works great, and after a while, it sticks.

/app/whisperx-env/WhisperX/.venv/lib/python3.10/site-packages/pyannote/audio/core/io.py:212: UserWarning: torchaudio._backend.list_audio_backends has been deprecated. This deprecation is part of a large refactoring effort to transition TorchAudio into a maintenance phase. The decoding and encoding capabilities of PyTorch for both audio and video are being consolidated into TorchCodec. Please see https://github.com/pytorch/audio/issues/3902 for more information. It will be removed from the 2.9 release. 
  torchaudio.list_audio_backends()
/app/whisperx-env/WhisperX/.venv/lib/python3.10/site-packages/speechbrain/utils/torch_audio_backend.py:57: UserWarning: torchaudio._backend.list_audio_backends has been deprecated. This deprecation is part of a large refactoring effort to transition TorchAudio into a maintenance phase. The decoding and encoding capabilities of PyTorch for both audio and video are being consolidated into TorchCodec. Please see https://github.com/pytorch/audio/issues/3902 for more information. It will be removed from the 2.9 release. 
  available_backends = torchaudio.list_audio_backends()
/app/whisperx-env/WhisperX/.venv/lib/python3.10/site-packages/torch/cuda/__init__.py:829: UserWarning: Can't initialize NVML
  warnings.warn("Can't initialize NVML")
Traceback (most recent call last):
  File "/home/appuser/.local/share/uv/python/cpython-3.10.19-linux-x86_64-gnu/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/appuser/.local/share/uv/python/cpython-3.10.19-linux-x86_64-gnu/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/app/whisperx-env/WhisperX/whisperx/__main__.py", line 102, in <module>
    cli()
  File "/app/whisperx-env/WhisperX/whisperx/__main__.py", line 98, in cli
    transcribe_task(args, parser)
  File "/app/whisperx-env/WhisperX/whisperx/transcribe.py", line 127, in transcribe_task
    model = load_model(
  File "/app/whisperx-env/WhisperX/whisperx/asr.py", line 340, in load_model
    model = model or WhisperModel(whisper_arch,
  File "/app/whisperx-env/WhisperX/.venv/lib/python3.10/site-packages/faster_whisper/transcribe.py", line 663, in __init__
    self.model = ctranslate2.models.Whisper(
RuntimeError: CUDA failed with error no CUDA-capable device is detected
Dominant language
Go
Stars
3.1k
Forks
264
PR merge metrics
No merged PRs in 30d

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from rishikanthc/Scriberr

All issues in rishikanthc/Scriberr

Similar issues

More Go issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.