OpenNMT / OpenNMT/CTranslate2

4.8.0 regression: WhisperModel.align() fails with parallel_for failed: cudaErrorInvalidDevice on Tesla T4 (works on 4.7.0)

Open
#2,064 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
4.7k
Forks
536
Avg merge
12h 12m
Merged PRs (30d)
4

Description

Description

After upgrading from CTranslate2 4.7.0 to 4.8.0, Whisper word-timestamp alignment (WhisperModel.align(), called by faster-whisper's add_word_timestampsfind_alignment) crashes on a Tesla T4 (sm_75) with:

RuntimeError: parallel_for failed: cudaErrorInvalidDevice: invalid device ordinal

Plain transcription (encode/generate) is unaffected — only the alignment path fails. Pinning back to 4.7.0 resolves it completely, with no other change. I suspect the Thrust 1.12.0 → CCCL 2.7.0 swap in 4.8.0 (the only GPU-infra change in that release per the changelog), since parallel_for is the failing Thrust primitive and align() is the op that uses it.

Environment
  • CTranslate2: 4.8.0 (broken) vs 4.7.0 (works) — single-variable change
  • faster-whisper: 1.2.0 (constraint ctranslate2>=4.0,<5, so it floats to 4.8.0)
  • GPU: NVIDIA Tesla T4 (compute capability 7.5), single GPU, no MIG, default device_index=0
  • Base image: nvidia/cuda:12.4.1-cudnn-runtime-ubuntu22.04, Python 3.11
  • Model: large-v3, compute_type="float16", device="cuda"
Reproduction
from faster_whisper import WhisperModel
model = WhisperModel("large-v3", device="cuda", compute_type="float16")
# ~30s speech clip; word_timestamps triggers WhisperModel.align()
segments, _ = model.transcribe("audio.wav", word_timestamps=True, vad_filter=False)
for s in segments:        # crash occurs while consuming the generator
    print(s.words)
Traceback
File ".../faster_whisper/transcribe.py", line 1253, in generate_segments
    self.add_word_timestamps(...)
File ".../faster_whisper/transcribe.py", line 1564, in add_word_timestamps
    alignments = self.find_alignment(...)
File ".../faster_whisper/transcribe.py", line 1683, in find_alignment
    results = self.model.align(...)
RuntimeError: parallel_for failed: cudaErrorInvalidDevice: invalid device ordinal
Workaround

Pin ctranslate2==4.7.0.

Notes

Reproduced deterministically across container restarts and model sizes (medium, large-v3). Likely related to but distinct from #2063 (also a 4.8.0 regression, but CPU-thread oversubscription on macOS).

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at WhisperModel.align(), reached through faster-whisper's add_word_timestamps and find_alignment, and compare the 4.7.0 and 4.8.0 GPU-infrastructure changes noted in the changelog. Reproduce on a Tesla T4 with the provided faster-whisper configuration and check the Thrust 1.12.0 to CCCL 2.7.0 change. Done means word-timestamp alignment completes on 4.8.0 without the invalid-device error while plain transcription remains unaffected.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.