4.8.0 regression: WhisperModel.align() fails with parallel_for failed: cudaErrorInvalidDevice on Tesla T4 (works on 4.7.0)
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 4.7k
- Forks
- 536
- Avg merge
- 12h 12m
- Merged PRs (30d)
- 4
Description
Description
After upgrading from CTranslate2 4.7.0 to 4.8.0, Whisper word-timestamp alignment (WhisperModel.align(), called by faster-whisper's add_word_timestamps → find_alignment) crashes on a Tesla T4 (sm_75) with:
RuntimeError: parallel_for failed: cudaErrorInvalidDevice: invalid device ordinal
Plain transcription (encode/generate) is unaffected — only the alignment path fails. Pinning back to 4.7.0 resolves it completely, with no other change. I suspect the Thrust 1.12.0 → CCCL 2.7.0 swap in 4.8.0 (the only GPU-infra change in that release per the changelog), since parallel_for is the failing Thrust primitive and align() is the op that uses it.
Environment
- CTranslate2:
4.8.0(broken) vs4.7.0(works) — single-variable change - faster-whisper:
1.2.0(constraintctranslate2>=4.0,<5, so it floats to4.8.0) - GPU: NVIDIA Tesla T4 (compute capability 7.5), single GPU, no MIG, default
device_index=0 - Base image:
nvidia/cuda:12.4.1-cudnn-runtime-ubuntu22.04, Python 3.11 - Model:
large-v3,compute_type="float16",device="cuda"
Reproduction
from faster_whisper import WhisperModel
model = WhisperModel("large-v3", device="cuda", compute_type="float16")
# ~30s speech clip; word_timestamps triggers WhisperModel.align()
segments, _ = model.transcribe("audio.wav", word_timestamps=True, vad_filter=False)
for s in segments: # crash occurs while consuming the generator
print(s.words)
Traceback
File ".../faster_whisper/transcribe.py", line 1253, in generate_segments
self.add_word_timestamps(...)
File ".../faster_whisper/transcribe.py", line 1564, in add_word_timestamps
alignments = self.find_alignment(...)
File ".../faster_whisper/transcribe.py", line 1683, in find_alignment
results = self.model.align(...)
RuntimeError: parallel_for failed: cudaErrorInvalidDevice: invalid device ordinal
Workaround
Pin ctranslate2==4.7.0.
Notes
Reproduced deterministically across container restarts and model sizes (medium, large-v3). Likely related to but distinct from #2063 (also a 4.8.0 regression, but CPU-thread oversubscription on macOS).
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at WhisperModel.align(), reached through faster-whisper's add_word_timestamps and find_alignment, and compare the 4.7.0 and 4.8.0 GPU-infrastructure changes noted in the changelog. Reproduce on a Tesla T4 with the provided faster-whisper configuration and check the Thrust 1.12.0 to CCCL 2.7.0 change. Done means word-timestamp alignment completes on 4.8.0 without the invalid-device error while plain transcription remains unaffected.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100