Reintroduce support for Compute Capability 5.0
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 4.7k
- Forks
- 536
- Avg merge
- 12h 12m
- Merged PRs (30d)
- 4
Description
Since a few months ago I was using CTranslate2 with CUDA 11 on a GTX 960M.
Recently I tried to update to CUDA 12, which is still supported for GTX 960M, and CTranslate2.
I expected the update to work, since documentation still reports compatibility with Compute Capability 3.5 (https://opennmt.net/CTranslate2/hardware_support.html) and I had no major issue updating PyTorch.
Unfortunately this was not the case, since I started receiving:
RuntimeError: parallel_for failed: cudaErrorNoKernelImageForDevice: no kernel image is available for execution on the device
Apparently this is a common issue, since I saw other related issues from users of GTX 9xx GPUs:
https://github.com/SYSTRAN/faster-whisper/issues/806
https://forums.developer.nvidia.com/t/runtimeerror-parallel-for-failed-cudaerrornokernelimagefordevice-no-kernel-image-is-available-for-execution-on-the-device/291404
https://github.com/m-bain/whisperX/issues/794
I tried to compile the code with Compute Cabability 5.0, but this was not enough, since some code was introduced that required Compute Capability 5.3 (which is not supported on my GPU), thus I disabled that code and recompiled.
After that I was able to run through quickstart, using cuda instead of cpu.
I was also able to run faster-whisper.
Is it possible to reintroduce support for Compute Capablity 5.0 in distributed wheels?
If yes I will be happy to provide a pull request.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the CUDA architecture settings used to build distributed wheels and identify the code introduced that requires Compute Capability 5.3. Rebuild with Compute Capability 5.0 enabled, verify the quickstart and faster-whisper paths on a GTX 960M, and confirm that the distributed wheels run without the no-kernel-image error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- build-system, machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100