OpenNMT / OpenNMT/CTranslate2

Reintroduce support for Compute Capability 5.0

Open
#1,765 9 comments 9 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
4.7k
Forks
536
Avg merge
12h 12m
Merged PRs (30d)
4

Description

Since a few months ago I was using CTranslate2 with CUDA 11 on a GTX 960M.

Recently I tried to update to CUDA 12, which is still supported for GTX 960M, and CTranslate2.

I expected the update to work, since documentation still reports compatibility with Compute Capability 3.5 (https://opennmt.net/CTranslate2/hardware_support.html) and I had no major issue updating PyTorch.
Unfortunately this was not the case, since I started receiving:

RuntimeError: parallel_for failed: cudaErrorNoKernelImageForDevice: no kernel image is available for execution on the device

Apparently this is a common issue, since I saw other related issues from users of GTX 9xx GPUs:
https://github.com/SYSTRAN/faster-whisper/issues/806
https://forums.developer.nvidia.com/t/runtimeerror-parallel-for-failed-cudaerrornokernelimagefordevice-no-kernel-image-is-available-for-execution-on-the-device/291404
https://github.com/m-bain/whisperX/issues/794

I tried to compile the code with Compute Cabability 5.0, but this was not enough, since some code was introduced that required Compute Capability 5.3 (which is not supported on my GPU), thus I disabled that code and recompiled.
After that I was able to run through quickstart, using cuda instead of cpu.
I was also able to run faster-whisper.

Is it possible to reintroduce support for Compute Capablity 5.0 in distributed wheels?

If yes I will be happy to provide a pull request.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the CUDA architecture settings used to build distributed wheels and identify the code introduced that requires Compute Capability 5.3. Rebuild with Compute Capability 5.0 enabled, verify the quickstart and faster-whisper paths on a GTX 960M, and confirm that the distributed wheels run without the no-kernel-image error.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
build-system, machine-learning
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.