OpenNMT / OpenNMT/CTranslate2

[Feature] CANN Backend support

Open
#1,609 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
4.7k
Forks
536
Avg merge
12h 12m
Merged PRs (30d)
4

Description

CANN (Compute Architecture of Neural Networks), developed by Huawei, is a heterogeneous computing architecture for AI scenarios.
It provides multi-layer programming interfaces to help users quickly build AI applications and services based on the Ascend platform.

CANN backend in CTranslate2, enables running AI models on the Ascend NPU extending the existing CPU & CUDA workflows.
One can find more on Ascend NPU and CANN library here.

Examples of projects that support CANN include ONNX Runtime & OpenCV.

Implementation as well as detailed description of tests, environment setup & benchmarks can be found in pull request https://github.com/OpenNMT/CTranslate2/pull/1606 .

Notes

  • In the context of the development of this feature we also submitted issue https://github.com/OpenNMT/CTranslate2/issues/1583 .
  • In case CANN Backend support has increased demand, a new Pull request/subproject will be introduced contributing the respective CI involving dedicated Ascend hardware.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with pull request 1606, which contains the CANN backend implementation and the associated test, environment setup, and benchmark details. Review issue 1583 and the Ascend CANN documentation linked here; the work is done when CTranslate2 can run AI models on Ascend NPUs with the described validation completed.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
ai, backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.