OpenNMT / OpenNMT/CTranslate2

Refactor GEMM backend registration and execution

Open
#424 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
C++
Stars
4.7k
Forks
536
Avg merge
12h 12m
Merged PRs (30d)
4

Description

The GEMM backend is selected at runtime depending on the requested compute type and CPU information. The dispatch to the correct implementation is done with a switch statement:

https://github.com/OpenNMT/CTranslate2/blob/3f6ac9cb22528c4b17b65783811f795ac6a85538/src/cpu/primitives.cc#L533-L612

This is not very clean and create large functions. The registration and execution of each backend should be more modular, for example by using class inheritance.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with src/cpu/primitives.cc around lines 533-612, where GEMM dispatch currently uses a switch statement. Trace how runtime compute type and CPU information select an implementation, then design a more modular registration and execution structure. Done means backend selection remains correct while the large switch-based functions are replaced with modular backend handling.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
backend, performance
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.