OpenNMT / OpenNMT/CTranslate2

CPU Dispatch incompatible with higher CPU baselines

Open
#1,895 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
4.7k
Forks
536
Avg merge
12h 12m
Merged PRs (30d)
4

Description

Our CC toolchain supports (in at least 1 production deployment) AVX by default but not AVX2. Using CPU dispatch in this case seemingly requires patching the CPU dispatch mechanism. In this case we don't need to compile lower architectures at all, and then the CPU dispatch mechanism should ellide architectures that are lower than the baseline. Otherwise we get undefined symbols, because the way kernel.cc decides which header to include is based on the capabilities of the compiler. An alternative solution would be to pass defines ourselves indicating which dispatch we are building when building for cpu dispatch, instead of relying on __AVX__ and friends.

We will likely fix this issue internally, but would like to know if it is likely that a patch addressing this could be accepted.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in kernel.cc, where header selection currently follows compiler CPU capability defines such as AVX. Trace the CPU dispatch build for a compiler baseline with AVX but not AVX2 and identify where lower architectures are still selected. Done means the dispatch build no longer produces undefined symbols for that baseline; no specific test is named in the issue.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.