NVIDIA / NVIDIA/TensorRT

instanceNormalizationPlugin: use public __half2float instead of private __internal_half2float

Open Beginner friendly
#4,810 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Module:Plugins
Dominant language
C++
Stars
13.4k
Forks
2.4k
Avg merge
5d 3h
Merged PRs (30d)
2

Description

Both InstanceNorm plugin sources convert half scale/bias weights to float with __internal_half2float, which is a private static inline helper inside cuda_fp16.hpp, not part of the public CUDA API. It compiles under nvcc, where that header brings the helper into scope, but fails under other CUDA-compatible compilers (clang-CUDA, and AMD-targeting CUDA toolchains), and it would also stand in the way of a HIP/ROCm port.

Call sites (current main)

  • plugin/instanceNormalizationPlugin/instanceNormalizationPlugin.cu, in the InstanceNormalizationV3Plugin(float, Weights const&, Weights const&, int32_t, float) constructor's copyWeights lambda (the kHALF branch).
  • plugin/instanceNormalizationPlugin/instanceNormalizationPluginLegacy.cu, in the InstanceNormalizationPlugin(float, Weights const&, Weights const&, int32_t, float) constructor's copyWeights lambda (the kHALF branch).

Both read:

auto const value = static_cast<unsigned short const*>(input.values);
output.push_back(__internal_half2float(value[c]));

Under a non-nvcc CUDA compiler this errors with:

error: use of undeclared identifier '__internal_half2float'

Proposed fix (public API, one line per site)

Wrap the raw bits in a __half_raw and call the public __half2float:

auto const value = static_cast<unsigned short const*>(input.values);
__half_raw raw;
raw.x = value[c];
output.push_back(__half2float(raw));

Verified

Minimal reproducer compiled with NVIDIA nvcc (CUDA 13.1, sm_75) and with a clang-based CUDA compiler:

code nvcc non-nvcc CUDA
original (__internal_half2float) compiles fails (undeclared identifier)
fix (__half_raw + __half2float) compiles compiles

The public __half2float runs the same conversion the private helper does, so the numeric behaviour is unchanged. The change just drops a dependency on an undocumented nvcc internal and makes the plugin portable to other CUDA compilers, at no cost under nvcc.

Filing this first per CONTRIBUTING (issue before PR). I'm happy to send a PR touching both files, DCO signed off, once this is acknowledged.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the kHALF branches in plugin/instanceNormalizationPlugin/instanceNormalizationPlugin.cu and plugin/instanceNormalizationPlugin/instanceNormalizationPluginLegacy.cu, specifically their constructor copyWeights lambdas. Confirm both call sites build with nvcc and a non-nvcc CUDA compiler, and verify that half scale and bias conversion retains its existing numeric behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
ai-infra-agents, compilers
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
88/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.