microsoft / microsoft/DirectXShaderCompiler

Warnings about float to int conversions are wrong

Open
#5,961 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug diagnostic tech-debt
Dominant language
C++
Stars
3.7k
Forks
900
Avg merge
2d 11h
Merged PRs (30d)
44

Description

See this comment on my PR that has now landed. As shown there, for certain out-of-range float to integral conversions, the compiler warns that the literal value it will use is incorrect. Here's a DXC Compiler Explorer example showing these specific warnings, some of which are clearly incorrect. Here's the output:

<source>:13:19: warning: implicit conversion from 'literal float' to 'int' changes value from 2147483648 to 2147483647 [-Wliteral-conversion]
    store(to_int(-2147483648.0)); // MaxNegative int: -2147483648
          ~~~~~~  ^~~~~~~~~~~~
<source>:14:19: warning: implicit conversion from 'literal float' to 'int' changes value from 1.797693134862316E+308 to 2147483647 [-Wliteral-conversion]
    store(to_int(-1.7976931348623158e+308)); // MaxNegative double: -2147483648 (clamp int)
          ~~~~~~  ^~~~~~~~~~~~~~~~~~~~~~~
<source>:15:18: warning: implicit conversion from 'literal float' to 'int' changes value from 1.797693134862316E+308 to 2147483647 [-Wliteral-conversion]
    store(to_int(1.7976931348623158e+308)); // MaxPositive double: 2147483647 (clamp int)
          ~~~~~~ ^~~~~~~~~~~~~~~~~~~~~~~
<source>:16:20: warning: implicit conversion from 'literal float' to 'uint' changes value from 1.797693134862316E+308 to 4294967295 [-Wliteral-conversion]
    store(to_uint(-1.7976931348623158e+308)); // MaxNegative double: 0 (clamp uint)
          ~~~~~~~  ^~~~~~~~~~~~~~~~~~~~~~~
<source>:17:19: warning: implicit conversion from 'literal float' to 'uint' changes value from 1.797693134862316E+308 to 4294967295 [-Wliteral-conversion]
    store(to_uint(1.7976931348623158e+308)); // MaxPositive double: 4294967295 (clamp uint)
          ~~~~~~~ ^~~~~~~~~~~~~~~~~~~~~~~

The comments on each line shows what the actual value will be. We can see that the warning doesn't always match the value. For example, the first one warns that the value will be converted to 2147483647, but in fact, it will be -2147483648.

Here's a C++ Compiler Explorer example showing what Clang 3.8 vs 3.9 does to similar C++ code. Pay attention to the warnings emitted by the compiler, rather than the actual output (the output is mostly garbage, which isn't unexpected since these conversions are UB). We can see that the 3.8 version is what DXC is emitting.

So we should update this warning code with a later version. In fact, in latest Clang, it doesn't bother saying what the literal value will be changed to since it's UB. It says, instead:

<source>:28:12: warning: implicit conversion of out of range value from 'double' to 'int' is undefined [-Wliteral-conversion]

But for DXC, if we want to make these well-defined, as my patch did, then we could go with the 3.9 changes instead.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start from the code that emits -Wliteral-conversion warnings and compare its behavior with the linked Clang 3.8 and 3.9 examples. Review the DXC Compiler Explorer cases and the comments showing actual converted values. Done means the warning no longer reports an incorrect result for out-of-range float-to-integer conversions and follows the chosen defined-conversion or undefined-conversion behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
compilers
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.