Undefined behaviour handling an enum enumerator that is the max value of its type
- Dominant language
- LLVM
- Stars
- 40.5k
- Forks
- 18.7k
- PR merge metrics
- PR metrics pending
Description
We first spotted this when running llvm-test-suite with a ubsan enabled flang.
```
FAILED: Fortran/gfortran/regression/CMakeFiles/gfortran-regression-execute-regression__enum_10_f90.dir/enum_10.f90.o Fortran/gfortran/regression/gfortran-regression-execute-regression__enum_10_f90.wd/enum_10.mod
/home/buildbot/workspace/arm-bbot-flang-sanitized/build/bin/flang -I/home/buildbot/workspace/arm-bbot-flang-sanitized/llvm-test-suite/Fortran/gfortran/regression -I/home/buildbot/workspace/arm-bbot-flang-sanitized/build/include/flang -I/home/buildbot/workspace/arm-bbot-flang-sanitized/build-llvm-test-suite/Fortran/gfortran/regression/gfortran-regression-execute-regression__enum_10_f90.wd -O3 -JFortran/gfortran/regression/gfortran-regression-execute-regression__enum_10_f90.wd -ffixed-line-length-72 -c Fortran/gfortran/regression/CMakeFiles/gfortran-regression-execute-regression__enum_10_f90.dir/enum_10.f90-pp.f90 -o Fortran/gfortran/regression/CMakeFiles/gfortran-regression-execute-regression__enum_10_f90.dir/enum_10.f90.o
/home/buildbot/workspace/arm-bbot-flang-sanitized/llvm-project/flang/lib/Semantics/resolve-names.cpp:6445:31: runtime error: signed integer overflow: 2147483647 + 1 cannot be represented in type 'int'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /home/buildbot/workspace/arm-bbot-flang-sanitized/llvm-project/flang/lib/Semantics/resolve-names.cpp:6445:31
```
This test cases uses a flag flang does not support, `-fshort-enums`. However, we are normally able to compile and run it. When ubsan is enabled, it fails as above.
A reduced version of the scenario is:
`$ cat /tmp/test.f90`
```fortran
program repro
enum, bind(c)
enumerator :: x = huge(1_4)
end enum
end program
```
The enumerator's value is the largest value of the kind 4, which is a 32-bit signed integer on this system (AArch64).
```console
$ ./bin/flang /tmp/test.f90 -o /dev/null -c
/home/davspi01/llvm-project/flang/lib/Semantics/resolve-names.cpp:6445:31: runtime error: signed integer overflow: 2147483647 + 1 cannot be represented in type 'int'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /home/davspi01/llvm-project/flang/lib/Semantics/resolve-names.cpp:6445:31
```
https://github.com/llvm/llvm-project/blob/3892d6e6a1c538aab3ab3646bc162ae37bc0e64a/flang/lib/Semantics/resolve-names.cpp#L6445
Perhaps this is tracking what the value of the next enumerator would be, if the source does not assign it a value.
Contributor guide
Research direction
Start in flang/lib/Semantics/resolve-names.cpp around line 6445 and reproduce the issue with the reduced /tmp/test.f90 program using the shown flang compile command. Trace how the next enumerator value is tracked; done means the maximum kind-4 enumerator compiles without the UBSan signed-overflow report.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- compilers
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 70/100