llvm / llvm/llvm-project

malloc incorrectly succeeds on big sizes with -fsanitizer=address

Open
#212,288 3 comments 0 reactions 0 assignees View on GitHub
compiler-rt:asan
Dominant language
LLVM
Stars
40.5k
Forks
18.7k
PR merge metrics
PR metrics pending

Description

I ran into this bug when testing a bleeding-edge GNU m4 with clang version 22.1.8 (Fedora 22.1.8-4.fc44) on x86-64. Take the following program `test-malloc.c`:

```c
#include
int
main (void)
{
return !!malloc ((size_t) -1 / 2 + 1);
}
```

and compile and run it in 32-bit mode on a large-enough x86-64 machine, as follows:

```
clang -m32 -fsanitize=address test-malloc.c
./a.out
echo $?
```

The last line should output 0, but it outputs 1 because the AddressSanitizer malloc mistakenly succeeds. This `malloc` should fail on this platform, because glibc `malloc` always fails when given sizes larger than `PTRDIFF_MAX`, for safety reasons to avoid later undefined behavior with pointer subtraction. This has been true since glibc 2.30 (2019). In this particular case, the AddressSanitizer `malloc` is more dangerous than the default `malloc`, which surely is not intended.

I do not observe the problem if I omit `-fsanitize=address`; exit status is 0.

I observe the same problem at runtime if I compile instead with gcc. I submitted [GCC bug 126436](https://gcc.gnu.org/bugzilla/show_bug.cgi?id=126436) on this topic, and GCC's Drea Pinski suggested that this was actually a bug upstream in LLVM's library so I am submitting this bug report here.

Other allocator functions like `realloc`, `calloc`, and `reallocarray` have the same bug. The AddressSanitizer versions mistakenly check only for `size_t` overflow; they should also check for `ptrdiff_t` overflow.

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the issue with test-malloc.c using clang -m32 -fsanitize=address, then inspect the AddressSanitizer implementations of malloc, realloc, calloc, and reallocarray. Done means allocations larger than PTRDIFF_MAX fail consistently on this platform, with coverage for the reported case and the related allocator functions.

Written by the indexing model from the issue text.

Assessment

Tech stack
c
Domain
compilers
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
67/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.