llvm / llvm/llvm-project

ubsan (-fsanitize=undefined) should detect undefined subtraction of two pointers

Open
#206,775 1 comment 0 reactions 0 assignees View on GitHub
compiler-rt:ubsan
Dominant language
LLVM
Stars
40.5k
Forks
18.7k
PR merge metrics
PR metrics pending

Description

Consider:

```c
typedef struct { int x, y; } A;
long f(int* p) { return (A*)(p+1) - (A*)p; }
```

This has unconditional undefined behavior: the offset between the pointers isn't a multiple of the size of the struct. (This is a part of the general rule that the operands to pointer subtraction must be elements of an array.) The generated IR makes assumptions about this (using "sdiv exact"), but ubsan currently doesn't detect it.

We're adding a flag -fstable-pointer-subtraction that makes this defined (#196392); this is asking for the corresponding check.

Contributor guide

Open the contributing guide

Research direction

Start by tracing UBSan's pointer-subtraction instrumentation and the handling of -fstable-pointer-subtraction referenced in issue #196392. Reproduce the C example and verify that UBSan reports the invalid subtraction without the flag, while the stable-pointer-subtraction mode remains defined; add or update coverage for both cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
c
Domain
compilers
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.