ubsan (-fsanitize=undefined) should detect undefined subtraction of two pointers
- Dominant language
- LLVM
- Stars
- 40.5k
- Forks
- 18.7k
- PR merge metrics
- PR metrics pending
Description
Consider:
```c
typedef struct { int x, y; } A;
long f(int* p) { return (A*)(p+1) - (A*)p; }
```
This has unconditional undefined behavior: the offset between the pointers isn't a multiple of the size of the struct. (This is a part of the general rule that the operands to pointer subtraction must be elements of an array.) The generated IR makes assumptions about this (using "sdiv exact"), but ubsan currently doesn't detect it.
We're adding a flag -fstable-pointer-subtraction that makes this defined (#196392); this is asking for the corresponding check.
Contributor guide
Research direction
Start by tracing UBSan's pointer-subtraction instrumentation and the handling of -fstable-pointer-subtraction referenced in issue #196392. Reproduce the C example and verify that UBSan reports the invalid subtraction without the flag, while the stable-pointer-subtraction mode remains defined; add or update coverage for both cases.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c
- Domain
- compilers
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100