Enable `-fstrict-overflow`
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 77.2k
- Forks
- 36k
- PR merge metrics
- PR metrics pending
Description
At the moment we compile releases with -fwrapv which makes the code a bit safer, but disables certain optimizations. From the GCC docs:
This option instructs the compiler to assume that signed arithmetic overflow of addition, subtraction and multiplication wraps around using twos-complement representation. This flag enables some optimizations and disables others.
My experiments with running sanitisers seem to suggest that we are nearly already ready for -fno-wrapv (or -fstrict-overflow in general). Doing so could lead to quite a few speedups, but we would need to be more careful with the code we write.
It might be worthwhile to get a few benchmarks.
(To be extra precise, we give -fwrapv for clang and gcc for any build that doesn't get --with-pydebug.)
Pitch
My plan right now is to adapt the build system so that only the modules that need it are build with -fwrapv, and the rest can be build with -fstrict-overflow.
We already have config machinery that can add specific CFLAGS for specific modules only.
Perhaps the whole thing can be gated behind a configure flag, like --with-strict-overflow.
If everything goes well, and this improves performance we can consider adding this functionality to one of the standard optimization options.
We can also work on making more modules -fstrict-overflow safe.
Previous discussion
@markshannon @ericsnowcurrently
Brought up on https://github.com/faster-cpython/ideas/issues/458 and inspired by https://github.com/python/cpython/issues/96678
Some previous issues around -fwrapv:
- https://bugs.python.org/issue11149
- https://bugs.python.org/issue1621
- https://bugs.python.org/issue1608
I'm sure there are more.
Progress so far
As far as is currently known, the three remaining modules that rely on defined integer overflow are fixed by:
_struct: https://github.com/python/cpython/pull/96739audioop: https://github.com/python/cpython/pull/96923_ctypes: https://github.com/python/cpython/pull/96925
Linked PRs
- gh-96823
- gh-139595
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the build-system configuration machinery that adds module-specific CFLAGS, then review the linked work in gh-96823 and gh-139595. The proposed change is a configure-gated strict-overflow build for eligible modules, with remaining overflow-dependent modules handled safely and performance checked with benchmarks.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c
- Domain
- build-system, compilers, performance
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100