python / python/cpython

Enable `-fstrict-overflow`

Open
#96,821 14 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

build performance type-feature
Dominant language
Python
Stars
77.2k
Forks
36k
PR merge metrics
PR metrics pending

Description

At the moment we compile releases with -fwrapv which makes the code a bit safer, but disables certain optimizations. From the GCC docs:

This option instructs the compiler to assume that signed arithmetic overflow of addition, subtraction and multiplication wraps around using twos-complement representation. This flag enables some optimizations and disables others.

My experiments with running sanitisers seem to suggest that we are nearly already ready for -fno-wrapv (or -fstrict-overflow in general). Doing so could lead to quite a few speedups, but we would need to be more careful with the code we write.

It might be worthwhile to get a few benchmarks.

(To be extra precise, we give -fwrapv for clang and gcc for any build that doesn't get --with-pydebug.)

Pitch

My plan right now is to adapt the build system so that only the modules that need it are build with -fwrapv, and the rest can be build with -fstrict-overflow.

We already have config machinery that can add specific CFLAGS for specific modules only.

Perhaps the whole thing can be gated behind a configure flag, like --with-strict-overflow.

If everything goes well, and this improves performance we can consider adding this functionality to one of the standard optimization options.

We can also work on making more modules -fstrict-overflow safe.

Previous discussion

@markshannon @ericsnowcurrently

Brought up on https://github.com/faster-cpython/ideas/issues/458 and inspired by https://github.com/python/cpython/issues/96678

Some previous issues around -fwrapv:

I'm sure there are more.

Progress so far

As far as is currently known, the three remaining modules that rely on defined integer overflow are fixed by:

Linked PRs
  • gh-96823
  • gh-139595

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the build-system configuration machinery that adds module-specific CFLAGS, then review the linked work in gh-96823 and gh-139595. The proposed change is a configure-gated strict-overflow build for eligible modules, with remaining overflow-dependent modules handled safely and performance checked with benchmarks.

Written by the indexing model from the issue text.

Assessment

Tech stack
c
Domain
build-system, compilers, performance
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.