astral-sh / astral-sh/python-build-standalone
Investigate x86_64 build versions (microarchitectures) and whether they can be simplified
- Dominant language
- Python
- Stars
- 4.4k
- Forks
- 314
- Avg merge
- 1d 9h
- Merged PRs (30d)
- 27
Description
We're currently spending a lot of CI time/cost on building for x86_64, x86_64-v2, x86_64-v3, and x86_64-v4. uv isn't using these yet (astral-sh/uv#9788). I'm not sure who is but it certainly makes our list of release binaries much noisier.
I'd like to understand
* whether these are actually worth building (with benchmarks) - I would guess it's more important that packages like numpy are optimized for your CPU than that Python and its dependencies are.
* whether we can use runtime detection instead of build-time - the [GCC `target_clones` extension](https://gcc.gnu.org/onlinedocs/gcc/Common-Function-Attributes.html#index-target_005fclones-function-attribute) might be useful here, or we could do the dispatching manually. In either case this would be patches to the relevant upstream projects, unless they already support this.
Ideally we would do a single, compatible, fast-enough build - either by deciding that v0 is fast enough or by doing runtime dispatching.
Related questions to answer:
* Is the Python JIT detecting the runtime µarch and making full use of it? Should it?
* I think we ended up compiling out some of the SIMD stuff in HACL, can we turn it back on? (Or can we turn off HACL entirely, since we ship OpenSSL?)
* I know glibc is good at providing multiple optimized memcpy etc. implementations and detecting the best one for thje runtime µarch. Does musl do that too, and if so, is that working properly in our static musl builds?
Contributor guide
Assessment
This issue has not been assessed yet.