conda-forge / conda-forge/conda-forge.github.io

META: glibc baseline in conda-forge - moving to 2.28 / alma8

Open
#2,383 22 comments 5 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
170
Forks
320
Avg merge
2d 10h
Merged PRs (30d)
5

Description

Since we've separated the baselines between docker_images and the sysroot (and the docker_image [bump](https://github.com/conda-forge/conda-forge-pinning-feedstock/pull/6626) has now happened), this question will inevitably come up, so we might as well have an issue to reference and discuss.

# Note: this is likely still years away!

With that said, here's the background. The timeline of changes for our images and sysroot baseline is as follows

| when | sysroot | docker image | comment |
|-|-|-|-|
| until Nov. 2021 | 2.12 (cos6) | cos6 | matching distro version, until discussion [here](https://github.com/conda-forge/conda-forge.github.io/issues/1436) eventually lead to split |
| Dec 2021 | 2.12 (cos6) | cos7 | https://github.com/conda-forge/conda-forge-pinning-feedstock/pull/2241 |
| July 2024 | 2.17 (cos7) | cos7 | This needed a lot of preparation (mainly rolling out [`stdlib`](https://github.com/conda-forge/conda-forge.github.io/issues/2102));
finally done [here](https://github.com/conda-forge/conda-forge-pinning-feedstock/pull/6070) |
| Nov 2024 | 2.17 (cos7) | alma9 | Due to how long things got held up (for stdlib & [CDTs](https://github.com/conda-forge/cdt-builds/issues/66)), we actually
ended up leapfrogging [alma8](https://github.com/conda-forge/conda-forge.github.io/issues/1941) and went directly to [alma9](https://github.com/conda-forge/conda-forge.github.io/issues/2342);
see [discussion](https://github.com/conda-forge/conda-forge-pinning-feedstock/issues/6283) and [execution](https://github.com/conda-forge/conda-forge-pinning-feedstock/pull/6626) |
| July 2026 | 2.17 (cos7) | alma10 | https://github.com/conda-forge/conda-forge-pinning-feedstock/pull/8727 |
| ??? | 2.28 (alma8) | ? | The topic of this issue 🙃 |

The reason we are able to separate these two things is because we can distinguish between what we compile against (i.e. the sysroot that we package and pin to a specific version), from the actual glibc version that's present in the image. Some more explanation is in this [announcement](https://conda-forge.org/news/2024/11/22/new-images/).

Part of the discussion in #1436 and why it took so long to move on, is that conda-forge has some large users (e.g. academic institutions) that move glacially w.r.t. infrastructure updates, and we try to support those use-cases as long as we feasibly can (not least because some core members are involved there). That meant we [went](https://github.com/conda-forge/conda-forge.github.io/pull/1980) all the way to the very end-of-life (even with the paid extension that's not actually relevant for us!) of `cos6` before dropping it. This was not actually a causal relationship, but just a reflection of how much we've tried to stay on cos6 as long as feasible.

For the move from `cos7` to `alma8`, the situation will very likely repeat, in that we want to be as defensive as possible with upgrading the baseline, while enabling individual feedstocks to pull in newer baseline (using `c_stdlib_version`) as necessary. A good reference is https://mayeut.github.io/manylinux-timeline/, which provides an overview of the consumer glibc version across all downloads from PyPI[^1]. As of Nov. 2024, around 85% of PyPI downloaders on linux have `__glibc>=2.28`, and this number rises to 94% if we're not counting people on python versions that are already EOL.

[^1]: I'd really like to have an equivalent for conda-forge, but that's another story

Note however that we didn't drop glibc 2.12 until it was below 0.1% of downloads (at least via the proxy of the PyPI downloads). Perhaps that's a little extreme (and also owed due to the stdlib-work that took a while to put in place), but in any case, it will likely have to be a _very_ low percentage of users or there'd have to be very strong technical constraints that force our hand before we move on.

In short: enjoy the new alma-based infrastructure, move the glibc baseline for your feedstock if it cannot be avoided, but the global baseline will likely still take quite a while to move.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.