scikit-learn / scikit-learn/scikit-learn-release
Use lock files and relative dependency cooldowns
Nobody has claimed this yet.
- Dominant language
- Shell
- Stars
- 2
- Forks
- 4
- Avg merge
- 7h 51m
- Merged PRs (30d)
- 5
Description
I think the use of requirements files with X.Y.Z pinned version constraints is good but not enough to ensure full reproducibility of the builds (there can be .post0 markers after the .Z component for instance). I think we should use an explicit lock file to manage:
- the cibuildwheel dependency itself (this will transitively lock the version of cibuildwheel's own dependencies as per the "pinned" policy documented in https://cibuildwheel.pypa.io/en/stable/options/#dependency-versions)
- the sdist build dependencies;
- the test dependencies.
The lockfile(s) should be automatically refreshed, e.g. on a monthly basis with a relative dependency cooldown policy of a week to reduce the exposure of the release process to ongoing OSS Supply Chain attacks.
The lockfile(s) in the release repo could either be managed via uv or pixi. I don't think it matters much because we don't need conda-forge for the build and testing process in this repo. All the Python-level builds dependencies are fetched from pypi.org and the rest come from the system image (e.g. manylinux). EDIT: actually we do fetch a fixed binary for libomp on macOS, so maybe we could use pixi and a lock file to do that more automatically.
When the lock files are updated, we should use relative dependency cooldowns such as uv's exclude-newer config or pixi's exlude-newer config).
Ideally, the build dependencies should also be locked explicitly in the repo as for the other dependencies. According to cibuildwheel's doc, the pip wheel command is used under the hood to setup the build env automatically. By default, it will install dependencies from the pyproject.toml file. However, it should be possible to pass an explicit pylock.toml export of a proper uv managed lockfile to pip wheel -r. Alternatively, we configure cibuildwheel's before-build hook to run a script to explicitly prepare the build environment manually using uv or pixi commands and disable build isolation and run pip wheel --no-build-isolation --no-deps.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading pyproject.toml and tools/configure_macos_env.sh, then review how cibuildwheel currently obtains build dependencies. Compare uv and pixi lockfile approaches, including relative dependency cooldowns and macOS libomp handling; done means build, sdist, and test dependencies are explicitly locked and refreshes are automated.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, shell
- Domain
- build-system, release
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100