scikit-learn / scikit-learn/scikit-learn-release

Use lock files and relative dependency cooldowns

Open
#1 8 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Shell
Stars
2
Forks
4
Avg merge
7h 51m
Merged PRs (30d)
5

Description

I think the use of requirements files with X.Y.Z pinned version constraints is good but not enough to ensure full reproducibility of the builds (there can be .post0 markers after the .Z component for instance). I think we should use an explicit lock file to manage:

The lockfile(s) should be automatically refreshed, e.g. on a monthly basis with a relative dependency cooldown policy of a week to reduce the exposure of the release process to ongoing OSS Supply Chain attacks.

The lockfile(s) in the release repo could either be managed via uv or pixi. I don't think it matters much because we don't need conda-forge for the build and testing process in this repo. All the Python-level builds dependencies are fetched from pypi.org and the rest come from the system image (e.g. manylinux). EDIT: actually we do fetch a fixed binary for libomp on macOS, so maybe we could use pixi and a lock file to do that more automatically.

When the lock files are updated, we should use relative dependency cooldowns such as uv's exclude-newer config or pixi's exlude-newer config).

Ideally, the build dependencies should also be locked explicitly in the repo as for the other dependencies. According to cibuildwheel's doc, the pip wheel command is used under the hood to setup the build env automatically. By default, it will install dependencies from the pyproject.toml file. However, it should be possible to pass an explicit pylock.toml export of a proper uv managed lockfile to pip wheel -r. Alternatively, we configure cibuildwheel's before-build hook to run a script to explicitly prepare the build environment manually using uv or pixi commands and disable build isolation and run pip wheel --no-build-isolation --no-deps.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading pyproject.toml and tools/configure_macos_env.sh, then review how cibuildwheel currently obtains build dependencies. Compare uv and pixi lockfile approaches, including relative dependency cooldowns and macOS libomp handling; done means build, sdist, and test dependencies are explicitly locked and refreshes are automated.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, shell
Domain
build-system, release
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.