apache / apache/arrow

[Python][Packaging] Drop archery and docker for our Windows wheels and build on the GitHub runner directly

Open
#48,987 7 comments 2 reactions 0 assignees View on GitHub
Component: Packaging Component: Python Type: enhancement
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 13h
Merged PRs (30d)
88

Description

### Describe the enhancement requested

One of the benefits of archery and docker for building wheels is reproducibility on local environments but for the specific case of Windows wheels not many maintainers have a Docker on Windows setup where they can reproduce them. When we have tried to set a Docker on Windows to reproduce the wheels has proven challenging.

We have had plenty of issues in the past on our Docker Windows wheels setup, installing Windows build tools, installing Python (pymanager), chocolatey. We currently have 5 different Dockerfiles to manage:
- Installation of build tools (python-wheel-windows-vs2022-base.dockerfile)
- Installation of Python (both free-threaded and non free-threaded) for building the wheel with pymanager (python-wheel-windows-vs2022.dockerfile)
- Installation of build tools for test (python-wheel-windows-test-vs2022-base.dockerfile)
- Python installation with chocolatey for tests (python-wheel-windows-test-vs2022.dockerfile)
- Python test for free-threaded with extra handling (python-free-threaded-wheel-windows-test-vs2022.dockerfile)

All of them are managed with a series of flags and environment variables which has proven complex to maintain.

I think we can probably take a step down and use tWindows GitHub runners without docker. Those already provide build tools installed, we can leverage the use of the setup-python GitHub action to install the Python version.

Removing the docker setup (only for Windows wheels) and using Windows GitHub runners can probably simplify our set-up.

### Component(s)

Packaging, Python

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.