apache / apache/beam

[Task]: Investigate native ARM runners for Python aarch64 wheels

Open
#39,620 0 comments 0 reactions 0 assignees View on GitHub
awaiting triage infra P2 python task
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
1d 20h
Merged PRs (30d)
196

Description

### What needs to happen?

Python wheel workflow currently builds Linux aarch64 wheel on an `ubuntu-22.04` x64 GitHub-hosted runner through QEMU.

In a representative PR run, aarch64 Python 3.14 leg took 91 minutes and determined almost all workflow latency. Other active legs took 5 to 26 minutes:

- Beam run: https://github.com/apache/beam/actions/runs/30917964633
- aarch64 job: https://github.com/apache/beam/actions/runs/30917964633/job/92021083741
- workflow configuration: https://github.com/apache/beam/blob/master/.github/workflows/build_wheels.yml

GitHub provides standard `ubuntu-24.04-arm` runners for public repositories. Other Apache projects already use them for native wheel builds. For example, Arrow Nanoarrow runs `cibuildwheel` on `ubuntu-24.04-arm`; a recent Linux ARM64 job completed successfully in about 11 minutes:

- GitHub runner documentation: https://docs.github.com/en/actions/reference/runners/github-hosted-runners
- cibuildwheel native ARM guidance: https://cibuildwheel.pypa.io/en/stable/faq/
- Arrow Nanoarrow workflow: https://github.com/apache/arrow-nanoarrow/blob/main/.github/workflows/python-wheels.yaml
- Example job: https://github.com/apache/arrow-nanoarrow/actions/runs/30664577816/job/91268851254

Could someone familiar with Beam Infra or Apache runner constraints confirm whether this wheel leg can use `ubuntu-24.04-arm`, or whether current QEMU path is intentional?

A bounded experiment could switch only aarch64 matrix row to native ARM, use `CIBW_ARCHS_LINUX=auto`, and skip QEMU for that row. Validation should compare wheel platform tags, `auditwheel show`, installation/import smoke tests, existing wheel tests, and wall-clock time against current QEMU job.

### Issue Priority

Priority: 2 (default / most normal work should be filed as P2)

### Issue Components

- [x] Component: Python SDK
- [ ] Component: Java SDK
- [ ] Component: Go SDK
- [ ] Component: Typescript SDK
- [ ] Component: IO connector
- [ ] Component: Beam YAML
- [ ] Component: Beam examples
- [ ] Component: Beam playground
- [ ] Component: Beam katas
- [ ] Component: Website
- [x] Component: Infrastructure
- [ ] Component: Spark Runner
- [ ] Component: Flink Runner
- [ ] Component: Prism Runner
- [ ] Component: Twister2 Runner
- [ ] Component: Hazelcast Jet Runner
- [ ] Component: Google Cloud Dataflow Runner

Contributor guide

Open the contributing guide

Research direction

Start with .github/workflows/build_wheels.yml and the linked current aarch64 job, then review GitHub runner and cibuildwheel ARM guidance. Confirm whether ubuntu-24.04-arm is usable for this matrix row and compare a bounded native-ARM run with QEMU using platform tags, auditwheel show, installation/import smoke tests, existing wheel tests, and wall-clock time.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, python
Domain
build-system, ci-cd, infrastructure
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
70/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.