astral-sh / astral-sh/python-build-standalone

Run the CPython test suite

Open
#642 0 comments 0 reactions 1 assignee Claimed by @geofft View on GitHub
Dominant language
Python
Stars
4.4k
Forks
314
Avg merge
1d 9h
Merged PRs (30d)
27

Description

We aren't running the CPython test suite regularly. We've had a handful of issues recently that would have been caught by the test suite, notably #622.

We advertise the test runner (and copy in the top-level runner which was removed in recent CPython versions) in PYTHON.json, and we have a runner _for_ the runner, but we don't actually run it in CI.

The only thing we run is that the PGO profiling portion of the upstream CPython Makefile runs a subset of the upstream test suite, and so we've had to patch a few of the tests out.

Almost certainly we will have some test failures in the remainder of the test suite.

docs/status.rst has some notes on test failures, but it hasn't been updated in several years.

Let's start by updating that, marking known test failures somehow, and putting this in CI in some fashion. I don't know if it should run on every PR and push to HEAD given how much CI we already expend. Maybe nightly + on release?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.