compas-dev / compas-dev/compas_cgal

pypi release on tag push

Open
#76 0 comments 1 reaction 2 assignees Claimed by @jf--- View on GitHub
Dominant language
C++
Stars
32
Forks
13
PR merge metrics
No merged PRs in 30d

Description

# PyPI release recipe — tag-triggered wheels for a nanobind/C++ binding

Pattern proven on tesseract_nanobind; transferable to any scikit-build-core + nanobind project with conda-forge C++ deps (e.g. compas_cgal).

Core shape: **one workflow per platform**, each `build → test-wheel → publish`, with publish tag-gated and authenticated via OIDC trusted publishing. No central release workflow; a release is complete when all platform workflows finish.

Why per-platform workflows instead of one big matrix:

- a flaky platform can be re-run without rebuilding the others (`skip-existing` makes re-runs idempotent)
- `paths-ignore` each other's workflow files, so touching the macOS pipeline doesn't burn Linux CI minutes
- each maps to its own PyPI trusted-publisher entry + GitHub environment — least-privilege per platform

## 1. Trigger

```yaml
on:
push:
branches: [main]
tags:
- '[0-9]+.[0-9]+.[0-9]+'
workflow_dispatch:

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
```

Bare-version tags, no `v` prefix. `git tag 1.2.3 && git push origin 1.2.3` fires all platform workflows in parallel. Same workflow also runs on main/PR pushes — only the publish job is tag-gated, so every PR exercises the full build+test path that a release will use.

## 2. Version — setuptools-scm, three defense layers

`pyproject.toml`:

```toml
[project]
dynamic = ["version"]

[tool.scikit-build.metadata.version]
provider = "scikit_build_core.metadata.setuptools_scm"

[tool.setuptools_scm]
local_scheme = "no-local-version"
```

Build job env:

```yaml
env:
SETUPTOOLS_SCM_PRETEND_VERSION_FOR_: ${{ github.ref_type == 'tag' && github.ref_name || '' }}
```

(`` = distribution name uppercased, `-`/`.` → `_`.)

The three layers, each motivated by a real shipped failure:

- **Pretend-version pin on tag builds** — CI steps that dirty the tree (generated `.pyi` stubs, patched configs) flip setuptools-scm into guess-next-dev mode and stamp `X.Y.Z.dev0` → stray dev release on PyPI.
- **`local_scheme = "no-local-version"`** — PyPI rejects `+g.d` local segments with HTTP 400; one bad wheel mid-batch aborts twine and fragments the release across platforms.
- **Publish-time assert** — refuse to upload any wheel whose filename version ≠ tag:

```bash
for whl in dist/*.whl; do
v=$(basename "$whl" | cut -d- -f2)
[ "$v" = "$GITHUB_REF_NAME" ] || { echo "::error::$whl is $v, expected $GITHUB_REF_NAME"; exit 1; }
done
```

Checkout needs `fetch-depth: 0` (scm wants tag history), and the pixi env needs `git` as a dependency (CI envs don't see system git).

## 3. Build

Per python-version matrix job:

1. `setup-pixi` with **`pixi-version` pinned** and **`locked: true`** — fail loud if `pixi.lock` drifts from the manifest instead of silently re-solving. A silently re-solved env once shipped ABI-corrupted wheels (dep built against one Eigen, libs against another → heap corruption).
2. typecheck
3. `pixi run build-wheel` → per-OS script: build with scikit-build-core, then repair/bundle with **auditwheel or patchelf** (Linux), **delocate** (macOS), **delvewheel `--analyze-existing`** (Windows).

C++ deps come from conda-forge via pixi (for compas_cgal: `cgal-cpp`, `gmp`, `mpfr`, `boost-cpp`, `eigen`) and get bundled into the wheel by the repair tool.

Platform notes:

- **Linux: build on the oldest-glibc runner you support** — that's what determines the manylinux tag. Build on ubuntu-22.04, test on ubuntu-24.04.
- **Never bundle libstdc++** — a second copy alongside the system one (pulled in by numpy) unifies `STB_GNU_UNIQUE` locale statics across copies and corrupts the heap. Exclude it in wheel repair; it's the platform's job.
- **abi3**: nanobind can target the stable ABI (`STABLE_ABI` in `nanobind_add_module`) — build one cp312-abi3 wheel and cover 3.12+ with it instead of one wheel per python.
- If a python-pinned conda dep closure blocks an old interpreter you must ship (e.g. cp39 for Rhino 8), split envs: a minimal env supplies the interpreter, `` env var points the build at the C++ stack from the main env. CGAL's closure is arch-only, so compas_cgal likely doesn't need this.

**ABI canary** (cheap, high value): immediately after building, `pip install --force-reinstall --no-deps` the wheel into the pixi env and run the handful of tests that cross the C++/Eigen boundary. Dep-set ABI rot dies here in ~30s instead of surfacing as scattered segfaults 15 min later.

## 4. Test — virgin environment

Separate `test-wheel` job (`needs: build`), on a **different runner image** than the build:

1. download wheel artifact
2. install into a plain `python -m venv` — no pixi, no conda → proves the wheel is genuinely self-contained
3. smoke import + version print
4. full pytest suite

If shipping abi3 wheels: add matrix rows where python > wheel-python (e.g. 3.13/3.14 installing the cp312 wheel) — that's the only real verification of the stable-ABI claim. Mark prerelease pythons `continue-on-error: true`.

Use `fail-fast: false` everywhere in release matrices — one flaky job cancelling siblings fragments the published wheel set.

## 5. Publish — tag-gated, OIDC trusted publishing

```yaml
publish:
needs: [build, test-wheel]
if: startsWith(github.ref, 'refs/tags/')
runs-on: ubuntu-latest # pypa action is Linux-only; wheels are just files here
environment: linux # one GitHub environment per platform workflow
permissions:
id-token: write # OIDC — no PyPI token stored anywhere
steps:
- uses: actions/download-artifact@v4
with: { pattern: 'python-*-linux-*', merge-multiple: true, path: dist/ }
- name: assert wheel version == tag
run: ... # see §2
- uses: pypa/gh-action-pypi-publish@release/v1
with:
skip-existing: true # idempotent re-runs — skip uploaded wheels, don't 400-abort
```

## One-time setup checklist for a new repo

1. PyPI → project → Publishing → add one **trusted publisher** per platform: repo, workflow filename, environment name.
2. GitHub → Settings → Environments → create `linux` / `macos` / `windows` (optionally with required reviewers as a manual release gate).
3. `pyproject.toml`: `dynamic = ["version"]` + scm provider + `no-local-version`.
4. Commit `pixi.lock`; pin `pixi-version` in `setup-pixi`; set `locked: true`.
5. First tag: push, watch all platform publishes land, `pip install ==X.Y.Z` from a clean machine.

## TLDR

push tag → per-platform workflows in parallel → locked-env build (version pinned to tag) → ABI canary → virgin-venv test → tag-gated publish (assert version==tag, OIDC, `skip-existing`) → PyPI.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.