InsightSoftwareConsortium / InsightSoftwareConsortium/ITK
Roadmap: free-threaded CPython, abi3t, and alternative Python binding generators
- Dominant language
- C++
- Stars
- 1.7k
- Forks
- 748
- Avg merge
- 1d 1h
- Merged PRs (30d)
- 64
Description
Roadmap issue, not a work item — nothing here is actionable until Python 3.15 ships. PR #6715 makes ITK Python wrapping `abi3` by default while keeping a version-specific build path. This issue records why that path exists, when it can be deleted, and what else in this space ITK does and does not intend to support.
**Tracking statement:**
> The non-abi3 path exists for one documented, time-limited reason, and should be removed when Python 3.15 and CMake 4.4 make abi3t reachable.
Why the framing is "one documented reason" rather than "distributions might want it"
Every candidate justification for keeping a non-`abi3` build was checked against primary sources. Exactly one is documented:
| Candidate reason | Status |
|---|---|
| Free-threaded CPython 3.13t / 3.14t | **Documented** (PEP 803). The only one. |
| `Py_DEBUG` builds | **Not documented** — the CPython stable-ABI page names no debug-build exclusion, only platform/compiler considerations. |
| CMake < 3.26 | Documented in CMake, but a toolchain constraint rather than a Python one. |
| Linux distributions / conda-forge | No documentation found requiring it. conda-forge documents `abi3` *support*, and `conda-forge/itk-feedstock` currently builds only `____cpython` variants (python 3.10/3.11/3.12 × hdf5 × platform — no free-threaded `t` variants). |
| Performance under `Py_LIMITED_API` | Unmeasured, and judged not a concern for ITK's coarse-grained call pattern. |
| PyPy / GraalPy | Would require a build that is neither `abi3` nor `abi3t`, but nothing documents ITK needing to support them. Out of scope — see below. |
From PEP 803:
> "The Stable ABI is currently not available for free-threaded builds. Extensions will fail to build for the Stable ABI on free-threaded Python (that is, when both `Py_LIMITED_API` and `Py_GIL_DISABLED` preprocessor macros are defined)."
Combined with `abi3t` targeting 3.15+, free-threaded CPython 3.13t/3.14t sits in a gap where `abi3` is a build failure and `abi3t` does not yet exist. The only buildable configuration there is version-specific.
Stating it this narrowly is what makes the option deletable later: a maintainer gets a named expiry condition instead of an open-ended "someone might need this."
Phase 1 — now: abi3 available everywhere, default not yet changed
PR #6715 makes `ITKVtkGlue` stable-ABI clean, removing the last module that
could not be built with `ITK_USE_PYTHON_LIMITED_API=ON`. Its typemaps use
Limited API calls only, which are valid in a full-API build too, so one
implementation serves both modes.
Not yet done, and deliberately out of scope for #6715:
- [ ] Change `ITK_USE_PYTHON_LIMITED_API` to default `ON`. Prototyped and
verified locally (both modes build 8565/8565 and pass 177/177
`ctest -R Python` on macOS arm64), but held back as separate work.
This also lets `find_package(Python3 ...)` run once instead of twice:
the current default is derived from the *discovered* interpreter
version, which forces a second search once the components are known.
- [ ] Record the PEP 803 expiry alongside the option, so a future maintainer
knows the `OFF` path is deletable rather than open-ended.
- [ ] Note for whoever does this: `Modules/Core/Common/src/CMakeLists.txt`
uses `_itk_python_target` at **two** sites. Updating only the first
breaks `ITK.macOS.Python` with undefined `_Py_BuildValue` — it fails
there and nowhere else because only a shared macOS build links
`libITKCommon` strictly enough to notice.
Phase 2 — gated on Python 3.15 + CMake 4.4: add abi3t
`abi3t` is a *subset* of `abi3`'s allowed APIs and a *superset* of its compatible interpreters, so the abi3 work in #6715 is a strict prerequisite. It is not a rebuild: PEP 803 requires both `Py_LIMITED_API` and `Py_TARGET_ABI3T`, and recommends shipping both variants under the wheel tag `abi3.abi3t`.
CMake already models this and **no new `find_package` component is needed** — `Development.SABIModule` and `USE_SABI` are unchanged. From the `FindPython3` documentation (added in CMake 4.4):
> "When `USE_SABI` is specified, if `Python3_FREE_THREADED` is true and Python version is 3.15 or upper, `Py_TARGET_ABI3T` preprocessor definition will be defined rather than `Py_LIMITED_API`."
Work items when the gate opens:
- [ ] Guard the free-threaded-below-3.15 gap. For a free-threaded 3.13t/3.14t interpreter with `USE_SABI` specified, CMake still defines `Py_LIMITED_API` and hands you the failing compile. ITK should detect `Python3_FREE_THREADED` and select the version-specific path automatically, so users are not required to know the option exists.
- [ ] Add a conditional CMake floor of 4.4 *inside* the abi3t branch only. The 3.26 floor for `Development.SABIModule` stays for everything else; do not raise `ITK_OLDEST_VALIDATED_POLICIES_VERSION` (currently 3.22.1).
- [ ] Remove `ITK_USE_PYTHON_LIMITED_API` once every supported interpreter can do `abi3` or `abi3t`.
- [ ] ITKPythonPackage: a second artifact per platform for `abi3.abi3t`, partially offsetting the wheel-count reduction `abi3` enables (relevant to #4656).
Phase 3 — gated on a thread-safety audit (the larger piece of work)
Buildable under free-threading is not the same as correct under free-threading. Removing the GIL removes the implicit serialisation that ITK's Python layer relies on today. Un-audited areas, at minimum:
- `itk::ObjectFactoryBase` registration and the factory registry
- the SWIG type registry
- the `itkPyCommand` callback path
This is expected to be substantially more work than the ABI change, and should not be conflated with it.
Alternative binding generators — scope
**nanobind** — gated on raising ITK's wrapping floor to Python 3.12, since nanobind's stable-ABI mode requires 3.12 while `ITK_WRAP_PYTHON_MINIMUM_VERSION` is 3.11. Not actionable before that floor moves.
**pybind11** — no Limited API support.
**PyPy / GraalPy — explicitly out of scope** unless someone brings a concrete use case. PyPy's benefit is JIT-compiling hot pure-Python loops, but ITK's Python layer is a thin SWIG shim over C++ with no such loop; PyPy runs C extensions through `cpyext`, whose per-call overhead is worse than CPython's, and the C-API boundary is exactly where ITK spends its time. No Discourse thread, issue, or downstream project (Slicer, SimpleITK, ANTs) has requested it. Recorded here so the question does not need re-litigating in every wrapping discussion.
References: [PEP 803](https://peps.python.org/pep-0803/) · [CPython stable ABI](https://docs.python.org/3/c-api/stable.html) · [CMake FindPython3](https://cmake.org/cmake/help/latest/module/FindPython3.html) · [quansight: abi3t](https://labs.quansight.org/blog/python-abi-abi3t)
Raised from review discussion on PR #6715 (@blowekamp asked about free-threading implications; @thewtex noted `abi3t` will need a separate build). Related: #6711, #4656.
Contributor guide
Research direction
This is a roadmap rather than an immediately actionable work item. When a phase opens, start with CMake/ITKSetPython3Vars.cmake, Wrapping/macro_files/itk_end_wrap_module.cmake, and both _itk_python_target sites in Modules/Core/Common/src/CMakeLists.txt; done means the gated ABI paths, expiry condition, and relevant builds or tests work without breaking existing configurations.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cmake, cpp, python
- Domain
- build-system, devtools
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100