conda-forge / conda-forge/conda-forge.github.io
document/best practice for tracking ABI apart from package version
- Dominant language
- JavaScript
- Stars
- 170
- Forks
- 320
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 5
Description
### Your question:
Many libraries version their ABI separately from their API/package version. ABI stability varies widely, with some being extremely stable (major package revisions don't break the ABI for years and are _backward_ compatible), while others are the opposite (patch releases can break the ABI). Right now, we only have a standard of semver-pinning the package (with varying degrees of strictness, as appropriate), _assuming_ the package version is a good-enough proxy. I think that works _okay_ most of the time.
What I'd like is to document a standard/recommended approach for packagers who _want_ to explicitly track the ABI, so that it's not the wild west of different strategies for different packages. I expect this to be a relatively small number of packages, only those that have explicit ABI stability plans / versioning.
Strategies include:
- put SOVERSION in the package name, as is done in most linux distros (e.g. `libzmq5`). This can technically lead to allowing concurrent installation of multiple ABI versions, if we really want (I'm not sure we do)
- track versioned `libsomething_abi` package with the SOVERSION as the major version (minor/patch version may not always be the most obvious, but might come from the package version or the library filename). There is some precedent in `python_abi`, though it has more to consider than the version.
- others?
It would be nice to have written down:
- a preferred strategy, package naming
- how run_exports/constraints should be done
- migration process for existing packages (repodata patches, if needed)
As a simplest possible example, [libfabric](https://ofiwg.github.io/libfabric/main/man/fabric.7.html#abi-changes) has an explicitly versioned ABI, and the upcoming 2.0 release drops some old APIs but maintains backward compatibility in the ABI, so it is a minor release (ABI 1.8). This _cannot_ be represented as a version constraint in the package, because `<2.0` excludes a compatible version, and `<3.0` _may_ exclude a compatible version, we cannot know until 3.0 is planned/released.
So the correct run_exports should result in a build with libfabric 1.19.1 being runtime compatible with:
- libfabric 1.14 (older, same abi)
- libfabric 2.0 (newer, major package release, minor abi release)
but if/when there is an abi break, it won't be accepted at runtime, and the package version in which it will happen _cannot be known_.
My current thought is to have:
- `libfabric_abi` empty metapackage, version matching the documented ABI version (set build string to avoid/ensure duplicate uploads?)
- `libfabric` has:
```yaml
run_exports:
- {{ pin_subpackage('libfabric_abi', max_pin='x') }}
- libfabric # unconstrained
run:
- libfabric_abi {{ abi_version }}.*
```
which would allow safe backward-and-forward ABI compatibility. Relevant to this strategy is that the unconstrained `libfabric` means we would need repodata patches for all published libfabric builds to depend on libfabric_abi, otherwise the run_exports won't properly constrain the libfabric builds that are published without a dependency on `libfabric_abi`. We could avoid the repodata patch if `libfabric_abi` also had a `run_constrained` on libfabric to exclude any previously-published libfabric versions that are not ABI compatible.
but I don't want to do this if other strategies are preferred and would be (or already are?) taken elsewhere in conda-forge.
links:
- https://github.com/conda-forge/libfabric-feedstock/issues/12 for the libfabric-specific discussion
- https://github.com/conda-forge/staged-recipes/pull/28225#issuecomment-2529599024 where `libpmix2` is proposed, which I haven't seen suggested elsewhere
This is related to #2326 in that it is another case where the package version doesn't adequately capture the ABI, but in that case there are other inputs to the ABI (the compiler version), whereas this is purely about the contents of the package.
Contributor guide
Assessment
This issue has not been assessed yet.