conda-forge / conda-forge/conda-forge.github.io

General setup of static outputs vs. shared ones

Open
#1,809 23 comments 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
170
Forks
320
Avg merge
2d 10h
Merged PRs (30d)
5

Description

Static builds are already an exception in conda-forge (see CFEP-18), but some do exist for specific use-cases. The micromamba feedstock is a good [example](https://github.com/conda-forge/micromamba-feedstock/blob/main/recipe/meta.yaml#L58) of a user of static libs (presumably to stay "micro" and have no runtime dependencies).

Several feedstocks of this kind follow a pattern as follows:
```
outputs:
- name: libxyz
build:
run_exports:
- {{ pin_subpackage('libxyz') }}
[...]
- name: libxyz-static
requirements:
build:
- [...]
host:
- {{ pin_subpackage("libxyz", exact=True) }}
run:
- {{ pin_subpackage("libxyz", exact=True) }}
```

This has some advantages & disadvantages:
* 👍
* deduplication of files between static & shared builds
* shared & static builds are co-installable
* I'd argue this is actually an anti-pattern, but it can become relevant for dependencies with run-exports, see below
* 👎
* libxyz-static pulls in both dynamic and static builds
* this makes it possible for static builds to "silently" depend on the shared builds
* in practice this is not a huge issue because run-exports from `libxzy` don't get picked up though for a `-static` host dep, so reliance on the dynamic lib would be detected by going boom at runtime
* CMake integration is impossible for both packages simultaneously
* either the targets are wrong for the non-static version ("libzstd.a" not found)
* or the CMake-specific files clobber each other

The CMake issue in particular is quite painful, because an ever-increasing number of packages in C/C++-land come with built-in CMake integration. For example, recent LLVM-builds failed on https://github.com/conda-forge/zstd-feedstock/issues/58. There, I untangled the dependence in https://github.com/conda-forge/zstd-feedstock/pull/62, but this has the following trade-off:

> [...] we can have only 3 out of the following 4 (AFAICT):
> 1. working CMake integration
> 1. no clobbering of CMake files
> 1. no manual hacking (of CMake files resp. upstream CMake integration)
> 1. `zstd` & `zstd-static` co-installable
>
> Currently I've chosen to give up 4. - perhaps an argument can be made that giving up 2. is less harmful despite being against best practice.

The lack of being able to co-install `libxyz` and `libxyz-static` would become problematic in the following scenario (quoting @hmaarrfk):
> 1. User builds package `a`, which depends on `zstd` dynamic. `zstd` exports a requirement of `zstd`.
> 1. User tries to build package `b` that depends on `a` and `zstd-static`. This user, can no longer build their package.
>
> Package `b` cannot depend on just `zstd-static` because the dependency on `zstd` is controlled by package `a`.

In this case, that PR was merged (and the existing consumers are not affected by the run-export-induced conflict between dynamic & static libs), but now I've encountered the same in libprotobuf, and fixing https://github.com/conda-forge/libprotobuf-feedstock/issues/68 is not possible without running into the same issue, hence I thought I'd open a wider discussion.

@hmaarrfk [sketched](https://github.com/conda-forge/zstd-feedstock/pull/62#issuecomment-1208041241) out a different approach:
> The other alternative, which I don't have time to flesh out, is to return to the previous system, but, to ensure that:
> 1. `cmake` works with the dynamic libraries.
> 1. `cmake` fails to automatically report the static libraries.
>
> This would favor the dynamic library usage, instead of forcing users of cmake+zstd to have the static libraries installed.

to which I said:
> Wouldn't that defeat the point of `zstd-static`? If I use an output named like that, I'd certainly expect the static lib to be picked up, not the dynamic one (i.e. compiling against `zstd-static` would seem to work, but would actually do the same as compiling against `zstd`, i.e. create a runtime dependence on `zstd`)

### Summary

* The setup currently used by several static outputs is incompatible with CMake, which is IMO a bigger downside than getting some file duplication and even co-installability
* In general, I think it's really iffy to have static and shared builds of the same libs in the same host environment.
* There are other feedstocks where this setup is not possible/sensible, e.g. https://github.com/conda-forge/abseil-cpp-feedstock/pull/35

## Proposal: Forbid co-installation of `libxyz-static` builds with `libxyz`

This would solve the CMake issues (each output could have its own copy of the respective CMake files without risks of inconsistency or clobbering). The only downside would be that users of `-static` packages cannot depend on another package (say `a`) which has been built against a shared version of the same lib. Since most of the static libs live pretty close to the bottom of the stack, I don't think this would be much of a problem. And even so, there would be a solution: build a (perhaps also static?) version of `a` against `libxyz-static`.

### I think (users of) static libs are special enough that we can inflict this (hypothetical!) pain on them

Thoughts @conda-forge/core?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.