haskell / haskell/cabal

[RFC] Separate linking

Open
#9,256 0 comments 2 reactions 0 assignees View on GitHub
type: RFC
Dominant language
Haskell
Stars
1.7k
Forks
750
Avg merge
4d 3h
Merged PRs (30d)
28

Description

This is a continuation from my comment on #696 and this RFC attempts to solve the problems brought up in that issue as well.

## Problems at hand

FFI library setup is currently a part of the `.cabal` file. This has a number of consequences:

1. Users cannot define directories within which FFI libraries are to be found, only extend directories via `cabal.project`'s `extra-*-dirs` fields.

2. Static library linking can be done in two ways:

- By bundling all the FFI library files together with the package. While this works for FFI functions that are not exposed outside of the library, writing raw FFI bindings correctly means invoking `make`/`configure` through `Setup.hs` and also adding extra flags for dynamic linking.

- Implicitly through `pkg-config`. This option is so obscure I only learned about it while writing this.

3. In case of using `pkg-config`, users do not have direct control over whether any given FFI library is linked statically or dynamically. The system behavior currently depends on which files are visible during linking. Brought up in #8713.

4. Users cannot supply FFI library macros, the only solution currently available is converting every macro into a flag that adds the respectful macro.

## Scope of influence

Admittedly due to the size of the ecosystem and the general lack of FFI-dependent packages not a lot of people encounter these issues. The only proper example I can provide at the time is that [zlib.cabal](https://github.com/haskell/zlib/blob/ee937e35dce60c911c7dcee8f5c66f103ed2db36/zlib.cabal#L109-L125) has to ship an entire copy of zlib just to support Windows.

Nonetheless the issues with the current design permeate to other places:

- Granular customization of FFI dependencies is near-impossible, requiring concerted effort from every package with FFI dependencies to provide every feature needed downstream.

- The `Build information` section of the `.cabal` file is bloated with an unstructured assortment of FFI options, taking up a solid half of the field list. The documentation suffers heavily because of this, so does the new user experience.

## The proposal

0. Introduce a concept of FFI build strategies, known conversions from non-Haskell code into linkable objects.

Currently `C`/`C++`/`Asm`/`C--`/`JS` are supported. Since each is processed [separately](https://github.com/haskell/cabal/blob/992765f15e3c023bb413c294e68b4cdf0aa869f4/Cabal/src/Distribution/Simple/GHC.hs#L1791-L1814) there is no need to keep them bundled up, they can all be converted into a customizable shape:

```cabal
using c | cxx | asm | cmm | js
sources: ...
options: ...
```

This will allow both for reuse in different places and for clear documentation on which fields are applicable for each case.

Note that for format consistency include directories and macros should reside slightly outside of the example I provided, at the level of the `using` clause, due to the fact that they influence C preprocessing as well. Both of these should still find their way down to the compilers that expect them.

1. Introduce a `.cabal` section called `link-dependency `. Comes in two flavors:

- Internal.

Only visible inside the `.cabal` file. Can only be static and the section has to provide include directories, macros and an FFI build strategy. All file paths are package-local.

- Exposed.

Can be resolved to either static or dynamic down the line, but the `.cabal` file can only provide default automated solutions, e.g. the `pkg-config` library name.

Link dependencies can additionally "depend" on other visible link dependencies to share include directories and macros. For example libvorbis has an explicit one-way [libogg dependency](https://gitlab.xiph.org/xiph/vorbis/-/blob/84c023699cdf023a32fa4ded32019f194afcdad0/include/vorbis/codec.h#L25) and libogg is a completely separate library.

Link dependencies may be referred to as `:` to resolve potential ambiguities.

2. Introduce a `link-depends` field in build information. The values are a list of visible link dependencies.

3. Introduce a `.project` section called `link-dependency `. Comes in three flavors:

- Include directories, macros and compiled library files, static or dynamic. This is the "raw" handling, the arguments consumed directly by GHC.

- Include directories, macros and an FFI build strategy. Ideally the declaration shape here is the same as for an internal link dependency in a `.cabal` file, with global paths allowed.

- System utility discovery plus macros. `pkg-config`, `frameworks` and `extra-libraries` are all part of this.

`.project` definitions for a given `` take precedence over `.cabal` definitions for that name. They however should not be able to override the aforementioned link dependency dependencies.

4. Phase out current FFI-related options from build information in `.cabal` file and from the relevant section in `.project` file.

As link dependencies now form a graph of their own, their building and linking order will be more explicit and the application of relevant compilation options can be scoped down.

Better names for all of these things are appreciated. I would call link dependencies foreign libraries if that name weren't [already taken](https://cabal.readthedocs.io/en/3.10/cabal-package.html#foreign-libraries).

## Backwards compatibility

Every existing set of include directories and FFI sources can be converted to a set of internal link dependencies.

Every existing set of `pkg-config`/`frameworks`/`extra-libraries` fields can be converted to a set of exposed link dependencies, however these should have their own special namespace as to not clash with proper exposed link dependencies.

## Further extensions

- Macros as flags for listing optional modules and dependencies in FFI binding libraries.

For example Vulkan provides hundreds of extensions through macros and libspng [has an option](https://github.com/randy408/libspng/blob/e5c1fc470fceaca08b8c30dc40768c28b82b9e12/docs/build.md#build-options) to use miniz instead of zlib.

Contributor guide

Open the contributing guide

Research direction

Start by reading the current FFI handling in Cabal/src/Distribution/Simple/GHC.hs, especially the referenced processing code, then review the linked #696 and #8713 discussions. Compare the proposal with the zlib.cabal example and current .cabal and .project configuration. Done would require an agreed design and implementation plan for link dependencies, rather than a single localized change.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, cpp, haskell, javascript
Domain
build-system, developer-experience
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.