haskell / haskell/cabal

priority of `build-tools`/`build-tool-depends` vs. pre-built `dist/build/*` in sdist

Open
#6,029 2 comments 0 reactions 0 assignees View on GitHub
cabal-install: cmd/sdist type: discussion
Dominant language
Haskell
Stars
1.7k
Forks
750
Avg merge
4d 3h
Merged PRs (30d)
28

Description

## Problem manifestation

Here's a case I stumbled over where I'm not sure we have the proper semantics:

```
$ cabal v2-repl -zb bytestring-lexing==0.4.0 -w ghc-8.0.2 -j1
Resolving dependencies...
Build profile: -w ghc-8.0.2 -O1
In order, the following will be built (use -v for more details):
- bytestring-lexing-0.4.0 (lib:bytestring-lexing) (requires build)
- fake-package-0 (lib) (first run)
Configuring bytestring-lexing-0.4.0...
Preprocessing library for bytestring-lexing-0.4.0..
Building library for bytestring-lexing-0.4.0..
[1 of 4] Compiling Data.ByteString.Lex.Internal ( src/Data/ByteString/Lex/Internal.hs, dist/build/Data/ByteString/Lex/Internal.o )

src/Data/ByteString/Lex/Internal.hs:28:5: warning: [-Wdeprecations]
In the use of ‘inlinePerformIO’
(imported from Data.ByteString.Internal):
Deprecated: "If you think you know what you are doing, use 'unsafePerformIO'. If you are sure you know what you are doing, use 'unsafeDupablePerformIO'. If you enjoy sharing an address space with a malevolent agent of chaos, try 'accursedUnutterablePerformIO'."
[2 of 4] Compiling Data.ByteString.Lex.Lazy.Double ( dist/build/Data/ByteString/Lex/Lazy/Double.hs, dist/build/Data/ByteString/Lex/Lazy/Double.o )

dist/build/Data/ByteString/Lex/Lazy/Double.hs:328:31: error:
• Couldn't match expected type ‘[AlexAcc
...
```

however, compare to

```
$ cabal get bytestring-lexing-0.4.0 && cd bytestring-lexing-0.4.0 && cabal v2-build -w ghc-8.0.2
Resolving dependencies...
Build profile: -w ghc-8.0.2 -O1
In order, the following will be built (use -v for more details):
- bytestring-lexing-0.4.0 (lib:bytestring-lexing) (first run)
Configuring bytestring-lexing-0.4.0...
Preprocessing library for bytestring-lexing-0.4.0..
Building library for bytestring-lexing-0.4.0..
[1 of 4] Compiling Data.ByteString.Lex.Internal ( src/Data/ByteString/Lex/Internal.hs, /tmp/bytestring-lexing-0.4.0/dist-newstyle/build/x86_64-linux/ghc-8.0.2/bytestring-lexing-0.4.0/build/Data/ByteString/Lex/Internal.o )

src/Data/ByteString/Lex/Internal.hs:28:5: warning: [-Wdeprecations]
In the use of ‘inlinePerformIO’
(imported from Data.ByteString.Internal):
Deprecated: "If you think you know what you are doing, use 'unsafePerformIO'. If you are sure you know what you are doing, use 'unsafeDupablePerformIO'. If you enjoy sharing an address space with a malevolent agent of chaos, try 'accursedUnutterablePerformIO'."
[2 of 4] Compiling Data.ByteString.Lex.Lazy.Double ( /tmp/bytestring-lexing-0.4.0/dist-newstyle/build/x86_64-linux/ghc-8.0.2/bytestring-lexing-0.4.0/build/Data/ByteString/Lex/Lazy/Double.hs, /tmp/bytestring-lexing-0.4.0/dist-newstyle/build/x86_64-linux/ghc-8.0.2/bytestring-lexing-0.4.0/build/Data/ByteString/Lex/Lazy/Double.o )

/tmp/bytestring-lexing-0.4.0/dist-newstyle/build/x86_64-linux/ghc-8.0.2/bytestring-lexing-0.4.0/build/Data/ByteString/Lex/Lazy/Double.hs:459:1: warning: [-Wtabs]
Tab character found here, and in 44 further locations.
Please use spaces instead.
[3 of 4] Compiling Data.ByteString.Lex.Integral ( src/Data/ByteString/Lex/Integral.hs, /tmp/bytestring-lexing-0.4.0/dist-newstyle/build/x86_64-linux/ghc-8.0.2/bytestring-lexing-0.4.0/build/Data/ByteString/Lex/Integral.o )
[4 of 4] Compiling Data.ByteString.Lex.Double ( /tmp/bytestring-lexing-0.4.0/dist-newstyle/build/x86_64-linux/ghc-8.0.2/bytestring-lexing-0.4.0/build/Data/ByteString/Lex/Double.hs, /tmp/bytestring-lexing-0.4.0/dist-newstyle/build/x86_64-linux/ghc-8.0.2/bytestring-lexing-0.4.0/build/Data/ByteString/Lex/Double.o )

/tmp/bytestring-lexing-0.4.0/dist-newstyle/build/x86_64-linux/ghc-8.0.2/bytestring-lexing-0.4.0/build/Data/ByteString/Lex/Double.hs:497:1: warning: [-Wtabs]
Tab character found here, and in 44 further locations.
Please use spaces instead.

src/Data/ByteString/Lex/Double.x:106:19: warning: [-Wdeprecations]
In the use of ‘inlinePerformIO’
(imported from Data.ByteString.Internal):
Deprecated: "If you think you know what you are doing, use 'unsafePerformIO'. If you are sure you know what you are doing, use 'unsafeDupablePerformIO'. If you enjoy sharing an address space with a malevolent agent of chaos, try 'accursedUnutterablePerformIO'."
```

## Diagnosis

The difference between the two builds is that in the first case, the pre-generated

- https://hackage.haskell.org/package/bytestring-lexing-0.4.0/src/dist/build/Data/ByteString/Lex/Double.hs

scanner is compiled (NB: that `alex` was still provisioned by `v2-build` to the build-process (in vain though) because it was declared via `build-tools`; but it wasn't invoked) , which is *not* compatible with GHC 8.0.2; in the 2nd case however the `build-tools: alex` declared in the package description is honored, and the

- https://hackage.haskell.org/package/bytestring-lexing-0.4.0/src/src/Data/ByteString/Lex/Double.x

is generated (and the pre-generated one in the `dist/build/` folder is ignored).

So the issue here is whether re-running the pre-processors (when those are specified in `build-tools`) shall take priority over using pre-generated ones in the sdist inside the magic `dist/build` folder.

And this is currently handled inconsistently in the local (unpacked) vs non-local case by `v2-build`.

## Suggested resolution

If `build-tools` for the respective pre-processor is specified in the `.cabal` file, `v2-build` shall consistently always re-generate the respective source *and* use it, regardless of its existence in a `dist/build` folder.

Contributor guide

Open the contributing guide

Research direction

Reproduce the two commands using the bytestring-lexing-0.4.0 package, comparing the .cabal build-tools declaration with the generated source under dist/build and src/Data/ByteString/Lex/Double.x. Trace v2-build's handling of local and non-local sdists, then verify that declared preprocessors consistently regenerate and use their sources instead of stale dist/build files.

Written by the indexing model from the issue text.

Assessment

Tech stack
haskell
Domain
build-system
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.