commercialhaskell / commercialhaskell/stack
MacOS: some stack-built dynamic hspec-discover broken - @rpath seems the problem
- Dominant language
- Haskell
- Stars
- 4.1k
- Forks
- 850
- Avg merge
- 10h 37m
- Merged PRs (30d)
- 4
Description
### General summary/comments (optional)
For various reasons that don't belong here, I make `stack` build dynamic executables.
`stack.yaml`:
```
. . . . .
ghc-options:
# All packages
"$locals": -Wall -dynamic
"$targets": -Wall -dynamic
"$everything": -Wall -O2 -dynamic
apply-ghc-options: everything
. . . . .
```
However, the exact path to the built shared libraries does not match the path to the built executable. As a result, `dyld` fails to locate the required dynamic library.
Note: it (whatever "it" may be in the context) apparently has no problem locating dynamic libraries provided by GHC itself, like:
```sh
$ type shake
shake is /Users/ur20980/.local/bin/shake
$ ll ~/.local/bin/shake
-rwxr-xr-x 1 ur20980 MITLL\Domain Users 2981500 Feb 23 10:15 /Users/ur20980/.local/bin/shake*
$ ll ~/.local/lib
ls: /Users/ur20980/.local/lib: No such file or directory
$ otool -L ~/.local/bin/shake
/Users/ur20980/.local/bin/shake:
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.250.1)
@rpath/libHSutf8-string-1.0.1.1-IRxNMq5AVf036EceEkASq3-ghc8.8.2.dylib (compatibility version 0.0.0, current version 0.0.0)
@rpath/libHSunordered-containers-0.2.10.0-KujsThAgCnwGXBG7PJnkNO-ghc8.8.2.dylib (compatibility version 0.0.0, current version 0.0.0)
@rpath/libHSrandom-1.1-CUqV1zxrwrE4K5XCdTZSYy-ghc8.8.2.dylib (compatibility version 0.0.0, current version 0.0.0)
@rpath/libHSprimitive-0.7.0.0-BJKNB1CdxR09IS7Mq0RtDA-ghc8.8.2.dylib (compatibility version 0.0.0, current version 0.0.0)
@rpath/libHStransformers-0.5.6.2-ghc8.8.2.dylib (compatibility version 0.0.0, current version 0.0.0)
@rpath/libHSheaps-0.3.6.1-BfpBACLByuH5FJavluAE4c-ghc8.8.2.dylib (compatibility version 0.0.0, current version 0.0.0)
@rpath/libHShashable-1.3.0.0-CzctMgLWbfP4pv5F2JiKgj-ghc8.8.2.dylib (compatibility version 0.0.0, current version 0.0.0)
@rpath/libHSfilepattern-0.1.1-5FtHNCV6BzJ8E2IbrVBriy-ghc8.8.2.dylib (compatibility version 0.0.0, current version 0.0.0)
@rpath/libHSextra-1.6.18-HWOrRmqchYFEYlpvn3rDP8-ghc8.8.2.dylib (compatibility version 0.0.0, current version 0.0.0)
@rpath/libHSprocess-1.6.7.0-ghc8.8.2.dylib (compatibility version 0.0.0, current version 0.0.0)
@rpath/libHSdirectory-1.3.4.0-ghc8.8.2.dylib (compatibility version 0.0.0, current version 0.0.0)
@rpath/libHSunix-2.7.2.2-ghc8.8.2.dylib (compatibility version 0.0.0, current version 0.0.0)
@rpath/libHStime-1.9.3-ghc8.8.2.dylib (compatibility version 0.0.0, current version 0.0.0)
@rpath/libHSfilepath-1.4.2.1-ghc8.8.2.dylib (compatibility version 0.0.0, current version 0.0.0)
@rpath/libHSbinary-0.8.7.0-ghc8.8.2.dylib (compatibility version 0.0.0, current version 0.0.0)
@rpath/libHSbytestring-0.10.10.0-ghc8.8.2.dylib (compatibility version 0.0.0, current version 0.0.0)
@rpath/libHSbase-4.13.0.0-ghc8.8.2.dylib (compatibility version 0.0.0, current version 0.0.0)
@rpath/libHSinteger-gmp-1.0.2.0-ghc8.8.2.dylib (compatibility version 0.0.0, current version 0.0.0)
@rpath/libHSghc-prim-0.5.3-ghc8.8.2.dylib (compatibility version 0.0.0, current version 0.0.0)
@rpath/libHSrts_thr-ghc8.8.2.dylib (compatibility version 0.0.0, current version 0.0.0)
$ find ~/.ghcup -name libHSunix-2.7.2.2-ghc8.8.2.dylib -print
/Users/ur20980/.ghcup/ghc/8.8.2/lib/ghc-8.8.2/unix-2.7.2.2/libHSunix-2.7.2.2-ghc8.8.2.dylib
$ ~/.local/bin/shake --version
Shake build system, version 0.18.4
$
```
To my surprise, `haddock` was built the same way, and direct invocation of it by the shell succeeded:
```sh
$ ll ~/.local/bin/haddock
-rwxr-xr-x 1 ur20980 MITLL\Domain Users 18148 Feb 23 10:42 /Users/ur20980/.local/bin/haddock*
$ ~/.local/bin/haddock --version
Haddock version 2.23.0, (c) Simon Marlow 2006
Ported to use the GHC API by David Waern 2006-2008
$ otool -L ~/.local/bin/haddock
/Users/ur20980/.local/bin/haddock:
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.250.1)
@rpath/libHShaddock-api-2.23.0-77k1Xo77uBk4biRFAKXgBO-ghc8.8.2.dylib (compatibility version 0.0.0, current version 0.0.0)
@rpath/libHSbase-4.13.0.0-ghc8.8.2.dylib (compatibility version 0.0.0, current version 0.0.0)
@rpath/libHSghc-prim-0.5.3-ghc8.8.2.dylib (compatibility version 0.0.0, current version 0.0.0)
@rpath/libHSrts_thr-ghc8.8.2.dylib (compatibility version 0.0.0, current version 0.0.0)
$ find ~/.local -name libHShaddock-api-2.23.0-77k1Xo77uBk4biRFAKXgBO-ghc8.8.2.dylib -print
$ find ~/.stack -name libHShaddock-api-2.23.0-77k1Xo77uBk4biRFAKXgBO-ghc8.8.2.dylib -print
/Users/ur20980/.stack/snapshots/x86_64-osx/f5f923dc5112fee405ccc7ef160dc8d688fd3557b7985e9f20450d6af00f295c/8.8.2/lib/x86_64-osx-ghc-8.8.2/libHShaddock-api-2.23.0-77k1Xo77uBk4biRFAKXgBO-ghc8.8.2.dylib
$
```
Another example with `hspec-discover` - one built with Cabal (also dynamic) works, one built with Stack crashes (without the manual workaround described below):
```sh
$ ll ~/.cabal/bin/hspec-discover
lrwxr-xr-x 1 ur20980 MITLL\Domain Users 63 Feb 18 14:10 /Users/ur20980/.cabal/bin/hspec-discover@ -> ../store/ghc-8.6.5/hspc-dscvr-2.7.1-6d828e87/bin/hspec-discover
$ otool -L ~/.cabal/bin/hspec-discover
/Users/ur20980/.cabal/bin/hspec-discover:
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.250.1)
@rpath/libHShspc-dscvr-2.7.1-81cd5775-ghc8.6.5.dylib (compatibility version 0.0.0, current version 0.0.0)
@rpath/libHSbase-4.12.0.0-ghc8.6.5.dylib (compatibility version 0.0.0, current version 0.0.0)
@rpath/libHSghc-prim-0.5.3-ghc8.6.5.dylib (compatibility version 0.0.0, current version 0.0.0)
@rpath/libHSrts-ghc8.6.5.dylib (compatibility version 0.0.0, current version 0.0.0)
$ ~/.cabal/bin/hspec-discover -h
Usage: hspec-discover SRC CUR DST [--module-name=NAME]
$ ll ~/.local/bin/hspec-discover
-rwxr-xr-x 1 ur20980 MITLL\Domain Users 13456 Feb 24 14:08 /Users/ur20980/.local/bin/hspec-discover*
$ otool -L ~/.local/bin/hspec-discover
/Users/ur20980/.local/bin/hspec-discover:
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.250.1)
@rpath/libHShspec-discover-2.7.1-Ha3JBjibn11KonhyYMZPqQ-ghc8.8.2.dylib (compatibility version 0.0.0, current version 0.0.0)
@rpath/libHSbase-4.13.0.0-ghc8.8.2.dylib (compatibility version 0.0.0, current version 0.0.0)
@rpath/libHSghc-prim-0.5.3-ghc8.8.2.dylib (compatibility version 0.0.0, current version 0.0.0)
@rpath/libHSrts-ghc8.8.2.dylib (compatibility version 0.0.0, current version 0.0.0)
$ ~/.local/bin/hspec-discover -h
dyld: Library not loaded: @rpath/libHShspec-discover-2.7.1-Ha3JBjibn11KonhyYMZPqQ-ghc8.8.2.dylib
Referenced from: /Users/ur20980/.local/bin/hspec-discover
Reason: image not found
Abort trap: 6
$
$ find ~/.stack -name libHShspec-discover-2.7.1-Ha3JBjibn11KonhyYMZPqQ-ghc8.8.2.dylib -print
/Users/ur20980/.stack/snapshots/x86_64-osx/a9cf3cc2cddd3cc7bddda2de466f06e3510275e1f229024b61ba5e90415b5b40/8.8.2/lib/x86_64-osx-ghc-8.8.2/libHShspec-discover-2.7.1-Ha3JBjibn11KonhyYMZPqQ-ghc8.8.2.dylib
/Users/ur20980/.stack/snapshots/x86_64-osx/9edf2d661911847801d37f343e8c4873d3c5038df5249478f205f0b8f34a1730/8.8.2/lib/x86_64-osx-ghc-8.8.2/libHShspec-discover-2.7.1-Ha3JBjibn11KonhyYMZPqQ-ghc8.8.2.dylib
$ find ~/.stack -name hspec-discover -print
/Users/ur20980/.stack/indices/Hackage/packages/hspec-discover
/Users/ur20980/.stack/snapshots/x86_64-osx/a9cf3cc2cddd3cc7bddda2de466f06e3510275e1f229024b61ba5e90415b5b40/8.8.2/bin/hspec-discover
/Users/ur20980/.stack/snapshots/x86_64-osx/9edf2d661911847801d37f343e8c4873d3c5038df5249478f205f0b8f34a1730/8.8.2/bin/hspec-discover
```
### Steps to reproduce
In the project that uses `hspec` and `hspec-discovery`, run `stack test`.
Include any `.yaml` configuration if relevant:
- `/stack.yaml`: [stack.yaml.txt](https://github.com/commercialhaskell/stack/files/4246441/stack.yaml.txt)
- `/package.yaml`: [package.yaml.txt](https://github.com/commercialhaskell/stack/files/4246451/package.yaml.txt)
`test/Spec.hs`:
```hs
-- file test/Spec.hs
{-# OPTIONS_GHC -F -pgmF hspec-discover #-}
```
`test/SpecHelper.hs`:
```hs
module SpecHelper
( module Test.Hspec
, module Phoityne.Example.Math
) where
import Test.Hspec
import Phoityne.Example.Math
```
`test/Phoityne/Example/MathSpec.hs`:
```hs
module Phoityne.Example.MathSpec where
import SpecHelper
spec :: Spec
spec =
describe "sumGs" $ do
context "with [1..10]" $
it "should be 55" $
sumG 1 10 `shouldBe` 55
context "with [10..1]" $
it "should be 55" $
sumG 10 1 `shouldBe` 55
context "with [1..1]" $
it "should be 0" $
sumG 1 1 `shouldBe` 0
main :: IO ()
main = hspec spec
```
### Expected
Successful completion of tests, like:
```sh
$ stack test
Stack has not been tested with GHC versions above 8.6, and using 8.8.2, this may fail
Stack has not been tested with Cabal versions above 2.4, but version 3.0.1.0 was found, this may fail
Str3Split> test (suite: Str3Split-test)
Phoityne.Example.Math
sumGs
with [1..10]
should be 55
with [10..1]
should be 55
with [1..1]
should be 0
Finished in 0.0004 seconds, used 0.0005 seconds of CPU time
3 examples, 0 failures
Str3Split> Test suite Str3Split-test passed
$
```
### Actual
```
$ stack test
. . . . .
Building executable 'Str3Split-exe' for Str3Split-0.1.0.0..
Linking .stack-work/dist/x86_64-osx/Cabal-3.0.1.0/build/Str3Split-exe/Str3Split-exe ...
Preprocessing test suite 'Str3Split-test' for Str3Split-0.1.0.0..
Building test suite 'Str3Split-test' for Str3Split-0.1.0.0..
dyld: Library not loaded: @rpath/libHShspec-discover-2.7.1-Ha3JBjibn11KonhyYMZPqQ-ghc8.8.2.dylib
Referenced from: /Users/ur20980/.stack/snapshots/x86_64-osx/a9cf3cc2cddd3cc7bddda2de466f06e3510275e1f229024b61ba5e90415b5b40/8.8.2/bin/hspec-discover
Reason: image not found
/Users/ur20980/src/Str3Split/test/Spec.hs:1:1: error:
`hspec-discover' failed in phase `Haskell pre-processor'. (Exit code: -6)
|
1 | -- file test/Spec.hs
| ^
-- While building package Str3Split-0.1.0.0 using:
/Users/ur20980/.stack/setup-exe-cache/x86_64-osx/Cabal-simple_mPHDZzAJ_3.0.1.0_ghc-8.8.2 --builddir=.stack-work/dist/x86_64-osx/Cabal-3.0.1.0 build lib:Str3Split exe:Str3Split-exe test:Str3Split-test --ghc-options " -fdiagnostics-color=always"
Process exited with code: ExitFailure 1
```
Note, that the library exists:
```
$ ll /Users/ur20980/.stack/snapshots/x86_64-osx/a9cf3cc2cddd3cc7bddda2de466f06e3510275e1f229024b61ba5e90415b5b40/8.8.2/lib/x86_64-osx-ghc-8.8.2/libHShspec-discover-2.7.1-Ha3JBjibn11KonhyYMZPqQ-ghc8.8.2.dylib
-rwxr-xr-x 1 ur20980 MITLL\Domain Users 181432 Feb 24 14:06 /Users/ur20980/.stack/snapshots/x86_64-osx/a9cf3cc2cddd3cc7bddda2de466f06e3510275e1f229024b61ba5e90415b5b40/8.8.2/lib/x86_64-osx-ghc-8.8.2/libHShspec-discover-2.7.1-Ha3JBjibn11KonhyYMZPqQ-ghc8.8.2.dylib*
$
```
Also, note that the path to the corresponding binary does not have the "intermediate directory `x86_64-osx-ghc-8.8.2` between `.../lib/` and the actual file.
### Workaround
Replace `@rpath` that "it" (`stack` invoked via VS Code) apparently cannot resolve, with an absolute (*and correct*!) path for the affected executable, like:
```
$ install_name_tool -change @rpath/libHShspec-discover-2.7.1-Ha3JBjibn11KonhyYMZPqQ-ghc8.8.2.dylib /Users/ur20980/.stack/snapshots/x86_64-osx/a9cf3cc2cddd3cc7bddda2de466f06e3510275e1f229024b61ba5e90415b5b40/8.8.2/lib/x86_64-osx-ghc-8.8.2/libHShspec-discover-2.7.1-Ha3JBjibn11KonhyYMZPqQ-ghc8.8.2.dylib /Users/ur20980/.stack/snapshots/x86_64-osx/a9cf3cc2cddd3cc7bddda2de466f06e3510275e1f229024b61ba5e90415b5b40/8.8.2/bin/hspec-discover
```
### Stack version
```
$ stack --version
Version 2.1.3, Git revision 0fa51b9925decd937e4a993ad90cb686f88fa282 (7739 commits) x86_64 hpack-0.31.2
$ ghcup debug-info
Script version: 0.0.8
Script variables:
GHC install location: /Users/ur20980/.ghcup/ghc
Binary install location: /Users/ur20980/.ghcup/bin
Tarball cache location: /Users/ur20980/.ghcup/cache
Downloader: curl -L --fail -O
Script update url: https://gitlab.haskell.org/haskell/ghcup/raw/master//ghcup
GHC download baseurl: https://downloads.haskell.org/~ghc
Meta download url https://gitlab.haskell.org/haskell/ghcup/raw/master//.download-urls
Meta download format 1
Meta version url https://gitlab.haskell.org/haskell/ghcup/raw/master//.available-versions
Meta version format 1
Detected system information:
Architecture: x86_64
Distribution: Darwin
Distro alias: darwin
Distro version: 18.7.0
```
### Method of installation
* Official binary, downloaded from stackage.org or fpcomplete's package repository
Contributor guide
Assessment
This issue has not been assessed yet.