commercialhaskell / commercialhaskell/stack

Feature discussion - Stack System Root

Open
#2,205 11 comments 2 reactions 0 assignees View on GitHub
type: discuss type: enhancement
Dominant language
Haskell
Stars
4.1k
Forks
850
Avg merge
10h 37m
Merged PRs (30d)
4

Description

I open this issue for tracking and discussing a new feature I'm implementing for Stack, which I describe here.

**Your comments are appreciated.**

There is a working implementation of the feature in [here](https://github.com/kernelim/stack/tree/stack-system-root).
# Intro

This change adds a new, optional configurable path that is available via `STACK_SYSTEM_ROOT` environment, and also override-able via a `stack-system-root` Stack config option, or a --stack-system-root command line argument.

The Stack System Root path is an additional directory bearing the same structure as `~/.stack`, but is considered a read-only cache for anything that is stored under `~/.stack`. When Stack
runs, it first checks for availability of stuff under the Stack System Root (downloads, installed GHC, indices, etc.), and only afterward tries to either download, build, or install stuff to `~/.stack`.

Any existing `STACK_ROOT` directory can be used as a Stack System Root, for another user, requiring only read-only access from that user.

This allows for:
- Reduced network accesses for each user.
- Savings in home dir sizes (and first-time compilation speeds)
- Faster compilations, because shared compiled packages can for each snapshot can be taken from the Stack System Root instead of re-compiled in the homedir.
- Therefore, a system-wide caching of all Stack's network accesses for all the users on the system that depend on the same snapshot.
- Further down the road, should make DEB and RPM packaging easier.
# Example

As `bob`, we can build some package, let's say `cpio-conduit`, but set our STACK_ROOT to be shared with another user:

```
bob@localhost $ export STACK_ROOT=/opt/shared/bob/stack-system-root
bob@localhost $ cd cpio-conduit
bob@localhost $ stack setup
bob@localhost $ stack build
```

Then, as `alice`, we start empty, and set `STACK_SYSTEM_ROOT` to that same path, which is read-only for us.

```
alice@localhost $ rm -rf ~/.stack
alice@localhost $ export STACK_SYSTEM_ROOT=/opt/shared/bob/stack-system-root
```

If we build the same package using the same Stack snapshot, then in this case, everything should be taken from the already populated `STACK_ROOT` of `bob`, even though it's read only. There wouldn't even be network accesses. If we depend on anything that `bob` does not have in the shared `STACK_ROOT`, then our own writable `STACK_ROOT` would be populated as usual.

```
alice@localhost $ cd cpio-conduit
alice@localhost $ stack build
[1 of 1] Compiling Main ( /tmp/stack5146/Setup.hs, /tmp/stack5146/Setup.o )
Linking /home/alice/.stack/setup-exe-cache/x86_64-linux/tmp-setup-Simple-Cabal-1.22.5.0-ghc-7.10.3 ...
cpio-conduit-0.7.0: configure
Configuring cpio-conduit-0.7.0...
cpio-conduit-0.7.0: build
Preprocessing library cpio-conduit-0.7.0...
[1 of 1] Compiling Data.CPIO ( src/Data/CPIO.hs, .stack-work/dist/x86_64-linux/Cabal-1.22.5.0/build/Data/CPIO.o )
In-place registering cpio-conduit-0.7.0...
cpio-conduit-0.7.0: copy/register
Installing library in
/home/alice/cpio-conduit/.stack-work/install/x86_64-linux/lts-5.8/7.10.3/lib/x86_64-linux-ghc-7.10.3/cpio-conduit-0.7.0-6ocIFacK3GqJwqA79J3bQu
Registering cpio-conduit-0.7.0...
```

After this build, `alice`'s own `.stack` is nearly empty (just some empty directories and an empty `package.cache`):

```
alice@localhost $ find /home/alice/.stack
/home/alice/.stack
/home/alice/.stack/config.yaml
/home/alice/.stack/snapshots
/home/alice/.stack/snapshots/x86_64-linux
/home/alice/.stack/snapshots/x86_64-linux/lts-5.8
/home/alice/.stack/snapshots/x86_64-linux/lts-5.8/7.10.3
/home/alice/.stack/snapshots/x86_64-linux/lts-5.8/7.10.3/pkgdb
/home/alice/.stack/snapshots/x86_64-linux/lts-5.8/7.10.3/pkgdb/package.cache
/home/alice/.stack/setup-exe-cache
/home/alice/.stack/setup-exe-cache/x86_64-linux
/home/alice/.stack/setup-exe-cache/x86_64-linux/setup-Simple-Cabal-1.22.5.0-ghc-7.10.3
```

Our ghc-pkg looks like:

```
alice@localhost $ stack exec -- ghc-pkg list
/opt/shared/bob/stack-system-root/programs/x86_64-linux/ghc-7.10.3/lib/ghc-7.10.3/package.conf.d
Cabal-1.22.5.0
array-0.5.1.0
base-4.8.2.0
bin-package-db-0.0.0.0
binary-0.7.5.0
bytestring-0.10.6.0
containers-0.5.6.2
deepseq-1.4.1.1
directory-1.2.2.0
filepath-1.4.0.0
ghc-7.10.3
ghc-prim-0.4.0.0
haskeline-0.7.2.1
hoopl-3.10.0.2
hpc-0.6.0.2
integer-gmp-1.0.0.0
pretty-1.1.2.0
process-1.2.3.0
rts-1.0
template-haskell-2.10.0.0
terminfo-0.4.0.1
time-1.5.0.1
transformers-0.4.2.0
unix-2.7.1.0
xhtml-3000.2.1
/opt/shared/bob/stack-system-root/snapshots/x86_64-linux/lts-5.8/7.10.3/pkgdb
async-2.1.0
attoparsec-0.13.0.1
base16-bytestring-0.1.1.6
blaze-builder-0.4.0.1
conduit-1.2.6.4
conduit-extra-1.1.11
exceptions-0.8.2.1
hashable-1.2.4.0
lifted-base-0.2.3.6
mmorph-1.0.6
monad-control-1.0.0.5
mtl-2.2.1
network-2.6.2.1
primitive-0.6.1.0
random-1.1
resourcet-1.1.7.3
scientific-0.3.4.6
stm-2.4.4.1
streaming-commons-0.1.15.2
text-1.2.2.0
transformers-base-0.4.4
transformers-compat-0.4.0.4
vector-0.11.0.0
zlib-0.6.1.1
/home/alice/.stack/snapshots/x86_64-linux/lts-5.8/7.10.3/pkgdb
/home/alice/cpio-conduit/.stack-work/install/x86_64-linux/lts-5.8/7.10.3/pkgdb
cpio-conduit-0.7.0
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.