commercialhaskell / commercialhaskell/stack

Get rid of persistent build errors due to non-atomic file writes in GHC and stack

Open
#4,559 14 comments 1 reaction 0 assignees View on GitHub
component: build resolution: upstream issue
Dominant language
Haskell
Stars
4.1k
Forks
850
Avg merge
10h 37m
Merged PRs (30d)
4

Description

GHC as of writing does not ensure that files are written atomically.

This means that a Ctrl+C, kill or reboot at the right time can result in truncated files.
GHC does not detect this, so that resuming/rerunning the build with `ghc --make` continues to show up error messages.

In such situation, the only workaround is to wipe all files (e.g. `stack`/`cabal`/`make` `clean`).

Specifically, we've observed the following to happen:

* object files being written half-way, resulting in persistent linker errors
* executable files being written half-way, so they can start executing but then crash
* users reporting downstream tooling bug reports that only a build directory wipe helped

There may be situations where `stack` has this problem too, but so far we believe all occurrences of this that we see are in GHC.

GHC issue about this: [#14533 - Make GHC more robust against PC crashes by using atomic writes](https://ghc.haskell.org/trac/ghc/ticket/14533)

## Repro

@lehins made a repro script that shows the problem in GHC at https://github.com/lehins/exec-kill-loop

## Planned solution

* GHC should use atomic writes (write to temp file, then `rename()` syscall).
* [x] Make repro script that shows the problem ([done](https://github.com/lehins/exec-kill-loop))
* [x] Decrease repro waiting time from hours to a few minutes by killing just at the right time, doing syscall interception ([done with `hatrace` test](https://github.com/nh2/hatrace/blob/e23d35a2d2c79e8bf49e9e2266b3ff7094267f29/test/HatraceSpec.hs#L185))
* [ ] Find all places where GHC does non-atomic file writes, and fix them ([ghc issue](https://ghc.haskell.org/trac/ghc/ticket/14533))
* [ ] If possible, add this as a system test to GHC CI

## Related issues

* https://github.com/commercialhaskell/stack/issues/1842 (and [GHC #9121 - Presence of dyn_o files not checked upon recompilation](https://ghc.haskell.org/trac/ghc/ticket/9121))

### Issues for which this _may_ be the reason

* https://github.com/commercialhaskell/stack/issues/2422#issuecomment-460362698
* https://github.com/commercialhaskell/stack/issues/4399#issuecomment-460362870
* https://stackoverflow.com/questions/54156236/how-to-resolve-however-the-given-installed-package-instance-does-not-exist-in

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.