new-build --enable-benchmarks does not cause benchmarks to be built
- Dominant language
- Haskell
- Stars
- 1.7k
- Forks
- 750
- Avg merge
- 4d 3h
- Merged PRs (30d)
- 28
Description
**Describe the bug**
Passing `--enable-benchmarks` to `new-build` does not cause benchmarks to be built.
**To Reproduce**
Here is Travis CI log for `megaparsec` which has broken benchmarks but still it's all green:
https://travis-ci.org/mrkkrp/megaparsec/jobs/585309471#L296
The exact command in this case is:
```
cabal new-build megaparsec megaparsec-tests --enable-tests --enable-benchmarks --flags=dev
```
yet even simpler command also doesn't build benchmarks:
```
cabal new-build --enable-benchmarks
````
It returns with exit code 0 even though the benchmarks are broken and do not compile.
> Please use version-prefixed commands (e.g. `v2-build` or `v1-build`) to avoid ambiguity.
I got at first a different result with `v2-build`: it failed as it should. Then after that `new-build` also started to fail. Then later it started to report that the build is "up to date" returning 0 (the code is still broken).
**Expected behavior**
When `--enable-benchmarks` is passed, all benchmarks should be built and if they do not compile we should reliably get non-zero exit status no matter what.
**System informataion**
- Operating system: NixOS
- `cabal` version 2.4.1.0
- `ghc` version 8.6.5
Contributor guide
Assessment
This issue has not been assessed yet.