haskell / haskell/cabal

new-build --enable-benchmarks does not cause benchmarks to be built

Open
#6,259 18 comments 0 reactions 1 assignee Claimed by @alt-romes View on GitHub
type: bug
Dominant language
Haskell
Stars
1.7k
Forks
750
Avg merge
4d 3h
Merged PRs (30d)
28

Description

**Describe the bug**

Passing `--enable-benchmarks` to `new-build` does not cause benchmarks to be built.

**To Reproduce**

Here is Travis CI log for `megaparsec` which has broken benchmarks but still it's all green:

https://travis-ci.org/mrkkrp/megaparsec/jobs/585309471#L296

The exact command in this case is:

```
cabal new-build megaparsec megaparsec-tests --enable-tests --enable-benchmarks --flags=dev
```

yet even simpler command also doesn't build benchmarks:

```
cabal new-build --enable-benchmarks
````

It returns with exit code 0 even though the benchmarks are broken and do not compile.

> Please use version-prefixed commands (e.g. `v2-build` or `v1-build`) to avoid ambiguity.

I got at first a different result with `v2-build`: it failed as it should. Then after that `new-build` also started to fail. Then later it started to report that the build is "up to date" returning 0 (the code is still broken).

**Expected behavior**

When `--enable-benchmarks` is passed, all benchmarks should be built and if they do not compile we should reliably get non-zero exit status no matter what.

**System informataion**
- Operating system: NixOS
- `cabal` version 2.4.1.0
- `ghc` version 8.6.5

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.