with-haddock cabal.project file option is ignored

**Describe the bug**
The `with-haddock` option is ignored, although passing `--with-haddock` to `cabal haddock` directly works.

**To Reproduce**
Use this in `cabal.project` file:
```
with-haddock: some non standard location
```
And run `cabal haddock -v3 all | grep -A1 "Using haddock"`, the reported path is not one given in `cabal.project` file.

Please use version-prefixed commands (e.g. `v2-build` or `v1-build`) to avoid ambiguity.

**Expected behavior**
A clear and concise description of what you expected to happen.

**System information**
- Operating system: `linux`
- `cabal-3.4.0.0-rc4`, `ghc-8.10.2` versions