`new-build` dependency resolution performance problems with custom builds

**Context**
I'm converting a project that uses custom builds and `stack` to use `cabal new-build`. I'm running into performance problems.

[repro.tar.gz](https://github.com/haskell/cabal/files/3774090/repro.tar.gz)

**Describe the bug**
A clear and concise description of what the bug is.

On a project with 22 packages, dependency resolution is taking 2 minutes. The structure of the project is as follows:

- `ex-base`, which has a lot of dependencies and uses custom build
- `ex-test`, which has a dependency on `ex-base` and produces both a library and an executable and which does not use custom build, due to #6011
- `dep01` through `dep20`, which setup-depends on `ex-base` and has a test suite that build-tool-depends on `ex-test`

**To Reproduce**
Steps to reproduce the behavior:

```
$ mkdir test
$ wget https://github.com/haskell/cabal/files/3774090/repro.tar.gz
$ tar xf repro.tar.gz
$ time cabal v2-build --dry-run
```

Please use version-prefixed commands (e.g. `v2-build` or `v1-build`) to avoid ambiguity.

**Expected behavior**
A clear and concise description of what you expected to happen.

It should be faster.

**System information**
- PopOS
- cabal-install version 3.0.0.0
- compiled using version 3.0.0.0 of the Cabal library
- The Glorious Glasgow Haskell Compilation System, version 8.2.2

**Additional context**
The actual project I'm trying to migrate has 72 packages.

Unpack repro.tar.gz and run the documented `cabal v2-build --dry-run` command with the stated Cabal and GHC versions. Compare the dependency-resolution timing with the reported two minutes and trace the relevant build-resolution path; done means the reproduction completes materially faster without breaking the custom-build dependency structure.

Written by the indexing model from the issue text.