Reconfiguration avoidance not affected by library visibility change

**Describe the bug**
Visibility of dependencies does not get taken into account when cabal chooses whether to reconfigure or not a component

**To Reproduce**
Steps to reproduce the behavior:
* Project with two packages: `a`, `b`
* `b` has a public sublibrary `sublib`
* a component of `a` depends on `b:sublib`
* `cabal new-build all`
* make `sublib` private
* `cabal new-build all`
* `a` does not get reconfigured and there's no failure

**Expected behavior**

a gets reconfigured because of the change in b and the command fails

**System information**
cabal 3.0.0.0, ghc 8.8.1