If we don't need a package, then we don't use cabal.project.local

**Describe the bug**
I get a warning. `ld: warning: directory not found for option '-L/opt/homebrew/Cellar/xz/5.2.5/lib'`
I noticed that the build for mac os throws a warning.
Then I decided to create a `cabal.project.local` file with path `/usr/local/Cellar/xz/5.2.5/lib` and was surprised that the warning was still there.
It turned out that the package is no longer needed, but now the cabal does not see the changes from the `cabal.project.local`.

I have found some solutions. Not good but working.
If you remove or replace a `package lzma` in a `cabal.project`, then the warning disappears.

**To Reproduce**
Steps to reproduce the behavior:

`cabal init`

add to `cabal.project` (that path usually use in ubuntu)
```
packages:
.

package lzma
extra-include-dirs: /opt/homebrew/Cellar/xz/5.2.5/include
extra-lib-dirs: /opt/homebrew/Cellar/xz/5.2.5/lib
```
add to `cabal.project.local` (that path usually use in mac os)
```
ignore-project: True

package lzma
extra-include-dirs: /usr/local/Cellar/xz/5.2.5/include
extra-lib-dirs: /usr/local/Cellar/xz/5.2.5/lib
```

add to `.cabal` (we need at least one lib who using lzma, if we want to get warning)
```
build-depends:
base >=4.7 && <5
, servant-swagger-ui
```

```
$ cabal v2-build
```

**Expected behavior**
No warning.

**System information**
mac os
cabal 3.6.2.0
ghc 9.0.2

Start by reproducing the warning with the shown cabal.project, cabal.project.local, and .cabal contents using cabal v2-build on macOS. Trace how the project configuration handles the unused lzma package and verify the fix by confirming that the stale Homebrew library warning disappears without the workaround in cabal.project.

Written by the indexing model from the issue text.