[BUG] npm ci still does not work with overrides correctly

Is there an existing issue for this?

• I have searched the existing issues

This issue is related to:
https://github.com/npm/cli/issues/4684, https://github.com/npm/cli/issues/4732, https://github.com/npm/cli/issues/4732

This issue exists in the latest npm version

• I am using the latest npm

Current Behavior

The release of npm@8.7.0 has made a fix for the aforementioned issues, but the same error occurs in my example repository on npm@8.10.0, and for any version above 8.6.0 for that matter, I tested #4732 with 8.7.0 and the error presented in that specific issue is resolved with any newer version of npm.

The bug is that if npm ci is used with certain dependencies and override combination the following error occurs:

npm ERR! `npm ci` can only install packages when your package.json and package-lock.json or npm-shrinkwrap.json are in sync. Please update your lock file with `npm install` before continuing.

This happens with any version of npm above 8.5.5, if 8.5.5 is used the error does not occur.

Furthermore, if you follow the instruction provided in the error to run npm install the security vulnerabilities that were originally resolved with the "overrides" are restored and the package-lock is updated. If you run npm ci after that the overrides are still not taken into consideration, so the overrides have been ignored.

I've set up a repository where the behavior is described in more detail which is listed under Steps To Reproduce

Expected Behavior

With npm ci I expect that the latest version of npm is able to use npm ci with the defined "overrides" without errors.

With npm install or npm install --package-lock-only I expect that the "overrides" configured are still respected and the package-lock is not updated and ignoring the overrides.

Steps To Reproduce

To test this do the following:

• Have npm 8.6.0 or greater installed (bug occurs for any version above 8.5.5)

1. Clone the repository git clone git@github.com:jchiem/npm-overrides-bug.git
2. Run npm ci
3. Run npm install or npm install --package-lock-only

For Step 2 you should get an error like:

For Step 3, it results in no errors, but the package-lock is updated and the node modules installed were done without overrides since security vulnerabilities that should have been resolved using the "overrides" in package.json are no longer 0.

If you run npm ci at this point, the installed packages retain the vulnerabilities from Step 3's npm install and the `"overrides" are ignored.

Environment

• npm: version above 8.5.5 (for example, 8.6.0 or 8.10.0)

• Node.js: 16.14.2

• OS Name: mac OS 12.3.1 (Monterey)

• System Model Name: MacBook Pro (16-inch, 2019)

• npm config:

; node bin location = /Users/joakim.chiem/.nvm/versions/node/v16.14.2/bin/node
; node version = v16.14.2
; npm local prefix = /Users/joakim.chiem/git/npm-overrides-bug
; npm version = 8.10.0
; cwd = /Users/joakim.chiem/git/npm-overrides-bug
; HOME = /Users/joakim.chiem
; Run `npm config ls -l` to show all defaults.

Start by cloning the linked npm-overrides-bug repository and reproducing the failure with npm ci, then compare the results of npm install and npm install --package-lock-only. Investigate npm CLI handling of package.json overrides and package-lock.json synchronization. Done means npm ci succeeds and both install commands preserve the configured overrides without updating the lockfile incorrectly.