[BUG] Ineffective caching of git dependencies

Is there an existing issue for this?

• I have searched the existing issues

This issue exists in the latest npm version

• I am using the latest npm

Current Behavior

After upgrading from Node 14 to Node 16.16, we're seeing an extreme increase in npm install times, in one case going from taking seconds to taking 17 minutes! We have a handful of git dependencies in those projects, built with TypeScript in their prepare script. I believe what made this process fast, and ultimately extremely useful, in Node 14 as that the packages were cached after the prepare step, which seems, in some cases, not to be the case after upgrading. In other cases, the cache is populated but seems to not be used, and sometime an update is not even properly performed.

Expected Behavior

npm install should be correct and fast by caching prepared git packages.

Steps To Reproduce

I have created a minimal project with a 30 second prepare script here and here, and a project that depends on it and can write the version of the dependency to the console here. The dependency there was was created with npm install git+ssh://git@github.com/kristoffer-zliide/npm-git-dep-test.git#9e32a70222842ce2033fb876d4af8182f369f22b with the output of "changed 1 package, and audited 2 packages in 3m", so even though the prepare step takes 30 seconds, installing the package takes more than four times that. Interestingly, running npm install without arguments changes git+ssh: to github:, but seemingly does nothing else, which is possibly also a bug. The project prints out "Version 1" as expected. Now

1. Change the git hash from 9e32a70222842ce2033fb876d4af8182f369f22b (Version 1) to 42146c03ce8b0d80de39d099baee5a2f1c979f89 (Version 2) in package.json
2. Run npm install - again this takes more than four times the expected 30 seconds
3. node index.js confirms that we're on Version 2
4. Change the git hash from 42146c03ce8b0d80de39d099baee5a2f1c979f89 back to 9e32a70222842ce2033fb876d4af8182f369f22b in package.json
5. Run npm install - this takes more than 2 minutes and it used to be almost instant on Node 14.

In the above scenario, an integrity property is added to the package-lock.json file with value sha512-iWdHaexWcOA0KPZtqFHI89eCkA34AB3wQCBJ8sP4Umcxqt0jWzL1N3o0KPB2vZD6+AqW03q/hn1i31Y3hmgvHQ==, and the prepared package is indeed to be found in npm-cache\_cacache\content-v2\sha512\89\67\ (confirmed with tar -xf 476...).

Clearly, there's some special handling of github URLs at play, so also (without the steps above) try

1. Uninstalling the package with npm uninstall npm-git-dep-test
2. Install the package from the other repo (git.zliide.com is a CNAME for bitbucket.org. The repo is public, but you need your ssh key to be added to a bitbucket account): npm install git+ssh://git@git.zliide.com/kristoffer_zliide/npm-git-dep-test.git#9e32a70222842ce2033fb876d4af8182f369f22b. This takes about one minute, now "only" twice the 30 seconds of the prepare script. Note that there's no integrity property, and that node index.js prints out Version 1
3. Change the git hash from 9e32a70222842ce2033fb876d4af8182f369f22b to 42146c03ce8b0d80de39d099baee5a2f1c979f89 in package.json
4. Run npm install - this takes no time, but also does nothing!!! node index.js still prints out Version 1. Is this a critical bug, or am I assuming too much of npm install?
5. Run npm update - this fixes it, again in about 1 minute; node index.js prints out Version 2. Still no integrity property. Instead of editing package.json and running npm install, doing a npm install git+ssh://git@git.zliide.com/kristoffer_zliide/npm-git-dep-test.git#42146c03ce8b0d80de39d099baee5a2f1c979f89, gets you here as well, without that scary Step 4.
6. Run npm install git+ssh://git@git.zliide.com/kristoffer_zliide/npm-git-dep-test.git#9e32a70222842ce2033fb876d4af8182f369f22b to downgrade to Version 1. This again takes about a minute and used to be almost instant on Node 14.

Environment

• npm: 8.11.0
• Node.js: 16.16.0
• OS Name: Windows 11
• npm config:

; "builtin" config from C:\Program Files\nodejs\node_modules\npm\npmrc

prefix = "C:\\Users\\...\\AppData\\Roaming\\npm"

; node bin location = C:\Program Files\nodejs\node.exe
; node version = v16.16.0
; npm local prefix = C:\Users\...\Source\temp\npm-git-test
; npm version = 8.11.0
; cwd = C:\Users\...\Source\temp\npm-git-test
; HOME = C:\Users\...

Reproduce the behavior in the linked npm-git-test project using package.json, package-lock.json, and index.js, comparing npm install with npm update for the two git commit hashes. Inspect the prepare script and the npm-cache_cacache\content-v2 path described in the report. Done means prepared git packages are reused correctly, commit changes are installed by npm install, and the documented timing and integrity behavior are corrected.