npm / npm/cli

[BUG] npm install hangs for several minutes while downloading tarballs from registry that redirects package downloads to s3 links

Open
#6,517 6 comments 19 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Bug Needs Triage Release 8.x
Dominant language
JavaScript
Stars
10.1k
Forks
4.7k
Avg merge
2d 2h
Merged PRs (30d)
19

Description

Is there an existing issue for this?
  • I have searched the existing issues
This issue exists in the latest npm version
  • I am using the latest npm
Current Behavior

When using a private npm registry, such as Artifactory, the registry may redirect clients to a pre-signed S3 download link when attempting to download a package. During the install process, the progress bar appears to hang for several minutes at the end. Without the progress bar, install times are as expected. My guess is that it might have something to do with the length of the s3 links (they are much longer than npmjs and regular artifactory links).

The progress bar will stay in this state for several minutes:
Screenshot 2023-06-01 at 9 35 18 AM

For both processes the package download links and dependencies have already been resolved in package-lock.json, node_modules deleted, and cache cleaned:

npm i                                                             ok  02:08:30 PM

added 614 packages, and audited 615 packages in 3m

107 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities
npm i --progress=false                                            ok  02:35:53 PM

added 614 packages, and audited 615 packages in 11s

107 packages are looking for funding
  run `npm fund` for details

Expected Behavior

npm install times are comparable to the time it takes when the progress bar is disabled when using registries such as Artifactory

Steps To Reproduce
  1. Make sure the npm environment is clean by deleting node_modules and running ‘npm cache clean --force’
  2. Configure the npm registry to a registry that redirects package downloads to pre-signed s3 links.
  3. Run npm install
Environment
  • npm: 8.19.3
  • Node.js: 16.16.0
  • OS Name: Ventura 32GB
  • System Model Name: MacBook Pro Apple M1 Max
    Also reproduced on Ubuntu 20.04 with npm 8.19.3
  • npm config:
; "user" config from /Users/xx/.npmrc

//xxxx/artifactory/api/npm/my-npm/:_authToken = (protected)
progress = true
registry = "https://xxxx/artifactory/api/npm/my-npm/"

; node bin location = /Users/xx/.nvm/versions/node/v16.16.0/bin/node
; node version = v16.16.0
; npm local prefix = /Users/xx/packages/my-npm-package
; npm version = 8.19.3
; cwd = /Users/xx/packages/my-npm-package
; HOME = /Users/xx
; Run `npm config ls -l` to show all defaults.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing npm install with progress enabled and disabled against a registry that redirects package downloads to pre-signed S3 links, using the cleanup steps and environment described above. Trace the progress handling during tarball downloads and verify that installs with the progress bar take roughly the same time as installs with --progress=false.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, nodejs
Domain
cli, devtools
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.