[BUG] npm exec --workspaces very slow
Nobody has claimed this yet.
- Dominant language
- JavaScript
- Stars
- 10.1k
- Forks
- 4.7k
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 19
Description
Is there an existing issue for this?
- I have searched the existing issues
This issue exists in the latest npm version
- I am using the latest npm
Current Behavior
We have a monorepo using npm workspaces, containing ~70 workspaces/packages.
We noticed that running npm exec --workspaces (for example, npm exec --workspaces --call 'pwd') at the top-level of this monorepo was incredibly slow - the npm process would be pegged at 100% CPU and it would take multiple seconds to print the directory path for each workspace.
The overall run time of npm exec --workspaces --call 'pwd' can take several minutes.
I was able to bisect NPM release versions to identify that this command was fast in version 8.7.0, but slow in version 8.8.0, so it appears that https://github.com/npm/cli/pull/4643 introduced the slowness (cc @ruyadorno). Indeed if I locally revert the change from that PR (modified revert due to subsequent changes from #5250), the performance issue disappears:
--- /Users/gavin/.nvm/versions/node/v16.17.0/lib/node_modules/npm/lib/commands/exec.js 2022-09-14 15:12:56.000000000 -0400
+++ /Users/gavin/.nvm/versions/node/v16.17.0/lib/node_modules/npm/lib/commands/exec-2.js 2022-09-14 15:12:37.000000000 -0400
@@ -25,7 +25,7 @@
async exec (_args, { locationMsg, runPath } = {}) {
// This is where libnpmexec will look for locally installed packages
- const localPrefix = this.npm.localPrefix
+ const localPrefix = runPath ? runPath : this.npm.localPrefix
Expected Behavior
npm exec --workspaces --call 'pwd' should very quickly (<2s) print out all workspace directories.
Steps To Reproduce
- In a large monorepo with ~70 workspaces
- With a default workspaces config
- Run
npm exec --workspaces --call 'pwd' - See slowness
Environment
- npm: 8.19.2
- Node.js: 16.17.0
- OS Name: macOS 12.5.1
- System Model Name: Macbook Pro (2021 M1 Max)
- npm config:
; node bin location = /Users/gavin/.nvm/versions/node/v16.17.0/bin/node
; node version = v16.17.0
; npm local prefix = /Users/gavin/dev/league-web
; npm version = 8.19.2
; cwd = /Users/gavin/dev/league-web
; HOME = /Users/gavin
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with lib/commands/exec.js and the change introduced by pull request #4643, then reproduce npm exec --workspaces --call 'pwd' in a monorepo with about 70 workspaces. Compare the behavior with npm 8.7.0 and the reported revert; done means all workspace directories print in under two seconds without pegging the CPU.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript, node.js
- Domain
- cli, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 38/100