vercel / vercel/ncc

Size of node_modules has considerable effect on build speed

Open
#372 2 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
JavaScript
Stars
9.8k
Forks
326
PR merge metrics
No merged PRs in 30d

Description

I have a repository with a quite a lot of dependencies (node_modules size about 225mb with 625 folders). Code is written in TypeScript.

I've noticed that ncc build seems to run quite slow (around 3 to 5 seconds per build). That didn't seem normal (most dependencies are for development, so they shouldn't be involved with the build process). I tried tweaking bunch of things to get an idea what's wrong, but nothing really seemed to have too much of an effect on the build time (removing imports of larger dependencies etc.)

I suspected TypeScript for a while, so I ended up checking how fast the now-examples/typescript build ran. It took about 0.5 to 1 second on my machine, so something was clearly different with that one.

Did a small experiment with adding a bunch of random dependencies to the now-examples/typescript repository, and seeing if that had any effect on the build time. It seems to slow the building a lot, even without the code importing any of those dependencies.

Here's how to reproduce this:

yarn global add @zeit/ncc
git clone https://github.com/zeit/now-examples
cd now-examples/typescript
yarn
ncc build --watch ./src/frontend/index.ts

With this, the build times are as follows:

ncc: Version 0.18.2
ncc: Compiling file index.js
ncc: Using typescript@3.4.5 (local user-provided)
ncc: Module directory "/now-examples/typescript/node_modules/node-fetch/lib" attempted to require "encoding" but could not be resolved, assuming external.
 6kB  dist/style.css
48kB  dist/index.js
54kB  [2318ms] - ncc 0.18.2
Watching for changes...
File change, rebuilding...
ncc: Module directory "/now-examples/typescript/node_modules/node-fetch/lib" attempted to require "encoding" but could not be resolved, assuming external.
 6kB  dist/style.css
48kB  dist/index.js
54kB  [1247ms] - ncc 0.18.2
Watching for changes...
File change, rebuilding...
ncc: Module directory "/now-examples/typescript/node_modules/node-fetch/lib" attempted to require "encoding" but could not be resolved, assuming external.
 6kB  dist/style.css
48kB  dist/index.js
54kB  [731ms] - ncc 0.18.2
Watching for changes...
File change, rebuilding...
ncc: Module directory "/now-examples/typescript/node_modules/node-fetch/lib" attempted to require "encoding" but could not be resolved, assuming external.
 6kB  dist/style.css
48kB  dist/index.js
54kB  [600ms] - ncc 0.18.2
Watching for changes...
File change, rebuilding...
ncc: Module directory "/now-examples/typescript/node_modules/node-fetch/lib" attempted to require "encoding" but could not be resolved, assuming external.
 6kB  dist/style.css
48kB  dist/index.js
54kB  [596ms] - ncc 0.18.2
Watching for changes...

Adding some random node modules after this:

yarn add lodash request chalk commander async react debug yargs minimist colors prop-types mkdirp react-dom fs-extra moment vue axios postgraphile semver babel babel-loader eslint

Ending up with node_modules of about 120mb and 432 folders. Then running the same build command as before, I end up with the following build times:

ncc: Version 0.18.2
ncc: Compiling file index.js
ncc: Using typescript@3.4.5 (local user-provided)
ncc: Module directory "/now-examples/typescript/node_modules/node-fetch/lib" attempted to require "encoding" but could not be resolved, assuming external.
 6kB  dist/style.css
48kB  dist/index.js
54kB  [3119ms] - ncc 0.18.2
Watching for changes...
File change, rebuilding...
ncc: Module directory "/now-examples/typescript/node_modules/node-fetch/lib" attempted to require "encoding" but could not be resolved, assuming external.
 6kB  dist/style.css
48kB  dist/index.js
54kB  [2723ms] - ncc 0.18.2
Watching for changes...
File change, rebuilding...
ncc: Module directory "/now-examples/typescript/node_modules/node-fetch/lib" attempted to require "encoding" but could not be resolved, assuming external.
 6kB  dist/style.css
48kB  dist/index.js
54kB  [1852ms] - ncc 0.18.2
Watching for changes...
File change, rebuilding...
ncc: Module directory "/now-examples/typescript/node_modules/node-fetch/lib" attempted to require "encoding" but could not be resolved, assuming external.
 6kB  dist/style.css
48kB  dist/index.js
54kB  [1717ms] - ncc 0.18.2
Watching for changes...
File change, rebuilding...
ncc: Module directory "/now-examples/typescript/node_modules/node-fetch/lib" attempted to require "encoding" but could not be resolved, assuming external.
 6kB  dist/style.css
48kB  dist/index.js
54kB  [1598ms] - ncc 0.18.2
Watching for changes...
File change, rebuilding...
ncc: Module directory "/now-examples/typescript/node_modules/node-fetch/lib" attempted to require "encoding" but could not be resolved, assuming external.
 6kB  dist/style.css
48kB  dist/index.js
54kB  [1831ms] - ncc 0.18.2
Watching for changes...

Seems that build times are 2-3 times higher with just random stuff in the node_modules folder that isn't even used.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the timing difference in now-examples/typescript with ncc build --watch ./src/frontend/index.ts, first with the original dependencies and then after the listed packages are added. Investigate why unused node_modules folders affect the build, using the reported ncc 0.18.2 output as the baseline; done means unused dependencies no longer cause this slowdown.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, node.js, typescript
Domain
build-system, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.