netlify / netlify/cli

`netlify dev` errors and hangs for large number of files

Open
#7,833 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

type: bug
Dominant language
TypeScript
Stars
1.9k
Forks
474
Avg merge
23h 30m
Merged PRs (30d)
53

Description

Describe the bug

I have a large site with lots of files.

 ➜  find _site/ -type f | wc -l
302005

When running netlify dev I get the following error message printed many times over continuously in the terminal.

 ›   Error: Netlify CLI has terminated unexpectedly
This is a problem with the Netlify CLI, not with your application.
If you recently updated the CLI, consider reverting to an older version by running:

npm install -g netlify-cli@VERSION

You can use any version from https://ntl.fyi/cli-versions.

Please report this problem at https://ntl.fyi/cli-error including the error details below.

While it says "including the error details below", there are no other error details printed.

The CPU is pegged at 100% and the server does not respond to HTTP requests. The Netlify CLI seems to be continuously starting subshells which are not waited on, creating many zombie processes. There are thousands of these started within a few seconds of running netlify dev.

 ➜  ps -ef
...
ian      2970231 2952121  0 07:23 pts/2    00:00:00 [sh] <defunct>
ian      2970233 2952121  0 07:23 pts/2    00:00:00 [sh] <defunct>
ian      2970244 2952121  0 07:23 pts/2    00:00:00 [sh] <defunct>
ian      2970246 2952121  0 07:23 pts/2    00:00:00 [sh] <defunct>
...

I notice subshells running npm in the output of ps from the same parent process.

ian      3006104 2952121  0 07:25 pts/2    00:00:00 /bin/sh -c npm -v
ian      3006017 2952121  0 07:25 pts/2    00:00:00 /bin/sh -c npm get prefix --global

The cli does not respond to Ctrl-C and I need to kill it with SIGKILL.

If I remove most of the files from the _site directory, the errors go away and the remainder of the files are served normally.

Steps to reproduce
  1. Run netlify dev
Configuration
[build]
base = "."
publish = "_site/"  # relative to `base` directory
command = "make fx-build"
# NOTE: This should match the Python version in `.python-version`.
environment = { PYTHON_VERSION = "3.13.5" }

[context.deploy-preview]
  # Skip deploy previews for Renovate PRs (branches starting with "renovate/")
  ignore = "sh -c '[ \"${HEAD#renovate/}\" != \"$HEAD\" ]'"

[[headers]]
  for = "/v1/*"
  [headers.values]
    # https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Access-Control-Allow-Origin
    access-control-allow-origin = "*"
Environment
 ➜  npx envinfo --system --binaries --npmPackages netlify-cli --npmGlobalPackages netlify-cli

  System:
    OS: Linux 5.15 Ubuntu 22.04 LTS 22.04 (Jammy Jellyfish)
    CPU: (4) x64 Intel(R) Core(TM) i7-7567U CPU @ 3.50GHz
    Memory: 14.01 GB / 15.51 GB
    Container: Yes
    Shell: 5.1.16 - /bin/bash
  Binaries:
    Node: 22.21.1 - /home/ian/.local/share/nodenv/versions/22.21.1/bin/node
    npm: 10.9.4 - /home/ian/.local/share/nodenv/versions/22.21.1/bin/npm
  npmPackages:
    netlify-cli: 23.13.0 => 23.13.0

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing netlify dev with the reported 302,005 files in _site/, using the shown Netlify configuration. Trace the netlify dev process and its repeated npm subshells, then verify that the server responds without repeated errors or accumulating zombie processes.

Written by the indexing model from the issue text.

Assessment

Tech stack
node.js, shell, typescript
Domain
cli, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.