Unitech / Unitech/pm2

PM2 is not restarting the simplest crashing app possible

Open
#5,160 10 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
JavaScript
Stars
43.3k
Forks
2.7k
PR merge metrics
No merged PRs in 30d

Description

What's going wrong?

PM2 by default should restart my app from what I know. However it doesn't restart the app if error is thrown in top level code, like:

file.js

throw new Error("boom");

How could we reproduce this issue?

Create file as above.
Run pm2 start file.js
Use pm2 list to see that app is not restarting and it's "online"

Supporting information

I first discovered this on my Raspberry, i thought it may be some issue with the OS, Node version or something, but I tested it on my local PC and it's the same. Even old version (4.4.1) is behaving the same, so this must be a long living bug I guess.

I tried changing the code to:

setTimeout(() => {
    throw new Error("g");
}, 100);

and then it works as expected

$ pm2 report

// ON RASPBERRY:
--- PM2 report ----------------------------------------------------------------
Date                 : Mon Aug 30 2021 19:44:23 GMT+0200 (czas środkowoeuropejski letni)
===============================================================================
--- Daemon -------------------------------------------------
pm2d version         : 5.1.1
node version         : 15.9.0
node path            : /home/dzek/.yarn/bin/pm2
argv                 : /home/dzek/.local/node-v15.9.0-linux-armv6l/bin/node,/home/dzek/.config/yarn/global/node_modules/pm2/lib/Daemon.js
argv0                : node
user                 : dzek
uid                  : 1001
gid                  : 1001
uptime               : 27min
===============================================================================
--- CLI ----------------------------------------------------
local pm2            : 5.1.1
node version         : 15.9.0
node path            : /home/dzek/.yarn/bin/pm2
argv                 : /home/dzek/.local/node-v15.9.0-linux-armv6l/bin/node,/home/dzek/.yarn/bin/pm2,report
argv0                : node
user                 : dzek
uid                  : 1001
gid                  : 1001
===============================================================================
--- System info --------------------------------------------
arch                 : arm
platform             : linux
type                 : Linux
cpus                 : ARMv6-compatible processor rev 7 (v6l)
cpus nb              : 1
freemem              : 120193024
totalmem             : 450981888
home                 : /home/dzek
===============================================================================
--- PM2 list -----------------------------------------------
┌────┬────────────────────┬──────────┬──────┬───────────┬──────────┬──────────┐
│ id │ name               │ mode     │ ↺    │ status    │ cpu      │ memory   │
├────┼────────────────────┼──────────┼──────┼───────────┼──────────┼──────────┤
│ 0  │ index              │ fork     │ 0    │ online    │ 2.9%     │ 40.5mb   │
└────┴────────────────────┴──────────┴──────┴───────────┴──────────┴──────────┘
===============================================================================
--- Daemon logs --------------------------------------------
/home/dzek/.pm2/pm2.log last 20 lines:
PM2        | 2021-08-30T19:16:33: PM2 log: ===============================================================================
PM2        | 2021-08-30T19:16:33: PM2 log: --- New PM2 Daemon started ----------------------------------------------------
PM2        | 2021-08-30T19:16:33: PM2 log: Time                 : Mon Aug 30 2021 19:16:33 GMT+0200 (czas środkowoeuropejski letni)
PM2        | 2021-08-30T19:16:33: PM2 log: PM2 version          : 5.1.1
PM2        | 2021-08-30T19:16:33: PM2 log: Node.js version      : 15.9.0
PM2        | 2021-08-30T19:16:33: PM2 log: Current arch         : arm
PM2        | 2021-08-30T19:16:33: PM2 log: PM2 home             : /home/dzek/.pm2
PM2        | 2021-08-30T19:16:33: PM2 log: PM2 PID file         : /home/dzek/.pm2/pm2.pid
PM2        | 2021-08-30T19:16:33: PM2 log: RPC socket file      : /home/dzek/.pm2/rpc.sock
PM2        | 2021-08-30T19:16:33: PM2 log: BUS socket file      : /home/dzek/.pm2/pub.sock
PM2        | 2021-08-30T19:16:33: PM2 log: Application log path : /home/dzek/.pm2/logs
PM2        | 2021-08-30T19:16:33: PM2 log: Worker Interval      : 30000
PM2        | 2021-08-30T19:16:33: PM2 log: Process dump file    : /home/dzek/.pm2/dump.pm2
PM2        | 2021-08-30T19:16:33: PM2 log: Concurrent actions   : 2
PM2        | 2021-08-30T19:16:33: PM2 log: SIGTERM timeout      : 1600
PM2        | 2021-08-30T19:16:33: PM2 log: ===============================================================================
PM2        | 2021-08-30T19:16:34: PM2 log: App [index:0] starting in -fork mode-
PM2        | 2021-08-30T19:16:34: PM2 log: App [index:0] online
$ pm2 report
// ON PC, with the "test fix" applied before generating report

--- PM2 report ----------------------------------------------------------------
Date                 : Mon Aug 30 2021 19:43:31 GMT+0200 (Central European Summer Time)
===============================================================================
--- Daemon -------------------------------------------------
pm2d version         : 4.4.1
node version         : 15.2.0
node path            : /home/dzek/.yarn/bin/pm2
argv                 : /usr/bin/node,/home/dzek/.config/yarn/global/node_modules/pm2/lib/Daemon.js
argv0                : node
user                 : dzek
uid                  : 1000
gid                  : 1000
uptime               : 4min
===============================================================================
--- CLI ----------------------------------------------------
local pm2            : 4.4.1
node version         : 15.2.0
node path            : /home/dzek/.yarn/bin/pm2
argv                 : /usr/bin/node,/home/dzek/.yarn/bin/pm2,report
argv0                : node
user                 : dzek
uid                  : 1000
gid                  : 1000
===============================================================================
--- System info --------------------------------------------
arch                 : x64
platform             : linux
type                 : Linux
cpus                 : Intel(R) Core(TM) i7-6700HQ CPU @ 2.60GHz
cpus nb              : 8
freemem              : 1936064512
totalmem             : 12561264640
home                 : /home/dzek
===============================================================================
--- PM2 list -----------------------------------------------
┌─────┬──────────┬─────────────┬─────────┬─────────┬──────────┬────────┬──────┬───────────┬──────────┬──────────┬──────────┬──────────┐
│ id  │ name     │ namespace   │ version │ mode    │ pid      │ uptime │ ↺    │ status    │ cpu      │ mem      │ user     │ watching │
├─────┼──────────┼─────────────┼─────────┼─────────┼──────────┼────────┼──────┼───────────┼──────────┼──────────┼──────────┼──────────┤
│ 0   │ index    │ default     │ 0.0.2   │ fork    │ 14599    │ 4m     │ 0    │ online    │ 0.2%     │ 50.7mb   │ dzek     │ disabled │
│ 1   │ test     │ default     │ 0.0.2   │ fork    │ 0        │ 0      │ 16   │ errored   │ 0%       │ 0b       │ dzek     │ disabled │
└─────┴──────────┴─────────────┴─────────┴─────────┴──────────┴────────┴──────┴───────────┴──────────┴──────────┴──────────┴──────────┘
===============================================================================
--- Daemon logs --------------------------------------------
/home/dzek/.pm2/pm2.log last 20 lines:
PM2        | 2021-08-30T19:42:37: PM2 log: App [test:1] exited with code [1] via signal [SIGINT]
PM2        | 2021-08-30T19:42:37: PM2 log: App [test:1] starting in -fork mode-
PM2        | 2021-08-30T19:42:37: PM2 log: App [test:1] online
PM2        | 2021-08-30T19:42:37: PM2 log: App [test:1] exited with code [1] via signal [SIGINT]
PM2        | 2021-08-30T19:42:37: PM2 log: App [test:1] starting in -fork mode-
PM2        | 2021-08-30T19:42:37: PM2 log: App [test:1] online
PM2        | 2021-08-30T19:42:37: PM2 log: App [test:1] exited with code [1] via signal [SIGINT]
PM2        | 2021-08-30T19:42:37: PM2 log: App [test:1] starting in -fork mode-
PM2        | 2021-08-30T19:42:37: PM2 log: App [test:1] online
PM2        | 2021-08-30T19:42:38: PM2 log: App [test:1] exited with code [1] via signal [SIGINT]
PM2        | 2021-08-30T19:42:38: PM2 log: App [test:1] starting in -fork mode-
PM2        | 2021-08-30T19:42:38: PM2 log: App [test:1] online
PM2        | 2021-08-30T19:42:38: PM2 log: App [test:1] exited with code [1] via signal [SIGINT]
PM2        | 2021-08-30T19:42:38: PM2 log: App [test:1] starting in -fork mode-
PM2        | 2021-08-30T19:42:38: PM2 log: App [test:1] online
PM2        | 2021-08-30T19:42:38: PM2 log: App [test:1] exited with code [1] via signal [SIGINT]
PM2        | 2021-08-30T19:42:38: PM2 log: App [test:1] starting in -fork mode-
PM2        | 2021-08-30T19:42:38: PM2 log: App [test:1] online
PM2        | 2021-08-30T19:42:39: PM2 log: App [test:1] exited with code [1] via signal [SIGINT]
PM2        | 2021-08-30T19:42:39: PM2 log: Script /Projects/o-0/services/doorbell/app/dist/test.js had too many unstable restarts (16). Stopped. "errored"

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the issue with the shown file.js, then run pm2 start file.js and pm2 list. Compare the top-level throw with the setTimeout case and trace the process lifecycle handling. Done means a top-level crash is detected and restarted consistently, with the resulting status and restart count behaving as expected.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, node.js
Domain
cli, devops
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.