Unitech / Unitech/pm2

pm2 reload not working in "fork" or "fork_mode" with multiple instances

Open
#5,580 3 comments 3 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
JavaScript
Stars
43.3k
Forks
2.7k
PR merge metrics
No merged PRs in 30d

Description

What's going wrong?

pm2 reload is not possible with 'fork_mode'.

//test.config.js

module.exports = {
  apps : [{
          name: "api-dev",
          script: "/home/developer/api-dev/source/dist/src/bin/www.js",
          instances: 4,
          exec_mode: "fork_mode",   //OR "fork"
          instance_var: "PORT",
         //wait_ready: true,
         //max_restarts: 15,
         //autostart: true,
         //restart_delay: 4000,
          watch: false,
         //force: true,
         //max_memory_restart: "2G",
          env: {
              "DEBUG": "test-backend:*",
              "NODE_ENV": "development",
              "GOOGLE_APPLICATION_CREDENTIALS": "XXXXXXXXX",
              "ELASTIC_ID": "XXXXXXXXX",
              "ELASTIC_USERNAME": "XXXXXXXX",
              "ELASTIC_PASSWORD": "XXXXXXXXXX",
              "PORT": 3002,
              "EXPRESS_SESSION_SECRET": "XXXXXXX"
          }
}]
}

I tried every possible fix out there on GitHub or pm2 docs, but as soon as I hit reload command only 1 process starts and other keep on restarting with an error.
It's working with pm2 start..* OR stop && delete && start but I can't always delete and start the process as our deploy pipeline runs 5-10 times a day. Upgraded PM2 to the Latest version 5.3.0 but not fixed.
We want to use fork_mode over cluster because we're using sticky sessions and don't want to change implementations.

How could we reproduce this issue?

pm2 reload test.config.js --only api-dev --update-env

Supporting information

--- PM2 report ----------------------------------------------------------------
Date                 : Mon Apr 03 2023 12:49:55 GMT+0200 (Central European Summer Time)
===============================================================================
--- Daemon -------------------------------------------------
pm2d version         : 5.3.0
node version         : 16.14.0
node path            : /home/developer/.nvm/versions/node/v16.14.0/bin/pm2
argv                 : /home/developer/.nvm/versions/node/v16.14.0/bin/node,/home/developer/.nvm/versions/node/v16.14.0/lib/node_modules/pm2/lib/Daemon.js
argv0                : node
user                 : developer
uid                  : 1000
gid                  : 1000
uptime               : 4087min
===============================================================================
--- CLI ----------------------------------------------------
local pm2            : 5.3.0
node version         : 16.14.0
node path            : /home/developer/.nvm/versions/node/v16.14.0/bin/pm2
argv                 : /home/developer/.nvm/versions/node/v16.14.0/bin/node,/home/developer/.nvm/versions/node/v16.14.0/bin/pm2,report
argv0                : node
user                 : developer
uid                  : 1000
gid                  : 1000
===============================================================================
--- System info --------------------------------------------
arch                 : x64
platform             : linux
type                 : Linux
cpus                 : QEMU Virtual CPU version 2.5+
cpus nb              : 4
freemem              : 6089424896
totalmem             : 8365875200
home                 : /home/developer
===============================================================================
--- PM2 list -----------------------------------------------
┌────┬──────────────────┬─────────────┬─────────┬─────────┬──────────┬────────┬──────┬───────────┬──────────┬──────────┬──────────┬──────────┐
│ id │ name             │ namespace   │ version │ mode    │ pid      │ uptime │ ↺    │ status    │ cpu      │ mem      │ user     │ watching │
├────┼──────────────────┼─────────────┼─────────┼─────────┼──────────┼────────┼──────┼───────────┼──────────┼──────────┼──────────┼──────────┤
│ 81 │ api-dev          │ default     │ 0.0.0   │ fork    │ 20759    │ 22s    │ 6    │ online    │ 0%       │ 292.1mb  │ dev… │ disabled │
│ 82 │ api-dev          │ default     │ 0.0.0   │ fork    │ 21268    │ 3s     │ 8    │ online    │ 100%     │ 156.6mb  │ dev… │ disabled │
│ 83 │ api-dev          │ default     │ 0.0.0   │ fork    │ 21252    │ 3s     │ 3    │ online    │ 100%     │ 161.0mb  │ dev… │ disabled │
│ 0  │ api-staging      │ default     │ 0.0.0   │ fork    │ 31592    │ 20h    │ 2    │ online    │ 0%       │ 120.5mb  │ dev… │ disabled │
└────┴──────────────────┴─────────────┴─────────┴─────────┴──────────┴────────┴──────┴───────────┴──────────┴──────────┴──────────┴──────────┘
Module
┌────┬──────────────────────────────┬───────────────┬──────────┬──────────┬──────┬──────────┬──────────┬──────────┐
│ id │ module                       │ version       │ pid      │ status   │ ↺    │ cpu      │ mem      │ user     │
├────┼──────────────────────────────┼───────────────┼──────────┼──────────┼──────┼──────────┼──────────┼──────────┤
│ 4  │ pm2-logrotate                │ 2.7.0         │ 29577    │ online   │ 0    │ 0%       │ 61.0mb   │ dev… │
└────┴──────────────────────────────┴───────────────┴──────────┴──────────┴──────┴──────────┴──────────┴──────────┘
host metrics | cpu: 54.5% | mem free: 74.1% | eth0: ⇓ 0.002mb/s ⇑ 0.007mb/s | disk: ⇓ 0.096mb/s ⇑ 0mb/s |
[PM2][WARN] Current process list is not synchronized with saved list. Type 'pm2 save' to synchronize.
===============================================================================
--- Daemon logs --------------------------------------------
/home/developer/.pm2/pm2.log last 20 lines:
PM2        | 2023-04-03T12:49:32: PM2 log: Stopping app:api-dev id:83
PM2        | 2023-04-03T12:49:32: PM2 log: pid=20608 msg=process killed
PM2        | 2023-04-03T12:49:32: PM2 log: App [api-dev:82] starting in -fork mode-
PM2        | 2023-04-03T12:49:32: PM2 log: App [api-dev:82] online
PM2        | 2023-04-03T12:49:32: PM2 log: App [api-dev:83] exited with code [0] via signal [SIGINT]
PM2        | 2023-04-03T12:49:32: PM2 log: pid=19670 msg=process killed
PM2        | 2023-04-03T12:49:32: PM2 log: App [api-dev:83] starting in -fork mode-
PM2        | 2023-04-03T12:49:32: PM2 log: App [api-dev:83] online
PM2        | 2023-04-03T12:49:43: PM2 log: App [api-dev:82] exited with code [1] via signal [SIGINT]
PM2        | 2023-04-03T12:49:43: PM2 log: App [api-dev:82] starting in -fork mode-
PM2        | 2023-04-03T12:49:43: PM2 log: App [api-dev:82] online
PM2        | 2023-04-03T12:49:43: PM2 log: App [api-dev:83] exited with code [1] via signal [SIGINT]
PM2        | 2023-04-03T12:49:43: PM2 log: App [api-dev:83] starting in -fork mode-
PM2        | 2023-04-03T12:49:43: PM2 log: App [api-dev:83] online
PM2        | 2023-04-03T12:49:51: PM2 log: App [api-dev:83] exited with code [1] via signal [SIGINT]
PM2        | 2023-04-03T12:49:51: PM2 log: App [api-dev:83] starting in -fork mode-
PM2        | 2023-04-03T12:49:51: PM2 log: App [api-dev:83] online
PM2        | 2023-04-03T12:49:51: PM2 log: App [api-dev:82] exited with code [1] via signal [SIGINT]
PM2        | 2023-04-03T12:49:51: PM2 log: App [api-dev:82] starting in -fork mode-
PM2        | 2023-04-03T12:49:51: PM2 log: App [api-dev:82] online

stackoverflow

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with test.config.js and reproduce the failure using pm2 reload test.config.js --only api-dev --update-env; inspect the reload behavior for fork mode with four instances and the daemon log output. Done means all configured fork-mode instances reload successfully without entering a restart loop, while preserving the existing sticky-session setup.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, node.js
Domain
cli, devops
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.