Unitech / Unitech/pm2

Sudden PM2 core dump. Asking for assistance in finding the root cause, if possible.

Open
#5,787 4 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
JavaScript
Stars
43.3k
Forks
2.7k
PR merge metrics
No merged PRs in 30d

Description

What's going wrong?

host syslog extract:
Mar 15 03:30:10 Center1 systemd[1]: Started Process Core Dump (PID 3079673/UID 0).
Mar 15 03:30:12 Center1 systemd-coredump[3079674]: Core file was truncated to 2147483648 bytes.
Mar 15 03:30:17 Center1 systemd-coredump[3079674]: Process 391241 (PM2 v5.3.0: God) of user 1000 dumped core.#012#012Stack trace of thread 391241:#012#0 0x00007f204686a00b n/a (n/a + 0x0)
Mar 15 03:30:17 Center1 systemd[1]: systemd-coredump@1-3079673-0.service: Succeeded.
Mar 15 03:30:17 Center1 systemd[1]: pm2-user.service: New main PID 391249 does not belong to service, and PID file is not owned by root. Refusing.
Mar 15 03:30:17 Center1 systemd[1]: pm2-user.service: Main process exited, code=dumped, status=6/ABRT
Mar 15 03:30:17 Center1 systemd[1]: pm2-user.service: Failed with result 'core-dump'.

How could we reproduce this issue?

Not sure if this can be easily replicated, this PM2 worked great for a long time, but perhaps the information contained in the core dump could be of interest to developers?
The core dump file is available, but its size is around 299 MB.
Ready to upload/send it wherever you say, if necessary.
Please close this case if it is not relevant to you. I apologize for any inconvenience.

Supporting information

Linux Mint 20.3 Una
Linux Center1 5.4.0-113-generic #127-Ubuntu SMP Wed May 18 14:30:56 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

PM2 report
Date : Mon Mar 18 2024 14:16:49 GMT+0200 (Eastern European Standard Time)

Daemon
pm2d version : 5.3.0
node version : 10.19.0
node path : not found
argv : /usr/bin/node,/usr/local/lib/node_modules/pm2/lib/Daemon.js
argv0 : node
user : user
uid : 1000
gid : 1000
uptime : 4747min

--- CLI ----------------------------------------------------
local pm2 : 5.3.0
node version : 10.19.0
node path : /usr/local/bin/pm2
argv : /usr/bin/node,/usr/local/bin/pm2,report
argv0 : node
user : user
uid : 1000
gid : 1000

--- System info --------------------------------------------
arch : x64
platform : linux
type : Linux
cpus : Intel(R) Xeon(R) E-2246G CPU @ 3.60GHz
cpus nb : 12
freemem : 20717719552
totalmem : 33243213824
home : /home/user

--- PM2 list -----------------------------------------------
┌────┬────────────────────┬─────────────┬─────────┬─────────┬──────────┬────────┬──────┬───────────┬──────────┬──────────┬──────────┬──────────┐
│ id │ name │ namespace │ version │ mode │ pid │ uptime │ ↺ │ status │ cpu │ mem │ user │ watching │
├────┼────────────────────┼─────────────┼─────────┼─────────┼──────────┼────────┼──────┼───────────┼──────────┼──────────┼──────────┼──────────┤
│ 2 │ db │ default │ N/A │ fork │ 10845 │ 3D │ 0 │ online │ 0.7% │ 68.8mb │ user │ disabled │
│ 3 │ line │ default │ N/A │ fork │ 10901 │ 3D │ 0 │ online │ 38.1% │ 88.2mb │ user │ disabled │
│ 4 │ linecleanup │ default │ N/A │ fork │ N/A │ 0 │ 0 │ stopped │ 0% │ 0b │ user │ disabled │
│ 1 │ registry_server │ default │ N/A │ fork │ 10821 │ 3D │ 5 │ online │ 1.1% │ 20.2mb │ user │ disabled │
└────┴────────────────────┴─────────────┴─────────┴─────────┴──────────┴────────┴──────┴───────────┴──────────┴──────────┴──────────┴──────────┘
Module
┌────┬──────────────────────────────┬───────────────┬──────────┬──────────┬──────┬──────────┬──────────┬──────────┐
│ id │ module │ version │ pid │ status │ ↺ │ cpu │ mem │ user │
├────┼──────────────────────────────┼───────────────┼──────────┼──────────┼──────┼──────────┼──────────┼──────────┤
│ 0 │ pm2-logrotate │ 2.7.0 │ 10694 │ online │ 0 │ 0.3% │ 100.0mb │ user │
└────┴──────────────────────────────┴───────────────┴──────────┴──────────┴──────┴──────────┴──────────┴──────────┘

--- Daemon logs --------------------------------------------
/home/user/.pm2/pm2.log last 20 lines:
PM2 | 2024-03-15T07:09:07: PM2 log: Application log path : /home/user/.pm2/logs
PM2 | 2024-03-15T07:09:07: PM2 log: Worker Interval : 30000
PM2 | 2024-03-15T07:09:07: PM2 log: Process dump file : /home/user/.pm2/dump.pm2
PM2 | 2024-03-15T07:09:07: PM2 log: Concurrent actions : 2
PM2 | 2024-03-15T07:09:07: PM2 log: SIGTERM timeout : 1600
PM2 | 2024-03-15T07:09:07: PM2 log:

PM2 | 2024-03-15T07:09:08: PM2 log: App [pm2-logrotate:0] starting in -fork mode-
PM2 | 2024-03-15T07:09:08: PM2 log: App [pm2-logrotate:0] online
PM2 | 2024-03-15T07:09:12: PM2 log: App [registry_server:1] starting in -fork mode-
PM2 | 2024-03-15T07:09:12: PM2 log: App [registry_server:1] online
PM2 | 2024-03-15T07:09:13: PM2 log: App [db:2] starting in -fork mode-
PM2 | 2024-03-15T07:09:13: PM2 log: App [db:2] online
PM2 | 2024-03-15T07:09:14: PM2 log: App [line:3] starting in -fork mode-
PM2 | 2024-03-15T07:09:14: PM2 log: App [line:3] online
PM2 | 2024-03-16T07:09:08: PM2 log: [PM2] This PM2 is not UP TO DATE
PM2 | 2024-03-16T07:09:08: PM2 log: [PM2] Upgrade to version 5.3.1
PM2 | 2024-03-17T07:09:08: PM2 log: [PM2] This PM2 is not UP TO DATE
PM2 | 2024-03-17T07:09:08: PM2 log: [PM2] Upgrade to version 5.3.1
PM2 | 2024-03-18T07:09:08: PM2 log: [PM2] This PM2 is not UP TO DATE
PM2 | 2024-03-18T07:09:08: PM2 log: [PM2] Upgrade to version 5.3.1

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the supplied host syslog extract and the available 299 MB core dump, then compare them with the PM2 report and daemon logs for PM2 5.3.0 on Node.js 10.19.0. No source file, test, reproduction, or expected code change is named; completion would require identifying a reproducible root cause and documenting the diagnosis or the additional evidence needed.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, linux, node.js
Domain
cli, devops
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.