RocketChat / RocketChat/Rocket.Chat

FatalProcessOutOfMemory

Open
#29,875 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
46.1k
Forks
13.9k
Avg merge
3d 3h
Merged PRs (30d)
130

Description

Hello everyone,

I’m having an issue here, and I’m not sure if anyone has experienced this before. After updating to version 6.2.10, the rocketchat service has been causing problems at least twice a day. The service stops, and I have to restart it to get it back to normal.
These are the logs I managed to collect when it stopped:

systemctl status rocketchat.service

rocketchat.service - The Rocket.Chat server
     Loaded: loaded (/lib/systemd/system/rocketchat.service; enabled; vendor preset: enabled)
     Active: failed (Result: core-dump) since Tue 2023-07-18 10:09:38 -03; 28min ago
    Process: 881 ExecStart=/usr/local/bin/node /opt/Rocket.Chat/main.js (code=dumped, signal=ABRT)
   Main PID: 881 (code=dumped, signal=ABRT)

Jul 20 10:09:38 rocketchat.damapel.com.br rocketchat[881]:  4: 0xb9a327 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/usr/local/bin/node]
Jul 20 06:09:38 rocketchat.damapel.com.br rocketchat[881]:  5: 0xd3a735  [/usr/local/bin/node]
Jul 20 10:09:38 rocketchat.damapel.com.br rocketchat[881]:  6: 0xd49a74  [/usr/local/bin/node]
Jul 20 10:09:38 rocketchat.damapel.com.br rocketchat[881]:  7: 0xd49d25 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/usr/l>
Jul 20 10:09:38 rocketchat.damapel.com.br rocketchat[881]:  8: 0xd4c6ec v8::internal::Heap::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::interna>
Jul 20 10:09:38 rocketchat.damapel.com.br rocketchat[881]:  9: 0xd1bd8b v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationType, v8::internal::AllocationOrigin) [/usr/local/bin/node]
Jul 20 10:09:38 rocketchat.damapel.com.br rocketchat[881]: 10: 0x104c56f v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [/usr/local/bin/node]
Jul 20 10:09:38 rocketchat.damapel.com.br rocketchat[881]: 11: 0x13a6259  [/usr/local/bin/node]
Jul 20 10:09:38 rocketchat.damapel.com.br systemd[1]: rocketchat.service: Main process exited, code=dumped, status=6/ABRT
Jul 20 10:09:38 rocketchat.damapel.com.br systemd[1]: rocketchat.service: Failed with result 'core-dump'.

tail -n 100 /var/log/syslog

Jul 20 10:09:23 rocketchat rocketchat[281186]: =================================
Jul 20 10:09:23 rocketchat rocketchat[281186]: === UnHandledPromiseRejection ===
Jul 20 10:09:23 rocketchat rocketchat[281186]: RangeError: Out of memory
Jul 20 10:09:23 rocketchat rocketchat[281186]:     at Fiber.<anonymous> (/opt/Rocket.Chat/programs/server/node_modules/fibers/fibers.js:92:16)
Jul 20 10:09:23 rocketchat rocketchat[281186]:     at Fiber.run (packages/rocketchat:mongo-config/server/index.js:32:13)
Jul 20 10:09:23 rocketchat rocketchat[281186]:     at FibersContextStore.run (/opt/Rocket.Chat/programs/server/npm/node_modules/@rocket.chat/core-services/dist/lib/ContextStore.js:27:12)
Jul 20 10:09:23 rocketchat rocketchat[281186]:     at LocalBroker.call (/opt/Rocket.Chat/programs/server/npm/node_modules/@rocket.chat/core-services/dist/LocalBroker.js:14:51)
Jul 20 10:09:23 rocketchat rocketchat[281186]:     at Api.call (/opt/Rocket.Chat/programs/server/npm/node_modules/@rocket.chat/core-services/dist/lib/Api.js:30:29)
Jul 20 10:09:23 rocketchat rocketchat[281186]:     at Proxy.<anonymous> (/opt/Rocket.Chat/programs/server/npm/node_modules/@rocket.chat/core-services/dist/lib/proxify.js:7:95)
Jul 20 10:09:23 rocketchat rocketchat[281186]:     at server/modules/listeners/listeners.module.ts:242:45
Jul 20 10:09:23 rocketchat rocketchat[281186]:     at /opt/Rocket.Chat/programs/server/npm/node_modules/meteor/promise/node_modules/meteor-promise/fiber_pool.js:43:40
Jul 20 10:09:23 rocketchat rocketchat[281186]:  => awaited here:
Jul 20 10:09:23 rocketchat rocketchat[281186]:     at Function.Promise.await (/opt/Rocket.Chat/programs/server/npm/node_modules/meteor/promise/node_modules/meteor-promise/promise_server.js:56:12)
Jul 20 10:09:23 rocketchat rocketchat[281186]:     at server/modules/listeners/listeners.module.ts:242:17
Jul 20 10:09:23 rocketchat rocketchat[281186]:     at /opt/Rocket.Chat/programs/server/npm/node_modules/meteor/promise/node_modules/meteor-promise/fiber_pool.js:43:40
Jul 20 10:09:23 rocketchat rocketchat[281186]: ---------------------------------
Jul 20 10:09:23 rocketchat rocketchat[281186]: Errors like this can cause oplog processing errors.
Jul 20 10:09:23 rocketchat rocketchat[281186]: Setting EXIT_UNHANDLEDPROMISEREJECTION will cause the process to exit allowing your service to automatically restart the process
Jul 20 10:09:23 rocketchat rocketchat[281186]: Future node.js versions will automatically exit the process
Jul 20 10:09:23 rocketchat rocketchat[281186]: =================================

Server Setup Information
Version of Rocket.Chat Server: 6.2.10
Operating System: Ubuntu Server 20.04
Deployment Method: Manual
NodeJS Version: 14.0.0
MongoDB Version: 4.4.22
Proxy: No
Firewalls involved: No

Any additional Information
Virtualized on Proxmox
image

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with server/modules/listeners/listeners.module.ts at line 242 and the reported Node.js fatal out-of-memory and unhandled-promise-rejection logs. Reproduce or investigate the failure using Rocket.Chat 6.2.10, Node.js 14.0.0, MongoDB 4.4.22, and Ubuntu 20.04. Done requires identifying the cause and demonstrating a tested fix or clearly documented resolution.

Written by the indexing model from the issue text.

Assessment

Tech stack
mongodb, node.js
Domain
backend, databases
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.