Exception while invoking method loadLocale 'Moment locale not found'

Description:

After running for a few hours, RocketChat stops functioning. Connections towards the Mongo database time out and users are not able to send and recieve messages.

Steps to reproduce:

1. Start RocketChat
2. Wait a few hours

Expected behavior:

It should work

Actual behavior:

LocalStore: store created at 
LocalStore: store created at 
LocalStore: store created at 
{"level":51,"time":"2021-11-25T08:06:17.938Z","pid":8,"hostname":"2bda0ca6075e","name":"Migrations","msg":"Not migrating, already at version 243"}
ufs: temp directory created at "/tmp/ufs"
Loaded the Apps Framework and loaded a total of 1 Apps!
+---------------------------------------------------+
|                   SERVER RUNNING                  |
+---------------------------------------------------+
|                                                   |
|  Rocket.Chat Version: 4.1.2                       |
|       NodeJS Version: 12.22.1 - x64               |
|      MongoDB Version: 5.0.4                       |
|       MongoDB Engine: wiredTiger                  |
|             Platform: linux                       |
|         Process Port: 3000                        |
|             Site URL: https://website.url         |
|     ReplicaSet OpLog: Enabled                     |
|          Commit Hash: 2aba8d2d82                  |
|        Commit Branch: HEAD                        |
|                                                   |
+---------------------------------------------------+
{"level":50,"time":"2021-11-25T08:09:57.073Z","pid":8,"hostname":"2bda0ca6075e","name":"System","msg":"Exception while invoking method loadLocale 'Moment locale not found: 2 [Moment locale not found: 2 [moment-locale-not-found]]'"}
{"level":50,"time":"2021-11-25T08:20:57.536Z","pid":8,"hostname":"2bda0ca6075e","name":"System","msg":"Exception while invoking method login 'User not found [403]'"}
{"level":50,"time":"2021-11-25T08:21:05.310Z","pid":8,"hostname":"2bda0ca6075e","name":"System","msg":"Exception while invoking method login 'User not found [403]'"}
{"level":50,"time":"2021-11-25T08:48:31.308Z","pid":8,"hostname":"2bda0ca6075e","name":"System","msg":"Exception while invoking method loadLocale 'Moment locale not found: 2 [Moment locale not found: 2 [moment-locale-not-found]]'"}
{"level":50,"time":"2021-11-25T09:09:24.111Z","pid":8,"hostname":"2bda0ca6075e","name":"System","msg":"Exception while invoking method loadLocale 'Moment locale not found: 2 [Moment locale not found: 2 [moment-locale-not-found]]'"}
{"level":50,"time":"2021-11-25T09:55:03.372Z","pid":8,"hostname":"2bda0ca6075e","name":"System","msg":"Exception while invoking method login 'User not found [403]'"}
{"level":40,"time":"2021-11-25T10:10:59.704Z","pid":8,"hostname":"2bda0ca6075e","name":"Push","msg":"Error sending push to gateway (not authorized) {\"statusCode\":401,\"content\":\"{\\\"code\\\":121,\\\"error\\\":\\\"rocket.chat cloud registration required\\\",\\\"requestId\\\":\\\"117628d0-3851-454f-a6d9-8911e9a79fbb\\\",\\\"status\\\":401}\",\"headers\":{\"access-control-allow-headers\":\"Content-Type, Authorization, Content-Length, X-Requested-With\",\"access-control-allow-methods\":\"GET, PUT, POST, DELETE, OPTIONS\",\"access-control-allow-origin\":\"*\",\"access-control-expose-headers\":\"Content-Type, Authorization, Cache-Control, Expires, Pragma, X-powered-by\",\"cache-control\":\"private, no-cache, no-store, must-revalidate\",\"content-length\":\"126\",\"content-type\":\"application/json; charset=utf-8\",\"date\":\"Thu, 25 Nov 2021 10:10:59 GMT\",\"expires\":\"-1\",\"pragma\":\"no-cache\",\"x-powered-by\":\"Rocket Fuel and Rocketeers\",\"connection\":\"close\"},\"data\":{\"code\":121,\"error\":\"rocket.chat cloud registration required\",\"requestId\":\"117628d0-3851-454f-a6d9-8911e9a79fbb\",\"status\":401}}"}
{"level":40,"time":"2021-11-25T10:10:59.711Z","pid":8,"hostname":"2bda0ca6075e","name":"Push","msg":"Error sending push to gateway (not authorized) {\"statusCode\":401,\"content\":\"{\\\"code\\\":121,\\\"error\\\":\\\"rocket.chat cloud registration required\\\",\\\"requestId\\\":\\\"4adeb4a1-6439-46fe-a7ac-0d7652cea229\\\",\\\"status\\\":401}\",\"headers\":{\"access-control-allow-headers\":\"Content-Type, Authorization, Content-Length, X-Requested-With\",\"access-control-allow-methods\":\"GET, PUT, POST, DELETE, OPTIONS\",\"access-control-allow-origin\":\"*\",\"access-control-expose-headers\":\"Content-Type, Authorization, Cache-Control, Expires, Pragma, X-powered-by\",\"cache-control\":\"private, no-cache, no-store, must-revalidate\",\"content-length\":\"126\",\"content-type\":\"application/json; charset=utf-8\",\"date\":\"Thu, 25 Nov 2021 10:10:59 GMT\",\"expires\":\"-1\",\"pragma\":\"no-cache\",\"x-powered-by\":\"Rocket Fuel and Rocketeers\",\"connection\":\"close\"},\"data\":{\"code\":121,\"error\":\"rocket.chat cloud registration required\",\"requestId\":\"4adeb4a1-6439-46fe-a7ac-0d7652cea229\",\"status\":401}}"}
{"level":40,"time":"2021-11-25T10:10:59.712Z","pid":8,"hostname":"2bda0ca6075e","name":"Push","msg":"Error sending push to gateway (not authorized) {\"statusCode\":401,\"content\":\"{\\\"code\\\":121,\\\"error\\\":\\\"rocket.chat cloud registration required\\\",\\\"requestId\\\":\\\"1feb8bba-4c8c-457d-923b-fbeb35ec33e1\\\",\\\"status\\\":401}\",\"headers\":{\"access-control-allow-headers\":\"Content-Type, Authorization, Content-Length, X-Requested-With\",\"access-control-allow-methods\":\"GET, PUT, POST, DELETE, OPTIONS\",\"access-control-allow-origin\":\"*\",\"access-control-expose-headers\":\"Content-Type, Authorization, Cache-Control, Expires, Pragma, X-powered-by\",\"cache-control\":\"private, no-cache, no-store, must-revalidate\",\"content-length\":\"126\",\"content-type\":\"application/json; charset=utf-8\",\"date\":\"Thu, 25 Nov 2021 10:10:59 GMT\",\"expires\":\"-1\",\"pragma\":\"no-cache\",\"x-powered-by\":\"Rocket Fuel and Rocketeers\",\"connection\":\"close\"},\"data\":{\"code\":121,\"error\":\"rocket.chat cloud registration required\",\"requestId\":\"1feb8bba-4c8c-457d-923b-fbeb35ec33e1\",\"status\":401}}"}
{"level":50,"time":"2021-11-25T11:15:30.052Z","pid":8,"hostname":"2bda0ca6075e","name":"System","msg":"Exception while invoking method loadLocale 'Moment locale not found: 2 [Moment locale not found: 2 [moment-locale-not-found]]'"}
{"level":50,"time":"2021-11-25T11:35:43.823Z","pid":8,"hostname":"2bda0ca6075e","name":"System","msg":"Exception while invoking method login 'User not found [403]'"}
{"level":50,"time":"2021-11-25T12:00:34.296Z","pid":8,"hostname":"2bda0ca6075e","name":"System","err":{"type":"Error","message":"ESOCKETTIMEDOUT","stack":"Error: ESOCKETTIMEDOUT\n    at ClientRequest.<anonymous> (/app/bundle/programs/server/npm/node_modules/meteor/http/node_modules/request/request.js:816:19)\n    at Object.onceWrapper (events.js:420:28)\n    at ClientRequest.emit (events.js:314:20)\n    at ClientRequest.EventEmitter.emit (domain.js:483:12)\n    at TLSSocket.emitRequestTimeout (_http_client.js:715:9)\n    at Object.onceWrapper (events.js:420:28)\n    at TLSSocket.emit (events.js:326:22)\n    at TLSSocket.EventEmitter.emit (domain.js:483:12)\n    at TLSSocket.Socket._onTimeout (net.js:483:8)\n    at listOnTimeout (internal/timers.js:554:17)\n    at processTimers (internal/timers.js:497:7)","code":"ESOCKETTIMEDOUT","connect":false},"msg":"ESOCKETTIMEDOUT"}
Exception in setInterval callback: MongoServerSelectionError: connection timed out
    at Timeout._onTimeout (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/core/sdam/topology.js:437:30)
    at listOnTimeout (internal/timers.js:554:17)
    at processTimers (internal/timers.js:497:7) {
  reason: TopologyDescription {
    type: 'Single',
    setName: null,
    maxSetVersion: null,
    maxElectionId: null,
    servers: Map { 'mongo:27017' => [ServerDescription] },
    stale: false,
    compatible: true,
    compatibilityError: null,
    logicalSessionTimeoutMinutes: null,
    heartbeatFrequencyMS: 10000,
    localThresholdMS: 15,
    commonWireVersion: 13
  }
}

Server Setup Information:

• Version of Rocket.Chat Server: 4.1.2
• Operating System: ubuntu
• Deployment Method: docker
• Number of Running Instances: 1
• DB Replicaset Oplog: enabled
• NodeJS Version: 12.22.1.
• MongoDB Version: 5.0.4

Does anybody have any idea how to solve this?

Start with the loadLocale method and the logged MongoDB connection-timeout stack, including the Meteor request path shown in the trace. Reproduce the long-running Docker deployment if possible, then determine the cause of the missing Moment locale and database timeout; done means the service remains usable without these errors.