RocketChat / RocketChat/Rocket.Chat
Rocketchat unstable after mongoDB restart (docker-composer v3 depends_on issue?)
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 46.1k
- Forks
- 13.9k
- Avg merge
- 3d 3h
- Merged PRs (30d)
- 130
Description
Description:
After a mongoDB restart (docker recreate), rocketchat becomes unstable.
Not sure if depends_on in docker-composer v2 forced a rocketchat restart.
In that case, how to handle that in newer docker-composer syntax versions?
Steps to reproduce:
- recreate the mongoDB container
- see some clients not being able to send all messages (feels random)
- restart rocketchat container
Expected behavior:
Rocketchat fully recovers after a mongoDB restart/recreate or docker forces a Rocketchat restart in such case.
Actual behavior:
Rocketchat becomes unstable, and stays that way till it's restarted (left it hours)
Server Setup Information:
- Version of Rocket.Chat Server: 4.6.3
- Operating System: official docker image
- Deployment Method: docker
- Number of Running Instances: 1
- DB Replicaset Oplog: 1
- NodeJS Version: 14.18.3
- MongoDB Version: 5.0.8
Additional context
Reading online this issue might not be there with the older docker-composer v2 "depends_on" syntax behavior.
I don't specify any version on top of my docker-composer file now.
Relevant logs:
Errors like this can cause oplog processing errors.
Setting EXIT_UNHANDLEDPROMISEREJECTION will cause the process to exit allowing your service to automatically restart the process
Future node.js versions will automatically exit the process
=================================
Exception in setInterval callback: MongoServerSelectionError: getaddrinfo ENOTFOUND mongo
at Timeout._onTimeout (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/core/sdam/topology.js:437:30)
at listOnTimeout (internal/timers.js:557:17)
at processTimers (internal/timers.js:500:7) {
reason: TopologyDescription {
type: 'Single',
setName: null,
maxSetVersion: null,
maxElectionId: null,
servers: Map(1) { 'mongo:27017' => [ServerDescription] },
stale: false,
compatible: true,
compatibilityError: null,
logicalSessionTimeoutMinutes: null,
heartbeatFrequencyMS: 10000,
localThresholdMS: 15,
commonWireVersion: 13
}
}
{"level":50,"time":"2022-05-26T14:46:12.417Z","pid":10,"hostname":"3d65f933ba3f","name":"LivechatEnterprise","section":"Queue","msg":"Error processing queue public","err":{"type":"MongoServerSelectionError","message":"getaddrinfo ENOTFOUND mongo","stack":"MongoServerSelectionError: getaddrinfo ENOTFOUND mongo\n at Timeout._onTimeout (/app/bundle/programs/server/npm/node_modules/meteor/npm-mongo/node_modules/mongodb/lib/core/sdam/topology.js:437:30)\n at listOnTimeout (internal/timers.js:557:17)\n at processTimers (internal/timers.js:500:7)\n => awaited here:\n at Function.Promise.await (/app/bundle/programs/server/npm/node_modules/meteor/promise/node_modules/meteor-promise/promise_server.js:56:12)\n at app/models/server/raw/OmnichannelQueue.ts:43:15\n at /app/bundle/programs/server/npm/node_modules/meteor/promise/node_modules/meteor-promise/fiber_pool.js:43:40\n => awaited here:\n at Function.Promise.await (/app/bundle/programs/server/npm/node_modules/meteor/promise/node_modules/meteor-promise/promise_server.js:56:12)\n at ee/app/livechat-enterprise/server/lib/LivechatEnterprise.js:269:4\n at /app/bundle/programs/server/npm/node_modules/meteor/promise/node_modules/meteor-promise/fiber_pool.js:43:40","name":"MongoServerSelectionError","reason":{"type":"Single","setName":null,"maxSetVersion":null,"maxElectionId":null,"servers":{},"stale":false,"compatible":true,"compatibilityError":null,"logicalSessionTimeoutMinutes":null,"heartbeatFrequencyMS":10000,"localThresholdMS":15,"commonWireVersion":13}},"msg":"getaddrinfo ENOTFOUND mongo"}
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the MongoDB container recreation with the Docker Compose deployment configuration and compare the relevant depends_on behavior. Inspect the failures referenced in app/models/server/raw/OmnichannelQueue.ts:43 and app/livechat-enterprise/server/lib/LivechatEnterprise.js:269. Done means Rocket.Chat either recovers after MongoDB restarts or the required restart behavior is clearly established and documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker-compose, mongodb, node.js
- Domain
- databases, devops
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100