Error sending push to gateway.

Description:

Something strange with push gateway server.

Steps to reproduce:

1. Just chat with someone.

Expected behavior:

Actual behavior:

1. Push notify missing.
2. Sometimes push notify doubles.

Server Setup Information:

• Version of Rocket.Chat Server: 3.8.0
• Operating System: Ubuntu 18.04
• Deployment Method: Docker
• Number of Running Instances: 1
• DB Replicaset Oplog:
• NodeJS Version: 12.18.4
• MongoDB Version:4.0.21

Client Setup Information

• Desktop App or Browser Version: MacOS, Web, Android, iOS.

Additional context

Relevant logs:

`server.js:204 Push ➔ error Error sending push to gateway (0 try) -> Error: failed [502] Bad Gateway
at Object.exports.makeErrorByStatus (packages/http.js:177:10)
at Request._callback (packages/http.js:141:24)
at Request.self.callback (/app/bundle/programs/server/npm/node_modules/meteor/http/node_modules/request/request.js:185:22)
at Request.emit (events.js:315:20)
at Request.EventEmitter.emit (domain.js:483:12)
at Request. (/app/bundle/programs/server/npm/node_modules/meteor/http/node_modules/request/request.js:1161:10)
at Request.emit (events.js:315:20)
at Request.EventEmitter.emit (domain.js:483:12)
at IncomingMessage. (/app/bundle/programs/server/npm/node_modules/meteor/http/node_modules/request/request.js:1083:12)
at Object.onceWrapper (events.js:421:28)
at IncomingMessage.emit (events.js:327:22)
at IncomingMessage.EventEmitter.emit (domain.js:483:12)
at endReadableNT (_stream_readable.js:1220:12)
at processTicksAndRejections (internal/process/task_queues.js:84:21) {
response: {
statusCode: 502,
content: 'Bad Gateway',
headers: {
date: 'Wed, 09 Dec 2020 12:24:58 GMT',
'content-length': '11',
'content-type': 'text/plain; charset=utf-8',
connection: 'close'
},
data: null
}
}

server.js:204 Push ➔ error Error sending push to gateway (0 try) -> Error: ESOCKETTIMEDOUT
at ClientRequest. (/app/bundle/programs/server/npm/node_modules/meteor/http/node_modules/request/request.js:816:19)
at Object.onceWrapper (events.js:421:28)
at ClientRequest.emit (events.js:315:20)
at ClientRequest.EventEmitter.emit (domain.js:483:12)
at TLSSocket.emitRequestTimeout (_http_client.js:709:9)
at Object.onceWrapper (events.js:421:28)
at TLSSocket.emit (events.js:327:22)
at TLSSocket.EventEmitter.emit (domain.js:483:12)
at TLSSocket.Socket._onTimeout (net.js:482:8)
at listOnTimeout (internal/timers.js:549:17)
at processTimers (internal/timers.js:492:7) {
code: 'ESOCKETTIMEDOUT',
connect: false
}
server.js:204 Push ➔ error Error sending push to gateway (1 try) -> Error: ESOCKETTIMEDOUT
at ClientRequest. (/app/bundle/programs/server/npm/node_modules/meteor/http/node_modules/request/request.js:816:19)
at Object.onceWrapper (events.js:421:28)
at ClientRequest.emit (events.js:315:20)
at ClientRequest.EventEmitter.emit (domain.js:483:12)
at TLSSocket.emitRequestTimeout (_http_client.js:709:9)
at Object.onceWrapper (events.js:421:28)
at TLSSocket.emit (events.js:327:22)
at TLSSocket.EventEmitter.emit (domain.js:483:12)
at TLSSocket.Socket._onTimeout (net.js:482:8)
at listOnTimeout (internal/timers.js:549:17)
at processTimers (internal/timers.js:492:7) {
code: 'ESOCKETTIMEDOUT',
connect: false
}
server.js:204 Push ➔ error Error sending push to gateway (2 try) -> Error: ESOCKETTIMEDOUT
at ClientRequest. (/app/bundle/programs/server/npm/node_modules/meteor/http/node_modules/request/request.js:816:19)
at Object.onceWrapper (events.js:421:28)
at ClientRequest.emit (events.js:315:20)
at ClientRequest.EventEmitter.emit (domain.js:483:12)
at TLSSocket.emitRequestTimeout (_http_client.js:709:9)
at Object.onceWrapper (events.js:421:28)
at TLSSocket.emit (events.js:327:22)
at TLSSocket.EventEmitter.emit (domain.js:483:12)
at TLSSocket.Socket._onTimeout (net.js:482:8)
at listOnTimeout (internal/timers.js:549:17)
at processTimers (internal/timers.js:492:7) {
code: 'ESOCKETTIMEDOUT',
connect: false
}
`

Start at server.js:204, where the push gateway errors and retry attempts are logged, and inspect the push delivery entry point around that path. Reproduce the missing and duplicated notifications using the provided server setup and logs; done means push requests no longer fail or create duplicate notifications.