Perfomance issues when upgrading from 6.3.3 to 6.4.2

Description:

Version 6.4.2 loads servers heavily. In the grafana we see many requests rate that were not that before.
Docker instances are restarting with OOM.

Steps to reproduce:

1. Update your RC from 6.3.3 to 6.4.2
2. Look at the production Graphаna
3. See docker logs

Expected behavior:

Performance and stability are no worse than the previous version 6.3.3

Actual behavior:

High CPU load on 5 servers, instance restarts

Server Setup Information:

• Version of Rocket.Chat Server: 6.4.2 Enterprise ~6000 active sessions
• Operating System: Oracle Linux Server release 8.6
• Deployment Method: docker
• Number of Running Instances: 16
• DB Replicaset Oplog: yes
• NodeJS Version: v14.21.3
• MongoDB Version: 5.0.15 / wiredTiger (oplog Enabled)

Client Setup Information

• Desktop App or Browser Version: Chrome 117 Rocket.Chat Electron 3.9.9, 3.9.6, 3.8.17
• Operating System: Windows, Mac, Linux

Additional context

there are a lot of calls to these API methods:

action.buttons
commands.list

In a few hours without disconnections, I saw in my Google Chrome(117) - 50 calling of API method commands.list

RC logs chart with API section for one hour:

before update

after update

There is also a dependence of this issue #30717, but it is not significant. We found out that there are not many such clients in our network.

Relevant logs:

Start by comparing the commands.list and action.buttons request rates and Docker logs between versions 6.3.3 and 6.4.2, using the Grafana charts and the possible relationship to issue #30717. Trace the server entry points handling these API methods and reproduce the upgrade scenario; done means request volume, CPU usage, and OOM-related restarts are no worse than in 6.3.3.