Long mongodb queries in 4.X versions

Description:

This issue is to @sampaiodiego and other back-end team members to consider.
It's about optimizing some back-end stuff like queries and indexes.
Previous similar issue was https://github.com/RocketChat/Rocket.Chat/issues/23052 and it was fixed in 3.18.1

But now in 4.X versions I see some long queries. Not so long, like it was before, but they definitely could be optimized by team.

Steps to reproduce:

Hard to tell. It I find them, I will update.

Expected behavior:

Optimized queries and optimized indexes

Actual behavior:

Lots of peaks on graph

Server Setup Information:

• Version of Rocket.Chat Server: 4.3.3
• Operating System: OL8.5
• Deployment Method: docker
• Number of Running Instances: 20
• DB Replicaset Oplog: Enabled
• NodeJS Version: 12.22.8
• MongoDB Version: 4.4.12

Client Setup Information

• Desktop App or Browser Version: 3.7.7
• Operating System: Windows 10

Additional context

I'm plan to collect such long queries in that thread to team investigation

Relevant logs:

First I see this long query. I see it is something in rocketchat.rocketchat_message collection and something with unread function. And 88332 in docsExamined it's pretty large amount to handle.

There is a almost 90k messages in that private chat

{"t":{"$date":"2022-02-15T11:22:27.947+03:00"},"s":"I",  "c":"WRITE",    "id":51803,   "ctx":"conn275","msg":"Slow query","attr":{"type":"update","ns":"rocketchat.rocketchat_message","command":{"q":{"rid":"hoYsQrcqNQRkHiyKz","unread":true,"ts":{"$lt":{"$date":"2022-02-11T12:57:02.685Z"}}},"u":{"$unset":{"unread":1},"$set":{"_updatedAt":{"$date":"2022-02-15T08:22:27.015Z"}}},"multi":true,"upsert":false},"planSummary":"IXSCAN { rid: 1, ts: 1, _updatedAt: 1 }","keysExamined":88332,"docsExamined":88332,"fromMultiPlanner":true,"replanned":true,"replanReason":"cached plan was less efficient than expected: expected trial execution to take 141 works but it took at least 1410 works","nMatched":0,"nModified":0,"numYields":443,"queryHash":"ECEA286E","planCacheKey":"818ECB28","locks":{"ParallelBatchWriterMode":{"acquireCount":{"r":444}},"ReplicationStateTransition":{"acquireCount":{"w":444}},"Global":{"acquireCount":{"w":444}},"Database":{"acquireCount":{"w":444}},"Collection":{"acquireCount":{"w":444}},"Mutex":{"acquireCount":{"r":1}}},"flowControl":{"acquireCount":444,"timeAcquiringMicros":196},"storage":{},"durationMillis":931}}

Start with the slow-query log for the rocketchat.rocketchat_message collection and the referenced issue 23052; the report provides no source file or test entry point. Gather the affected unread queries and indexes, then confirm that the documented query peaks and high docsExamined counts are reduced.