RocketChat / RocketChat/Rocket.Chat
[Bug] Rocket.Chat metrics shows only some aggregated values since 4.x version
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 46.1k
- Forks
- 13.9k
- Avg merge
- 3d 3h
- Merged PRs (30d)
- 130
Description
Description:
On current 4.3.1 version we have very strange Rocket.Chat metrics graphs - they not fluent, but too sharp.
Looks like instances send only aggregated in some time period values.
We have alerts about to heavy change of connected users on a regular basis, which not good and it wasn't like that on 3.x versions.
Looks like update of graphs appears only at 13 minutes of every hour.
Steps to reproduce:
- Update to 4.x ROcket.Chat version
- Check Rocket.Chat graphs like
Users Presence - That graphs not shows current values, and updated once at hour (at 13 minutes on our prod instance)
Expected behavior:
Smooth, fluent graphs for Rocket.Chat metrics with current values at any moment of time.
Actual behavior:
Prometheus settings

User Presence

Messages Sent Total

Total Rooms

But some graphs are still normal (smooth)
Notifications

OpLog

Event loop lag

Server Setup Information:
- Version of Rocket.Chat Server: 4.3.1
- Operating System: OL8.5
- Deployment Method: docker
- Number of Running Instances: 20
- DB Replicaset Oplog: Enabled
- NodeJS Version: 12.22.8
- MongoDB Version: 4.4.11
Client Setup Information
- Desktop App or Browser Version: 3.7.6
- Operating System: Windows 10
Additional context
Users presence graph very critical for us - we have set alert on heavy changes of connected users, and now that alert triggers all the time (once per hour on morning and evening - when users come to work and go back to home).
Relevant logs:
No
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the metric update behavior on Rocket.Chat 4.3.1 with the reported Prometheus settings and Docker deployment. Compare the Users Presence, Messages Sent Total, and Total Rooms graphs with the Notifications, OpLog, and event loop lag metrics. Done means affected metrics update smoothly with current values instead of appearing only once per hour.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- prometheus, typescript
- Domain
- backend, observability
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100