element-hq / element-hq/synapse
Synapse creates metrics with name collisions
- Dominant language
- Python
- Stars
- 4.6k
- Forks
- 600
- Avg merge
- 5d 22h
- Merged PRs (30d)
- 51
Description
### Problem
There are some metrics which are created in Synapse that use the same metric name. This leads to the metrics being overridden by the latest metric to be registered https://github.com/element-hq/synapse/blob/18232871d0773108a77bb22f31edb5023119886c/synapse/util/metrics.py#L313-L337
So far there are 2 known instances of this happening:
- We instantiate multiple `SyncRestServlet`, one per listener, and in the `__init__` we setup a new `LruCache`.
- We instantiate multiple `ApplicationService` (one per configured application service) which use the `@cached` decorator on some methods.
### Background
This is a long-standing issue and these logs are harmless. Previously, metrics were just being clobbered silently. Now with https://github.com/element-hq/synapse/pull/18828, we have some error logs to call this out.
Contributor guide
Research direction
Start with synapse/util/metrics.py at the linked registration logic, then trace the metric creation paths for SyncRestServlet and ApplicationService. Determine how their per-instance caches produce duplicate names and define a way to keep registrations distinct without the harmless error logs; verify both reported cases.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- observability
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100