matrix-org / matrix-org/matrix-viewer

Add response time metrics (keep track when the page timeouts)

Open
#45 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

A-metrics A-tracing T-Enhancement
Dominant language
JavaScript
Stars
84
Forks
10
PR merge metrics
No merged PRs in 30d

Description

Add metric when the page times out. Record the Matrix API that is still running and the duration.

Things to record in each event:

- Response status code we ended up sending
- Total time spent on the server rendering the request (this will just end up being the timeout configured)
- Homeserver
- Potential for high cardinality although the list will be limited to a known set of homeservers (https://github.com/matrix-org/matrix-public-archive/issues/5)
- Room ID
- Since this has a very high cardinality (lots of possible values), we might not be able to index this but would be good to have on each metric event to inspect.
- These extra details are nice if we want to investigate why a particular room/homeserver combo is timing out
- Matrix API endpoint path that is still running when we timed out (like `/join`, `/messages`)
- Is this useful? Would be nice to know where most requests get stuck at

We can also send a success metric and response time to compare against how many requests we're failing to serve vs total traffic.

### Dev notes

We probably just need to add something like [`prom-client`](https://github.com/siimon/prom-client), expose a Prometheus `/metrics` scrape endpoint that serves `await register.metrics()`, then add a scrape annotation to the K8s service (which is still being finalized)

Adjacent: Here is an example middleware from the Gitter webapp that logs and metrics when a request is pending for more than 60 seconds, https://gitlab.com/gitterHQ/webapp/-/blob/676fadc3693260c8c51f448a0ca4c3e180d1b4a2/server/web/middlewares/pending-request.js#L50-84

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the proposed prom-client integration, the Prometheus /metrics endpoint using await register.metrics(), and the Kubernetes service configuration. Compare the pending-request.js example linked in the issue, then define metrics for timeout and successful requests with the listed response, duration, homeserver, room, and endpoint details; done means the endpoint is scrapeable and the request outcomes are recorded.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, kubernetes, prometheus
Domain
backend, infrastructure, observability
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.