Add metrics and RT monitoring
- Dominant language
- Go
- Stars
- 12.4k
- Forks
- 510
- PR merge metrics
- No merged PRs in 30d
Description
Hi,
I would like to add metrics to allow RT monitoring, to allow automatic & manual test on remote servers verify that the traffic is actually passing through toxiproxy, verify that they stop, see clients connects and disconnects and so on.
I've been experimenting with the code and got something working, and would like to consult before I create a PR.
The functionality I added does as follows:
- added a /metrics endpoint that returns a map that describes the number of total messages per proxy.
- added a /events endpoint that returns a list of messages (proxy, client, upstream, event type and timestamp) for up to 20 minutes ago. A token is returned with each response that can be used in the next request to receive only unseen events.
The way I did it is by creating a metrics module, with a RegisterEvent method. In proxy.go I report when client connects, in link.go I report that a clients disconnects.
Things got trickier when trying to report messages. I used the fact that noop toxic is always present to report from there. The problem is that noop toxic doesn't have any context to its work, so I had to add ProxyName and Upstream to NoopProxy, and had to pass this information on to and from where it is created, namely Link and ToxicStub.
Does this make sense? In addition, I wasn't able to tell in NoopProxy which client sent a specific message. Any idea how can I do that?
Thanks!
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the proposed metrics module and the entry points named in proxy.go, link.go, NoopProxy, Link, and ToxicStub. Review the open pull request and the /metrics and /events behavior described in this issue. Done would require an agreed design for connection, message, and client attribution plus tests that verify the monitoring data.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- backend, observability
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100