Metric labels rewriting for statsd sink
- Dominant language
- Go
- Stars
- 30.1k
- Forks
- 4.6k
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 43
Description
#### Feature Description
Metric labels are passed as-is to go-metrics where they are handled differently depending on the implementation. In the case of the statsd reporter, labels are joined into one string with a dot as a separator creating a hierarchical type metric in Graphite. This works well for most of the cases but it gets complicated when a label itself contains a dot, e.g. a hostname or an IP address.
It would be great to have an option to sanitize or rewrite labels.
#### Use Case(s)
Labels like this can be observed in metrics like [`consul.client.rpc.failed`](https://github.com/hashicorp/consul/blob/abce1f0eeead917adef3caa3a4d8e3e1f227255c/agent/consul/client.go#L335) and [`consul.catalog.service.query-tag`](https://github.com/hashicorp/consul/blob/a7f69b615a192892d9c3c60626baa32c7d7078b5/agent/consul/catalog_endpoint.go#L475).
Such behavior makes it hard to properly aggregate metrics, especially when single metric gets split into multiple metrics with different levels in hierarchy due to label length, e.g.
`consul.client.rpc.failed.consul1.local.count`
`consul.client.rpc.failed.consul2.local.count`
`consul.client.rpc.failed.consul1.cloud.azure.count`
which creates the following hierarchy
```
consul
- client
- rpc
- failed
- consul1
- local
- count
- cloud
- azure
- count
- consul2
- local
- count
```
As you can see `count` in `local` and `cloud.azure` are on a different level.
If dots in labels could be rewritten to some other character, like an underscore `_`, the example metric above would have the same depth of hierarchy.
`consul.client.rpc.failed.consul1_local.count`
`consul.client.rpc.failed.consul1_cloud_azure.count`
which results in this hierarchy:
```
consul
- client
- rpc
- failed
- consul1_local
- count
- consul1_cloud_azure
- count
- consul2_local
- count
```
Contributor guide
Assessment
This issue has not been assessed yet.