elastic / elastic/apm-server

Set _doc_count to value of `{Transaction,Span}.RepresentativeCount`

Open
#11,131 1 comment 0 reactions 0 assignees View on GitHub
8.10-candidate
Dominant language
Go
Stars
1.3k
Forks
543
Avg merge
1d 18h
Merged PRs (30d)
109

Description

We've recently had some requests/questions around upscaling sampled transactions (and spans). For instance, see https://github.com/elastic/kibana/issues/160757. We've also had some internal users who have trouble building dashboards on sampled transactions. Previously this was less of an issue when we still persisted unsampled transactions.

What we'd like APM Server to do is set `_doc_count` for transactions and spans where `RepresentativeCount` is set (if not, it should be 1). The result of this is that Elasticsearch will automatically upscale bucketing aggregations by looking at the `_doc_count` meta field. The end result should be that all APM app _and_ custom dashboard visualisations should be automatically upscaled.

Showing more representative statistics when using transaction or spans is a plus in itself, but additionally it will:
- Allow users to drilldown into or build dashboards for fields that are not part of the aggregation key, e.g. transaction labels
- Allow the UI to switch to transactions or spans for smaller time ranges, allowing for more fidelity / responsiveness (e.g. change point detection on a 15m time range does not work if we are using 1m metrics, but it will work if we are using events)

We will keep the warning in the UI that sampled stats are being shown, with possibly some change in the wording - one example where the statistics are shown incorrectly is when the user filters on a trace id.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.