Set _doc_count to value of `{Transaction,Span}.RepresentativeCount`
- Dominant language
- Go
- Stars
- 1.3k
- Forks
- 543
- Avg merge
- 1d 18h
- Merged PRs (30d)
- 109
Description
We've recently had some requests/questions around upscaling sampled transactions (and spans). For instance, see https://github.com/elastic/kibana/issues/160757. We've also had some internal users who have trouble building dashboards on sampled transactions. Previously this was less of an issue when we still persisted unsampled transactions.
What we'd like APM Server to do is set `_doc_count` for transactions and spans where `RepresentativeCount` is set (if not, it should be 1). The result of this is that Elasticsearch will automatically upscale bucketing aggregations by looking at the `_doc_count` meta field. The end result should be that all APM app _and_ custom dashboard visualisations should be automatically upscaled.
Showing more representative statistics when using transaction or spans is a plus in itself, but additionally it will:
- Allow users to drilldown into or build dashboards for fields that are not part of the aggregation key, e.g. transaction labels
- Allow the UI to switch to transactions or spans for smaller time ranges, allowing for more fidelity / responsiveness (e.g. change point detection on a 15m time range does not work if we are using 1m metrics, but it will work if we are using events)
We will keep the warning in the UI that sampled stats are being shown, with possibly some change in the wording - one example where the statistics are shown incorrectly is when the user filters on a trace id.
Contributor guide
Assessment
This issue has not been assessed yet.