elastic / elastic/apm-aggregation

Better estimate for aggregated events count

Open
#118 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
2
Forks
11
PR merge metrics
No merged PRs in 30d

Description

## Motivation

The current algorithm calculates events total for the whole `CombinedMetrics`. This approach works well as long as we need to merge forward. However, when we need to split or partition `CombinedMetrics` then we can no longer estimate the events accurately.

## Solution

Track events total for each aggregate (including overflow) rather than for the whole `CombinedMetrics`. This will require more storage but we will be able to track accurate events total for splits or partitions. Extra storage cost based on current limits will be:

`(max_span_groups (40_000) + max_txn_groups (40_000) + max_svc_txn_groups (8_000) + overflows_per_svc (8_000) + overflow_global (1)) * (events_total_size_tracked_by_float64) 8 bytes = ~770KB`

The extra storage will be used by pebble.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.