elastic / elastic/apm-agent-python

Custom Time-Series Metric Type for Python APM Agent

Open
#2,139 0 comments 0 reactions 0 assignees View on GitHub
agent-python community triage
Dominant language
Python
Stars
431
Forks
239
Avg merge
5d 10h
Merged PRs (30d)
7

Description

**Is your feature request related to a problem? Please describe.**

I’m interested in implementing custom metrics like:

- Processing time for a request, which would be a float value (e.g., 60.0s).
- HTTP status code for a request, represented as an integer (e.g., 2XX, 5XX).

There are additional scenario-specific metrics I’d like to track as well(eg. token-usage etc.).

For my use case here I won't be able to use any of the available custom metric type like gauge, counter etc as counter would just keep a single value which is the current count which can be incremented or decremented, so I would not be able to keep multiple values/entries like for HTTP Code or processing times & the same goes for gauge which can be set to a certain value & when updated that values gets updated to a new value. Gauge can be useful for a metric like health status (viz Good & Bad) & counter would be useful for showing the current count of request processed or similar.

**Describe the solution you'd like**

Checking the source code of gauge and counter helps me understand that both utilize a
a variable with maybe int or float type.
After looking at the source code of all available metrics, I think none of them can be utilized to store and send list-like (timeseries/multiple values, like this [200,500,200,400,512] ) values. Histogram has self._counts which is of type list, but it saves frequency and not the actual value itself.

I was hoping that I could extend the BaseMetric class to create a metric which stores values in a list or dict like datastructure to be able to hold timeseries like data.
```
class ProcessingTime(BaseMetric):
__slots__ = BaseMetric.__slots__ + ("_data",)

def __init__(self, name, reset_on_collect=False) -> None:
"""
Creates a new ProcessingTime metric.

:param name: Label of the metric.
:param reset_on_collect: Flag to reset the values when collected.
"""
super(ProcessingTime, self).__init__(name, reset_on_collect=reset_on_collect)
self._data = {}

def add_video(self, video_id: str, processing_time: float) -> None:
"""
Adds processing time for a given video ID.

:param video_id: Alphanumeric ID of the video processed.
:param processing_time: Processing time in seconds.
"""
if not isinstance(video_id, str) or not video_id:
raise ValueError("Video ID must be a non-empty string.")
if not isinstance(processing_time, (int, float)):
raise ValueError("Processing time must be a numeric value.")
self._data[video_id] = float(processing_time)
```

How would I go about doing that ?
Also If this is something that would be helpful to the community maybe we can a add new metric type.

Also I would appreciate a more detailed guide or additional examples on how to effectively implement and use custom metrics in Python applications.

**Describe alternatives you've considered**

I don't want to have transactions & spans, but metrics so that I can plot a graph, currently I have achieved this by sending extra parameters in log & indexing them & then plotting them logger.info("Example message!", extra={"processing_time": 30.0}). But this method is slow & due to storage constraints I can't store historical data of longer periods(>5 days in my case).

**Additional context**

Please have a look at this [conversation](https://discuss.elastic.co/t/request-for-detailed-guide-on-implementing-custom-metrics-in-python-apm/366853) at elastic discussion forum which prompted this feature request.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.