elastic / elastic/apm-agent-python
Custom Time-Series Metric Type for Python APM Agent
- Dominant language
- Python
- Stars
- 431
- Forks
- 239
- Avg merge
- 5d 10h
- Merged PRs (30d)
- 7
Description
**Is your feature request related to a problem? Please describe.**
I’m interested in implementing custom metrics like:
- Processing time for a request, which would be a float value (e.g., 60.0s).
- HTTP status code for a request, represented as an integer (e.g., 2XX, 5XX).
There are additional scenario-specific metrics I’d like to track as well(eg. token-usage etc.).
For my use case here I won't be able to use any of the available custom metric type like gauge, counter etc as counter would just keep a single value which is the current count which can be incremented or decremented, so I would not be able to keep multiple values/entries like for HTTP Code or processing times & the same goes for gauge which can be set to a certain value & when updated that values gets updated to a new value. Gauge can be useful for a metric like health status (viz Good & Bad) & counter would be useful for showing the current count of request processed or similar.
**Describe the solution you'd like**
Checking the source code of gauge and counter helps me understand that both utilize a
a variable with maybe int or float type.
After looking at the source code of all available metrics, I think none of them can be utilized to store and send list-like (timeseries/multiple values, like this [200,500,200,400,512] ) values. Histogram has self._counts which is of type list, but it saves frequency and not the actual value itself.
I was hoping that I could extend the BaseMetric class to create a metric which stores values in a list or dict like datastructure to be able to hold timeseries like data.
```
class ProcessingTime(BaseMetric):
__slots__ = BaseMetric.__slots__ + ("_data",)
def __init__(self, name, reset_on_collect=False) -> None:
"""
Creates a new ProcessingTime metric.
:param name: Label of the metric.
:param reset_on_collect: Flag to reset the values when collected.
"""
super(ProcessingTime, self).__init__(name, reset_on_collect=reset_on_collect)
self._data = {}
def add_video(self, video_id: str, processing_time: float) -> None:
"""
Adds processing time for a given video ID.
:param video_id: Alphanumeric ID of the video processed.
:param processing_time: Processing time in seconds.
"""
if not isinstance(video_id, str) or not video_id:
raise ValueError("Video ID must be a non-empty string.")
if not isinstance(processing_time, (int, float)):
raise ValueError("Processing time must be a numeric value.")
self._data[video_id] = float(processing_time)
```
How would I go about doing that ?
Also If this is something that would be helpful to the community maybe we can a add new metric type.
Also I would appreciate a more detailed guide or additional examples on how to effectively implement and use custom metrics in Python applications.
**Describe alternatives you've considered**
I don't want to have transactions & spans, but metrics so that I can plot a graph, currently I have achieved this by sending extra parameters in log & indexing them & then plotting them logger.info("Example message!", extra={"processing_time": 30.0}). But this method is slow & due to storage constraints I can't store historical data of longer periods(>5 days in my case).
**Additional context**
Please have a look at this [conversation](https://discuss.elastic.co/t/request-for-detailed-guide-on-implementing-custom-metrics-in-python-apm/366853) at elastic discussion forum which prompted this feature request.
Contributor guide
Assessment
This issue has not been assessed yet.