awslabs / awslabs/aws-embedded-metrics-python

Allow filtering of properties and suppression of logging

Open
#57 0 comments 2 reactions 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
231
Forks
44
PR merge metrics
No merged PRs in 30d

Description

The idea proposed here roughly corresponds to the concept of "logging levels", but aims to be separate from a strict logging levels paradigm. Based on a discussion with @jaredcnance.

I want the ability to add large numbers of properties to the `MetricsLogger` in my code, far more than would be economical to output for execution in production. Then, based on some criteria, like sampling, an error occurring, some sort of "debug" flag, etc., I want to filter down the properties that get flushed. Additionally, I may want to suppress the output entirely.

# Proposed solution
1. Methods on `MetricsLogger` like `set_property()` and `put_metric()` will add an optional keyword argument called `tag` that takes a string or list of strings.
2. `MetricsLogger` will add two properties, `pre_filter` and `filter`, that can be set to callables.
1. `pre_filter` is called when a method is called with the `tag` keyword argument. It takes as input the context object, the tags, and the inputs to the method (e.g., property name and value), and returns a boolean indicating whether to actually perform the action or to silently ignore the input
For convenience, `pre_filter` can be set to a list of tags, equivalent to a callable that matches the given tags against any in the list.
2. `filter` is called when the output is being collated. It takes as input the context object and any uncaught error that the metric scope (if there is one) has handled. It returns a boolean or a list of tags. If it returns a boolean, and the value is False, the logging is suppressed. If the boolean is True, all properties/metrics will be included in the output. If it returns a list of tags, the output is filtered to include untagged properties/metrics and any tagged properties/metrics that have at least one tag matching the list.
For convenience, `filter` can be set to a boolean or a list of tags, equivalent to a callable that returns that value.
3. If no `pre_filter` is set, tagged properties/metrics will _not_ be dropped.
4. If no `filter` is set, all tagged properties/metrics will be included in the output.

The reason to call it `filter` rather than `post_filter` is to encourage its use over `pre_filter`, which should generally only be used if memory/GC usage is a concern. The delayed action of `filter` allows decisions on properties to be made with context that may come in later (for example, increased logging verbosity in case of an error).

# Use cases
## 1. "Debug" logging
To simply have a set of verbose properties, I might do the following:
```python
metrics.filter = ["debug"] if os.environ.get("DEBUG", "false") in ["1", "true"] else []
# ...
metrics.set_property("some_debug_property", some_debug_value)
```

## 2. Verbose logging in case of an error
```python
global is_error = False
metrics.filter = lambda context, uncaught_err: ["only_on_error"] if is_error else []

try:
metrics.set_property("verbose_but_relevant_for_an_error", value, tags=["only_on_error"])
# ...
except:
is_error = True
```

## 3. Sampling
```python
global sampled = bool(random.random < float(os.environ["SAMPLING_RATE"]))
metrics.filter = sampled
```

## 4. Combined uses
```python
global is_error = False
global sampled = bool(random.random < float(os.environ["SAMPLING_RATE"]))
def filter(context, uncaught_err):
tags = []
if is_error:
tags.append("on_error")
if sampled:
tags.append("on_sample")
debug = os.environ.get("DEBUG", "false") in ["1", "true"]
if verbose:
tags.append("debug")
return tags

metrics.filter = filter

metrics.set_property("foo", "foo value", tags=["debug", "on_error"])

metrics.set_property("bar", "bar value", tags=["on_sample", "on_error"])

```

# Questions
- Should a property/metric have multiple tags or only a single one? I can see value in multiple tags but maybe it's too much complexity.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.