awslabs / awslabs/aws-embedded-metrics-python
Allow filtering of properties and suppression of logging
- Dominant language
- Python
- Stars
- 231
- Forks
- 44
- PR merge metrics
- No merged PRs in 30d
Description
The idea proposed here roughly corresponds to the concept of "logging levels", but aims to be separate from a strict logging levels paradigm. Based on a discussion with @jaredcnance.
I want the ability to add large numbers of properties to the `MetricsLogger` in my code, far more than would be economical to output for execution in production. Then, based on some criteria, like sampling, an error occurring, some sort of "debug" flag, etc., I want to filter down the properties that get flushed. Additionally, I may want to suppress the output entirely.
# Proposed solution
1. Methods on `MetricsLogger` like `set_property()` and `put_metric()` will add an optional keyword argument called `tag` that takes a string or list of strings.
2. `MetricsLogger` will add two properties, `pre_filter` and `filter`, that can be set to callables.
1. `pre_filter` is called when a method is called with the `tag` keyword argument. It takes as input the context object, the tags, and the inputs to the method (e.g., property name and value), and returns a boolean indicating whether to actually perform the action or to silently ignore the input
For convenience, `pre_filter` can be set to a list of tags, equivalent to a callable that matches the given tags against any in the list.
2. `filter` is called when the output is being collated. It takes as input the context object and any uncaught error that the metric scope (if there is one) has handled. It returns a boolean or a list of tags. If it returns a boolean, and the value is False, the logging is suppressed. If the boolean is True, all properties/metrics will be included in the output. If it returns a list of tags, the output is filtered to include untagged properties/metrics and any tagged properties/metrics that have at least one tag matching the list.
For convenience, `filter` can be set to a boolean or a list of tags, equivalent to a callable that returns that value.
3. If no `pre_filter` is set, tagged properties/metrics will _not_ be dropped.
4. If no `filter` is set, all tagged properties/metrics will be included in the output.
The reason to call it `filter` rather than `post_filter` is to encourage its use over `pre_filter`, which should generally only be used if memory/GC usage is a concern. The delayed action of `filter` allows decisions on properties to be made with context that may come in later (for example, increased logging verbosity in case of an error).
# Use cases
## 1. "Debug" logging
To simply have a set of verbose properties, I might do the following:
```python
metrics.filter = ["debug"] if os.environ.get("DEBUG", "false") in ["1", "true"] else []
# ...
metrics.set_property("some_debug_property", some_debug_value)
```
## 2. Verbose logging in case of an error
```python
global is_error = False
metrics.filter = lambda context, uncaught_err: ["only_on_error"] if is_error else []
try:
metrics.set_property("verbose_but_relevant_for_an_error", value, tags=["only_on_error"])
# ...
except:
is_error = True
```
## 3. Sampling
```python
global sampled = bool(random.random < float(os.environ["SAMPLING_RATE"]))
metrics.filter = sampled
```
## 4. Combined uses
```python
global is_error = False
global sampled = bool(random.random < float(os.environ["SAMPLING_RATE"]))
def filter(context, uncaught_err):
tags = []
if is_error:
tags.append("on_error")
if sampled:
tags.append("on_sample")
debug = os.environ.get("DEBUG", "false") in ["1", "true"]
if verbose:
tags.append("debug")
return tags
metrics.filter = filter
metrics.set_property("foo", "foo value", tags=["debug", "on_error"])
metrics.set_property("bar", "bar value", tags=["on_sample", "on_error"])
```
# Questions
- Should a property/metric have multiple tags or only a single one? I can see value in multiple tags but maybe it's too much complexity.
Contributor guide
Assessment
This issue has not been assessed yet.