apache / apache/airflow

Document Airflow Metrics Optimal Values

Open
#14,583 3 comments 0 reactions 0 assignees View on GitHub
area:metrics kind:documentation
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 9h
Merged PRs (30d)
472

Description

**Description**

Users of Airflow appreciate the [metrics](https://airflow.apache.org/docs/apache-airflow/stable/logging-monitoring/metrics.html) that Airflow emits, however are often confused as to what values indicate a healthy system versus a system that might be exhibiting problems.

This feature request is to add value ranges to the metrics documentation as to what values are appropriate, as well as troubleshooting information as to how to address when those metrics fall outside recommended values.

**Use case / motivation**

For example, looking at `dag_processing.total_parse_time` under [Gauges](https://airflow.apache.org/docs/apache-airflow/stable/logging-monitoring/metrics.html?highlight=metrics#gauges), what are appropriate values here? As a user, how would I know that my scheduler is under-powered, or that I have too many/too complex DAGs, or that some other issue exists?

**Are you willing to submit a PR?**

Yes

**Related Issues**

N/A

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.