apache / apache/airflow

Cannot send custom statsd metrics in a plugin

Open
#69,172 0 comments 0 reactions 0 assignees View on GitHub
area:core area:metrics area:plugins kind:bug needs-triage
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 9h
Merged PRs (30d)
472

Description

### Under which category would you file this issue?

Airflow Core

### Apache Airflow version

3.2.1

### What happened and how to reproduce it?

A while back, I was using a plugin to send custom statsd metrics after a fail or success dag run :

```
from airflow.stats import Stats
metric_success = f"brs_dag_last_5_runs_states.success.{dag_id}"
metric_failed = f"brs_dag_last_5_runs_states.failed.{dag_id}"
Stats.gauge(metric_success, success_count)
Stats.gauge(metric_failed, failed_count)
```

And it was working fine. But, it's not working anymore : no metric is sent to my statsd_exporter.

Here is my configuration :

```
AIRFLOW__METRICS__STATSD_HOST: 10.2.218.29
AIRFLOW__METRICS__STATSD_INFLUXDB_ENABLED: 'True'
AIRFLOW__METRICS__STATSD_ON: 'True'
AIRFLOW__METRICS__STATSD_PORT: '9125'
AIRFLOW__METRICS__STATSD_PREFIX: prod
```

Please note : I don't have any issue with the "native" statsd metrics from Airflow : my stats_exporter is collecting them.

After some debug of the statsd object inside my plugin file :

```
log.info("Stats object: %r", Stats)

for attr in ("_instance", "_logger", "factory"):
log.info("Stats.%s = %r", attr, getattr(Stats, attr, None))
```

```
2026-06-30T09:03:49.079501Z [info ] Stats object: [airflow.utils.log.logging_mixin.LoggingMixin] loc=brs_tools.py:90
2026-06-30T09:03:49.079666Z [info ] Stats._instance = None [airflow.utils.log.logging_mixin.LoggingMixin] loc=brs_tools.py:92
2026-06-30T09:03:49.079812Z [info ] Stats._logger = None [airflow.utils.log.logging_mixin.LoggingMixin] loc=brs_tools.py:92
2026-06-30T09:03:49.079946Z [info ] Stats.factory = [airflow.utils.log.logging_mixin.LoggingMixin] loc=brs_tools.py:92
2026-06-30T09:03:49.080074Z [info ] Stats.gauge returned: None [airflow.utils.log.logging_mixin.LoggingMixin] loc=brs_tools.py:96
```

### What you think should happen instead?

The metrics inside my plugin should be sent to statsd.

### Operating System

Ubuntu 24.04 + Docker

### Deployment

Docker-Compose

### Apache Airflow Provider(s)

_No response_

### Versions of Apache Airflow Providers

apache-airflow-providers-amazon==9.6.1
apache-airflow-providers-apache-cassandra==3.9.4
apache-airflow-providers-apache-hdfs==4.11.5
apache-airflow-providers-apache-kafka==1.13.3
apache-airflow-providers-apache-spark==6.0.1
apache-airflow-providers-celery==3.18.0
apache-airflow-providers-cncf-kubernetes==10.16.1
apache-airflow-providers-common-compat==1.14.3
apache-airflow-providers-common-io==1.7.2
apache-airflow-providers-common-messaging==2.0.3
apache-airflow-providers-common-sql==1.35.0
apache-airflow-providers-docker==4.5.5
apache-airflow-providers-elasticsearch==6.5.3
apache-airflow-providers-fab==3.6.2
apache-airflow-providers-ftp==3.14.3
apache-airflow-providers-git==0.3.1
apache-airflow-providers-google==21.2.0
apache-airflow-providers-grpc==3.9.4
apache-airflow-providers-hashicorp==4.6.0
apache-airflow-providers-http==6.0.2
apache-airflow-providers-microsoft-azure==13.1.2
apache-airflow-providers-microsoft-mssql==4.5.2
apache-airflow-providers-mysql==6.5.2
apache-airflow-providers-odbc==4.12.2
apache-airflow-providers-openlineage==2.15.0
apache-airflow-providers-oracle==4.5.3
apache-airflow-providers-postgres==6.6.3
apache-airflow-providers-redis==4.4.4
apache-airflow-providers-sendgrid==4.2.3
apache-airflow-providers-sftp==5.7.4
apache-airflow-providers-slack==9.10.0
apache-airflow-providers-smtp==3.0.0
apache-airflow-providers-snowflake==6.12.2
apache-airflow-providers-ssh==5.0.1
apache-airflow-providers-standard==1.12.3

### Official Helm Chart version

Not Applicable

### Kubernetes Version

_No response_

### Helm Chart configuration

_No response_

### Docker Image customizations

_No response_

### Anything else?

For the time being, I'm sending "manually" the metrics :

```
from functools import lru_cache

from airflow.configuration import conf
from statsd import StatsClient

@lru_cache
def get_statsd_client() -> StatsClient:
return StatsClient(
host=conf.get("metrics", "statsd_host"),
port=conf.getint("metrics", "statsd_port"),
prefix=conf.get("metrics", "statsd_prefix", fallback=None),
)

def statsd_gauge(metric_name: str, value: int) -> None:
get_statsd_client().gauge(metric_name, value)

statsd_gauge(metric_success, success_count)
statsd_gauge(metric_failed, failed_count)
```

### Are you willing to submit PR?

- [ ] Yes I am willing to submit a PR!

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

Contributor guide

Open the contributing guide

Research direction

Start by tracing airflow.stats.Stats and the airflow.sdk._shared.observability.metrics.stats module shown in the logs, then compare their initialization with the native Airflow metrics path. Reproduce the plugin call using the supplied statsd configuration and verify that custom gauge calls reach statsd_exporter, rather than returning through NoStatsLogger.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
observability
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.