Extra metrics are being sent to DataDog when using Allow list with pattern matching enabled
- Dominant language
- Python
- Stars
- 46.9k
- Forks
- 17.8k
- Avg merge
- 2d 7h
- Merged PRs (30d)
- 484
Description
### Apache Airflow version
main (development)
### If "Other Airflow 2 version" selected, which one?
_No response_
### What happened?
with `AIRFLOW__METRICS__METRICS_USE_PATTERN_MATCH` set to `true` and `AIRFLOW__METRICS__METRICS_ALLOW_LIST` set to `.*get_astronauts.*` , I notice several other metrics being sent to datadog that don't match the pattern:

### What you think should happen instead?
Following metrics shouldn't be showing up on datadog when the allow list is `.*get_astronauts.*`:
1. `airflow.dag.task.duration.*`
2. `airflow.ti.*`
3. `datadog.*` (Not sure if we have any control to block these)
### How to reproduce
env variables:
AIRFLOW__METRICS__STATSD_DATADOG_ENABLED | true
-- | --
AIRFLOW__METRICS__STATSD_DATADOG_TAGS | testNo:C2
DATADOG_API_KEY | ● ● ● ● ● ● ●
AIRFLOW__METRICS__METRICS_USE_PATTERN_MATCH | true
AIRFLOW__METRICS__METRICS_ALLOW_LIST | `.*print_astronaut_craft.*`
Once you update your datadog key here, you will start seeing metrics in datadog-->metrics-->summary . use the tag `testNo:C2` to filter
### Operating System
Debian GNU/Linux 11 (bullseye)
### Versions of Apache Airflow Providers
_No response_
### Deployment
Astronomer
### Deployment details
_No response_
### Anything else?
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
Contributor guide
Research direction
Reproduce the issue with METRICS_USE_PATTERN_MATCH=true, the stated METRICS_ALLOW_LIST pattern, and the Datadog settings, then inspect the metrics allow-list and pattern-matching path. Use the testNo:C2 tag to review the resulting metrics in Datadog. Done means only metrics matching the configured pattern appear, with the scope of datadog.* behavior clarified.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- observability-sre
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100